在Google上搜索后,我在最新的spark-cassandra-
connector中喜欢了这个问题。参数
spark.cassandra.input.split.size_in_mbDefault value是64
MB,在代码中被解释为64个字节。所以尝试
spark.cassandra.input.split.size_in_mb = 64 * 1024 *1024 = 67108864
听到就是一个例子:
public static void main(String[] args) { SparkConf conf = new SparkConf(true).setMaster("local[4]") .setAppName("App_Name") .set("spark.cassandra.connection.host", "127.0.0.1") .set("spark.cassandra.input.split.size_in_mb","67108864"); JavaSparkContext sc = new JavaSparkContext(conf); JavaRDD<Demo_Bean> empRDD = javaFunctions(sc).cassandraTable("dev", "demo"); System.out.println("Row Count"+empRDD.count());}


