实现spark streaming demo时,代码:
1 public static void main (String[] args) { 2 3 4 SparkConf conf = new SparkConf().setAppName("Spark_Streaming").setMaster("local"); 5 JavaSparkContext sc = new JavaSparkContext(conf); 6 JavaStreamingContext jssc = new JavaStreamingContext(sc, new Duration(2)); 7 JavaSQLContext sqlCtx = new JavaSQLContext(sc); 8 9 10 String[] filters = new String[] {"soc"}; 11 12 JavaReceiverInputDStream<Status> receiverStream = TwitterUtils.createStream(jssc,filters); 13 14 15 16 jssc.start(); 17 jssc.awaitTermination(); 18 19 }
运行以上代码会出现异常:No output streams registered, so nothing to execute
原因是当发生输出操作的时候,会触发流计算操作,当在 DStream上没有输出操作或者计算时,通常需要执行某些操作,比如以任意方法:
1 print() 2 foreachRDD(func) 3 saveAsObjectFiles(prefix, [suffix]) 4 saveAsTextFiles(prefix, [suffix]) 5 saveAsHadoopFiles(prefix, [suffix])
可参考:http://spark.apache.org/docs/latest/streaming-programming-guide.html#output-operations
时间: 2024-10-09 17:27:35