11
项目遇到的问题之Spark
source link: https://zjinc36.github.io/2020/08/29/2020-20200829-%E9%A1%B9%E7%9B%AE%E9%81%87%E5%88%B0%E7%9A%84%E9%97%AE%E9%A2%98%E4%B9%8BSpark/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Aug 29, 2020BigData
SparkStreaming优雅关闭
如何优雅的关闭SparkStreaming任务（将写好的代码打包，Spark-Submit） => Kill -9 xxx ?
开启另外一个线程每5秒监听HDFS上一个文件是否存在。如果检测到存在，调用ssc.stop()方法关闭SparkStreaming任务（当你要关闭任务时，可以创建你自定义监控的文件目录）
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK