ERROR TaskSchedulerImpl: Ignoring update with state FINISHED for TID 870 because its task set is gone (this is likely the result of receiving duplicate task finished status updates) or its executor has been marked as failed解决方案
加长executor的心跳时间
--confspark.network.timeout=10000000 --conf spark.executor.heartbeatInterval=10000000参考
在https://mallikarjuna_g.gitbooks.io/spark/content/spark-taskschedulerimpl.html里有提到：If the TaskSetManager for tid could not be found (in taskIdToTaskSetManager registry), you should see the following ERROR message in the logs:
ERROR Ignoring update with state [state] for TID [tid] because its task set is gone (this is likely the result of receiving duplicate task finished status updates)
