执行distcp任务失败,获取yarn任务日志查看task任务超时失败
查看日志中超时异常
Diagnostics report from attempt_1647496740778_1063_m_000019_3: AttemptID:attempt_1647496740778_1063_m_000019_3 task timeout set: 1800000s, taskTimedOut: false; task stuck timeout set: 1200s, taskStuck: true
attempt_1647496740778_1063_m_000019_3 TaskAttempt Transitioned from RUNNING to FAIL_CONTAINER_CLEANUP
attempt_1647496740778_1063_m_000019_3 TaskAttempt Transitioned from FAIL_CONTAINER_CLEANUP to FAIL_TASK_CLEANUP
attempt_1647496740778_1063_m_000019_3 transitioned from state FAIL_TASK_CLEANUP to FAILED
解决办法:
加大超时参数
设置-Dmapreduce.task.timeout=3600000
mapreduce.task.stuck.timeout-ms=3600000
或者修改mapred-site.xml中mapreduce.task.stuck.timeout-ms配置



