栏目分类:
子分类:
返回
名师互学网用户登录
快速导航关闭
当前搜索
当前分类
子分类
实用工具
热门搜索
名师互学网 > IT > 前沿技术 > 大数据 > 大数据系统

Failure task Diagnostics 构建 cube ,reduce 内存分配不足

Failure task Diagnostics 构建 cube ,reduce 内存分配不足

项目场景:

kylin 构建 cube 时报错,原因是 reduce 内存分配不足

问题描述:
Failure task Diagnostics:
[2021-11-25 18:53:22.778]Container [pid=4143,containerID=container_1637894495535_0007_01_000013] is running 285661696B beyond the 'VIRTUAL' memory limit. Current usage: 124.1 MB of 1 GB physical memory used; 2.4 GB of 2.1 GB virtual memory used. Killing container.
Dump of the process-tree for container_1637894495535_0007_01_000013 :
	|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
	|- 4143 4142 4143 4143 (bash) 0 0 9793536 290 /bin/bash -c /opt/module/jdk1.8.0_212/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN   -Xmx820m -Djava.io.tmpdir=/opt/module/hadoop-3.1.3/data/nm-local-dir/usercache/hadoop/appcache/application_1637894495535_0007/container_1637894495535_0007_01_000013/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/opt/module/hadoop-3.1.3/logs/userlogs/application_1637894495535_0007/container_1637894495535_0007_01_000013 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog -Dyarn.app.mapreduce.shuffle.logger=INFO,shuffleCLA -Dyarn.app.mapreduce.shuffle.logfile=syslog.shuffle -Dyarn.app.mapreduce.shuffle.log.filesize=0 -Dyarn.app.mapreduce.shuffle.log.backups=0 org.apache.hadoop.mapred.YarnChild 192.168.10.103 36822 attempt_1637894495535_0007_r_000000_1 13 1>/opt/module/hadoop-3.1.3/logs/userlogs/application_1637894495535_0007/container_1637894495535_0007_01_000013/stdout 2>/opt/module/hadoop-3.1.3/logs/userlogs/application_1637894495535_0007/container_1637894495535_0007_01_000013/stderr  
	|- 4160 4143 4143 4143 (java) 411 18 2530725888 31474 /opt/module/jdk1.8.0_212/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx820m -Djava.io.tmpdir=/opt/module/hadoop-3.1.3/data/nm-local-dir/usercache/hadoop/appcache/application_1637894495535_0007/container_1637894495535_0007_01_000013/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/opt/module/hadoop-3.1.3/logs/userlogs/application_1637894495535_0007/container_1637894495535_0007_01_000013 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog -Dyarn.app.mapreduce.shuffle.logger=INFO,shuffleCLA -Dyarn.app.mapreduce.shuffle.logfile=syslog.shuffle -Dyarn.app.mapreduce.shuffle.log.filesize=0 -Dyarn.app.mapreduce.shuffle.log.backups=0 org.apache.hadoop.mapred.YarnChild 192.168.10.103 36822 attempt_1637894495535_0007_r_000000_1 13 

[2021-11-25 18:53:22.831]Container killed on request. Exit code is 143
[2021-11-25 18:53:22.832]Container exited with a non-zero exit code 143. 

解决方案:

参考官网文档:http://kylin.apache.org/docs20/install/advance_settings.html


在 cube 中插入修改,重启,构建成功!

转载请注明:文章转载自 www.mshxw.com
本文地址:https://www.mshxw.com/it/600218.html
我们一直用心在做
关于我们 文章归档 网站地图 联系我们

版权所有 (c)2021-2022 MSHXW.COM

ICP备案号:晋ICP备2021003244-6号