1）NameNode 内存计算
每个文件块大概占用 150byte，一台服务器 128G 内存为例，能存储多少文件块呢？
128 * 1024 * 1024 * 1024 / 150Byte ≈ 9.1 亿
G MB KB Byte
2）Hadoop2.x 系列，配置 NameNode 内存
NameNode 内存默认 2000m，如果服务器内存 4G，NameNode 内存可以配置 3g。在
hadoop-env.sh 文件中配置如下。
HADOOP_NAMENODE_OPTS=-Xmx3072m
3）Hadoop3.x 系列，配置 NameNode 内存
（1）hadoop-env.sh 中描述 Hadoop 的内存是动态分配的
# The maximum amount of heap to use (Java -Xmx). If no unit # is provided, it will be converted to MB. Daemons will # prefer any Xmx setting in their respective _OPT variable. # There is no default; the JVM will autoscale based upon machine # memory size. # export HADOOP_HEAPSIZE_MAX= # The minimum amount of heap to use (Java -Xms). If no unit # is provided, it will be converted to MB. Daemons will # prefer any Xms setting in their respective _OPT variable. # There is no default; the JVM will autoscale based upon machine # memory size. # export HADOOP_HEAPSIZE_MIN= HADOOP_NAMENODE_OPTS=-Xmx102400m
（2）查看 NameNode 占用内存
[root@hadoop102 ~]$ jps 3088 NodeManager 2611 NameNode 3271 JobHistoryServer 2744 DataNode 3579 Jps
[root@hadoop102 ~]$ jmap -heap 2611 Heap Configuration: MaxHeapSize = 1031798784 (984.0MB)
（3）查看 DataNode 占用内存
[root@hadoop102 ~]$ jmap -heap 2744 Heap Configuration: MaxHeapSize = 1031798784 (984.0MB)
查看发现 hadoop102 上的 NameNode 和 DataNode 占用内存都是自动分配的，且相等。
不是很合理。
经验参考：
https://docs.cloudera.com/documentation/enterprise/6/releasenotes/topics/rg_hardware_requirements.html#concept_fzz_dq4_gbb
具体修改：hadoop-env.sh
export HDFS_NAMENODE_OPTS="-Dhadoop.security.logger=INFO,RFAS -Xmx1024m" export HDFS_DATANODE_OPTS="-Dhadoop.security.logger=ERROR,RFAS -Xmx1024m"
