哪些技术让mapreduce使用简单化(简述mapreduce程序运行流程)

1.在/data目录下创建一个文件

vim wordcount.txt

放入一些词

2.返回hadoop文件夹，在hdfs上创建一个文件夹

bin/hdfs dfs -mkdir -p /wordcount/input

3.将/data目录下刚创建的文件放到hdfs的这个文件夹下

bin/hdfs dfs -put /home//data/wordcount.txt /wordcount/input

执行命令

bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1.3.jar  /wordcount/input /wordcount/output

发现有报错，查看报错信息

Please check whether your etc/hadoop/mapred-site.xml contains the below configuration:

  yarn.app.mapreduce.am.env
  HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}


  mapreduce.map.env
  HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}


  mapreduce.reduce.env
  HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}

在etc/hadoop/mapred-site.xml中添加报错中需要的信息

配置完成后，重启服务

然后重新运行

4.查看生成的文件是否正确

bin/hdfs dfs -cat /wordcount/output/*