在执行一个spark程序的时候，明明把yarn-site.xml,core-site.xml,hdfs-site.xml都打包进去了，
可是还是报错
Exception in thread "main" org.apache.spark.SparkException: When running with master 'yarn' either HADOOP_CONF_DIR or YARN_CONF_DIR must be set in the environment.
at org.apache.spark.deploy.SparkSubmitArguments.error(SparkSubmitArguments.scala:657)
at org.apache.spark.deploy.SparkSubmitArguments.validateSubmitArguments(SparkSubmitArguments.scala:290)
at org.apache.spark.deploy.SparkSubmitArguments.validateArguments(SparkSubmitArguments.scala:251)
at org.apache.spark.deploy.SparkSubmitArguments.(SparkSubmitArguments.scala:120)
at org.apache.spark.deploy.SparkSubmit$$anon$2$$anon$1.(SparkSubmit.scala:907)
at org.apache.spark.deploy.SparkSubmit$$anon$2.parseArguments(SparkSubmit.scala:907)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:81)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:920)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:929)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
从报错上可以看出是环境没配置好
spark中的HADOOP_CONF_DIR和 YARN_CONF_DIR没配置
做法：
cd ${SPARK_HOME}/conf
cp spark-env.sh.template spark-ev.sh
vi spark-env.sh
添加
export JAVA_HOME=/usr/local/jdk export SPARK_MASTER_HOST=qianfeng01 export SPARK_MASTER_PORT=7077 export HADOOP_CONF_DIR=/usr/local/hadoop/etc/hadoop export YARN_CONF_DIR=/usr/local/hadoop/etc/hadoop
重点是添加上面的后两行！我是要把spark改成集群模式
