2、安装hadoop
(1)配置ssh：ssh-keygen -t rsa => 一直enter => cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys => 确认mac已关闭防火墙和开启远程登录(系统偏好设置下：安全与隐私=>防火墙 关闭 ；共享 => 远程登录 打钩)
(2)
【一线大厂Java面试题解析+后端开发学习笔记+最新架构讲解视频+实战项目源码讲义】
开源分享完整内容戳这里
brew下载安装hadoop：brew install hadoop => 可以查看到/usr/local/Cellar/hadoop/3.0.0已经安装的hadoop位置
(3)配置hadoop-env.sh ： vim /usr/local/Cellar/hadoop/3.0.0/libexec/etc/hadoop/hadoop-env.sh => 将
export HADOOP_OPTS="-Djava.net.preferIPv4Stack=true -Dsun.security.krb5.debug=true -Dsun.security.spnego.debug"后添加
export HADOOP_OPTS="$HADOOP_OPTS -Djava.net.preferIPv4Stack=true -Djava.security.krb5.realm= -Djava.security.krb5.kdc="
export JAVA_HOME="/Library/Java/JavaVirtualMachines/jdk1.8.0_161.jdk/Contents/Home"
（4）配置core-site.xml【hdfs地址和端口】：vim /usr/local/Cellar/hadoop/3.0.0/libexec/etc/hadoop/core-site.xml => 添加配置
hadoop.tmp.dir
/usr/local/Cellar/hadoop/hdfs/tmp
A base for other temporary directories.
fs.default.name
hdfs://localhost:8020
并且建立文件夹 mkdir /usr/local/Cellar/hadoop/hdfs & mkdir /usr/local/Cellar/hadoop/hdfs/tmp
(5)配置mapred-site.xml【mapreduce和jobtracker的地址和端口】：
先备份：cp /usr/local/Cellar/hadoop/3.0.0/libexec/etc/hadoop/mapred-site.xml mapred-site-bak.xml
再编辑：vim /usr/local/Cellar/hadoop/3.0.0/libexec/etc/hadoop/mapred-site.xml => 添加配置
mapred.job.tracker
localhost:8021
(6)配置hdfs-site.xml【修改hdfs备份数】：vim /usr/local/Cellar/hadoop/3.0.0/libexec/etc/hadoop/hdfs-site.xml => 添加配置
dfs.replication
1
(7)格式化hdfs文件系统格式：hdfs namenode -format
(8)启动及关闭hadoop服务：
/usr/local/Cellar/hadoop/3.0.0/libexec/start-dfs.sh => 守护进程：namenodes、datanodes、secondary namenodes，浏览器中访问http://localhost:9870 ,注意端口号是9870而不是50070
/usr/local/Cellar/hadoop/3.0.0/libexec/start-yarn.sh => yarn服务进程：resourcemanager、nodemanagers，浏览器中访问http://localhost:8088 和 http://localhost:8042
/usr/local/Cellar/hadoop/3.0.0/libexec/stop-yarn.sh
/usr/local/Cellar/hadoop/3.0.0/libexec/stop-dfs.sh
3、安装scala
brew下载安装scala：brew install scala => 可以查看到/usr/local/Cellar/scala/2.12.5存在或scala -version查看版本
4、安装spark
spark官网下载，下载的时候注意查看其需要依赖的hadoop版本是否满足，下载后mac下直接用解压软件解压并将解压后的文件mv至/usr/local下也即/usr/local/spark-2.3.0-bin-hadoop2.7
cd /usr/local/spark-2.3.0-bin-hadoop2.7/bin & spark-shell
5、系统环境变量~/.bash_profile，便于在任何目录下执行一些命令
