jdk1.8 mysql5.7以上(但是驱动要使用8版本的)
安装
qianfeng01:master worker qianfeng02:worker qianfeng03:worker1、环境
##三台虚拟机需要 install -y psmisc2、初始化元数据库(打开mysql)
mysql -uroot -p'@Mmforu45'
下面在mysql中创建数据库,新增用户,授权
##建库 CREATE DATABASE dolphinscheduler DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; ##密码 set global validate_password_policy=0; set global validate_password_length=4; ##创建用户 CREATE USER 'dolphinscheduler'@'%' IDENTIFIED BY '1234'; ##权限 GRANT ALL PRIVILEGES ON dolphinscheduler.* TO 'dolphinscheduler'@'%'; flush privileges;
注意
set global validate_password_policy=0; set global validate_password_length=4; 这两个没有在新增用户之前执行,会报错 ERROR 1819 (HY000): Your password does not satisfy the current policy requirements3、设置小海豚的配置
安装包
链接:https://pan.baidu.com/s/1VLvMzOl-1c2e4VxNDxkkIQ?pwd=mzrh
提取码:mzrh
–来自百度网盘超级会员V2的分享
把安装包和mysql驱动扔到服务器上
tar -xvf apache-dolphinscheduler-2.0.5-bin.tar.gz -c /usr/local/ cd /usr/local/ mv apache-dolphinscheduler-2.0.5-bin dolphinscheduler
cp mysql-connector-java-8.0.16.jar /usr/local/dolphinscheduler/lib/
注意:
我一开始mysql驱动选择的是5.7版本,但是mysql初始化报错的内容是driver,我也没找到具体原因,就只能把5.7版本当8版本使用了。就是使用8版本的驱动,初始化是成功的(我第一次使用5.7驱动其实也初始化成功了)
最重要的步骤
cd /usr/local/dolphinscheduler/conf/config/ vi install_config.conf
# # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # # --------------------------------------------------------- # INSTALL MACHINE # --------------------------------------------------------- # A comma separated list of machine hostname or IP would be installed DolphinScheduler, # including master, worker, api, alert. If you want to deploy in pseudo-distributed # mode, just write a pseudo-distributed hostname # Example for hostnames: ips="ds1,ds2,ds3,ds4,ds5", Example for IPs: ips="192.168.8.1,192.168.8.2,192.168.8.3,192.168.8.4,192.168.8.5" ips="qianfeng01,qianfeng02,qianfeng03" #集群主机号 # Port of SSH protocol, default value is 22. For now we only support same port in all `ips` machine # modify it if you use different ssh port sshPort="22" # A comma separated list of machine hostname or IP would be installed Master server, it # must be a subset of configuration `ips`. # Example for hostnames: masters="ds1,ds2", Example for IPs: masters="192.168.8.1,192.168.8.2" masters="qianfeng01" #主节点 # A comma separated list of machine: or : .All hostname or IP must be a # subset of configuration `ips`, And workerGroup have default value as `default`, but we recommend you declare behind the hosts # Example for hostnames: workers="ds1:default,ds2:default,ds3:default", Example for IPs: workers="192.168.8.1:default,192.168.8.2:default,192.168.8.3:default" workers="qianfeng01:default,qianfeng02:default,qianfeng03:default" #从节点 # A comma separated list of machine hostname or IP would be installed Alert server, it # must be a subset of configuration `ips`. # Example for hostname: alertServer="ds3", Example for IP: alertServer="192.168.8.3" alertServer="qianfeng01" #报警主机 # A comma separated list of machine hostname or IP would be installed API server, it # must be a subset of configuration `ips`. # Example for hostname: apiServers="ds1", Example for IP: apiServers="192.168.8.1" apiServers="qianfeng01" #开发api主机 # A comma separated list of machine hostname or IP would be installed Python gateway server, it # must be a subset of configuration `ips`. # Example for hostname: pythonGatewayServers="ds1", Example for IP: pythonGatewayServers="192.168.8.1" # pythonGatewayServers="ds1" #上面python可以注释掉 # The directory to install DolphinScheduler for all machine we config above. It will automatically be created by `install.sh` script if not exists. # Do not set this configuration same as the current path (pwd) installPath="/usr/local/" #上面的是一键安装脚本是把本机的小海豚安装到其他机器上,路径在其他机器要存在,不然会报错,本机的小海豚也会被删没 # The user to deploy DolphinScheduler for all machine we config above. For now user must create by yourself before running `install.sh` # script. The user needs to have sudo privileges and permissions to operate hdfs. If hdfs is enabled than the root directory needs # to be created by this user deployUser="root" #部署的账户名 # The directory to store local data for all machine we config above. Make sure user `deployUser` have permissions to read and write this directory. dataBasedirPath="/tmp/dolphinscheduler" # --------------------------------------------------------- # DolphinScheduler ENV # --------------------------------------------------------- # JAVA_HOME, we recommend use same JAVA_HOME in all machine you going to install DolphinScheduler # and this configuration only support one parameter so far. javaHome="/usr/local/jdk" #java安装路径 # DolphinScheduler API service port, also this is your DolphinScheduler UI component's URL port, default value is 12345 apiServerPort="12345" #web端口 # --------------------------------------------------------- # Database # NOTICE: If database value has special characters, such as `.*[]^${}+?|()@#&`, Please add prefix `` for escaping. # --------------------------------------------------------- # The type for the metadata database # Supported values: ``postgresql``, ``mysql`, `h2``. DATABASE_TYPE="mysql" #数据库类型 # Spring datasource url, following : / ? format, If you using mysql, you could use jdbc # string jdbc:mysql://127.0.0.1:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8 as example SPRING_DATASOURCE_URL="jdbc:mysql://qianfeng03:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8" #mysqlurl # Spring datasource username SPRING_DATASOURCE_USERNAME="dolphinscheduler" #mysql账户 # Spring datasource password SPRING_DATASOURCE_PASSWORD="1234" #密码 # --------------------------------------------------------- # Registry Server # --------------------------------------------------------- # Registry Server plugin name, should be a substring of `registryPluginDir`, DolphinScheduler use this for verifying configuration consistency registryPluginName="zookeeper" #zookeeper # Registry Server address. registryServers="qianfeng01:2181,qianfeng02:2181,qianfeng03:2181" # Registry Namespace registryNamespace="dolphinscheduler" # --------------------------------------------------------- # Worker Task Server # --------------------------------------------------------- # Worker Task Server plugin dir. DolphinScheduler will find and load the worker task plugin jar package from this dir. taskPluginDir="lib/plugin/task" # resource storage type: HDFS, S3, NONE resourceStorageType="HDFS" #资源类型选择 # resource store on HDFS/S3 path, resource file will store to this hdfs path, self configuration, please make sure the directory exists on hdfs and has read write permissions. "/dolphinscheduler" is recommended resourceUploadPath="/dolphinscheduler" # if resourceStorageType is HDFS,defaultFS write namenode address,HA, you need to put core-site.xml and hdfs-site.xml in the conf directory. # if S3,write S3 address,HA,for example :s3a://dolphinscheduler, # Note,S3 be sure to create the root directory /dolphinscheduler defaultFS="hdfs://qianfeng01:8020" #就是在web端前面的内容 # if resourceStorageType is S3, the following three configuration is required, otherwise please ignore s3Endpoint="http://192.168.xx.xx:9010" s3AccessKey="xxxxxxxxxx" s3SecretKey="xxxxxxxxxx" # resourcemanager port, the default value is 8088 if not specified resourceManagerHttpAddressPort="8088" # if resourcemanager HA is enabled, please set the HA IPs; if resourcemanager is single node, keep this value empty yarnHaIps= #没有就是空 # if resourcemanager HA is enabled or not use resourcemanager, please keep the default value; If resourcemanager is single node, you only need to replace 'yarnIp1' to actual resourcemanager hostname singleYarnIp="qianfeng01" # who has permission to create directory under HDFS/S3 root path # Note: if kerberos is enabled, please config hdfsRootUser= hdfsRootUser="root" #hadoop免密 # kerberos config # whether kerberos starts, if kerberos starts, following four items need to config, otherwise please ignore kerberosStartUp="false" # kdc krb5 config file path krb5ConfPath="$installPath/conf/krb5.conf" # keytab username,watch out the @ sign should followd by \ keytabUserName="hdfs-mycluster\@ESZ.COM" # username keytab path keytabPath="$installPath/conf/hdfs.headless.keytab" # kerberos expire time, the unit is hour kerberosExpireTime="2" # use sudo or not sudoEnable="true" # worker tenant auto create workerTenantAutoCreate="false"
vi /usr/local/dolphinscheduler/script/create-dolphinscheduler.sh
##Java路径一定要改 export JAVA_HOME=/usr/local/jdk4、初始化数据库
cd /usr/local/dolphinscheduler/script ./create-dolphinscheduler.sh
初始化之前,mysql的驱动包一定要放在lib目录下!!!
5、一键安装(就是系统自动把本机的小海豚根据配置scp到从节点)三台机器的zookeeper一定要打开(如果只打开一台不会报错,但是web无法访问!)
这里不需要手动scp文件到其他服务器了,直接运行shell!
zkServer.sh start
cd /usr/local/dolphinscheduler ./install.sh6、访问
网址: http://qianfeng01:12345/dolphinscheduler/ 账号: admin 密码: dolphinscheduler123



