flink的StandaloneHA部署flink1.8.0+hadoop2.6.5+zookeeper3.4.14

小鱼儿 2022-11-15 14:11 125阅读 0赞

# 1.说明 #

## 目标 ##

1）三台机器Centos7

<table> 
 <thead> 
  <tr> 
   <th>机器</th> 
   <th>JobManager</th> 
   <th>TaskManager</th> 
   <th>JouralNode</th> 
   <th>DataNode</th> 
   <th>NameNode</th> 
   <th>Zkfc</th> 
   <th>Zookeeper</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>192.168.11.131</td> 
   <td>√</td> 
   <td>√</td> 
   <td>√</td> 
   <td>√</td> 
   <td>√</td> 
   <td>√</td> 
   <td>√</td> 
  </tr> 
  <tr> 
   <td>192.168.11.132</td> 
   <td>-</td> 
   <td>√</td> 
   <td>√</td> 
   <td>√</td> 
   <td>√</td> 
   <td>√</td> 
   <td>√</td> 
  </tr> 
  <tr> 
   <td>192.168.11.133</td> 
   <td>√</td> 
   <td>√</td> 
   <td>√</td> 
   <td>√</td> 
   <td>-</td> 
   <td>-</td> 
   <td>√</td> 
  </tr> 
 </tbody> 
</table>

即JobManager部署131、132两台服务器，而NameNode部署131、133（Zkfc同步部署）

2）实现自启动上述服务

## 前提 ##

1）jdk已安装  
2）zookeeper的安装、自启动请参考：https://blog.csdn.net/sndayYU/article/details/100537922、https://blog.csdn.net/sndayYU/article/details/114047145

# 具体操作 #

## 1.免密登录 ##

参考：https://blog.csdn.net/sndayYU/article/details/115036682  
用户flink，别名分别时node1、node2、node3

**注意**  
同理131的root免密到132、133

## 2.相关目录 ##

/flink/为安装flink、hadoop、zookeeper安装目录，/data/为其数据及日志目录

chown flink:flink /data  
chown flink:flink /flink  
chmod 755 /data  
chmod 755 /flink

## 3.zookeeper ##

### 基本配置 ###

zookeeper的安装、自启动请参考：https://blog.csdn.net/sndayYU/article/details/100537922、https://blog.csdn.net/sndayYU/article/details/114047145

*  安装目录/flink/zookeeper-3.4.14/
 *  data目录/data/zookeeper/data/
 *  log目录/data/zookeeper/log/

dataDir=/data/zookeeper/data
    dataLogDir=/data/zookeeper/logs
    
    server.1=node1:2888:3888
    server.2=node2:2888:3888
    server.3=node3:2888:3888

### 自启动 ###

（参考：https://blog.csdn.net/sndayYU/article/details/114047145）：  
systemctl start zookeeper

脚本/lib/systemd/system/zookeeper.service

[Unit]
    Description=zookeeper
    [Service]
    Type=forking
    Environment=ZOO_LOG_DIR=/data/zookeeper
    Environment=JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.201.b09-2.el7_6.x86_64
    ExecStart=/flink/zookeeper-3.4.14/bin/zkServer.sh start
    ExecStop=/flink/zookeeper-3.4.14/bin/zkServer.sh stop
    PrivateTmp=true
    User=flink
    Group=flink
     
    [Install]
    WantedBy=multi-user.target

## 4.hadoop ##

### 基本配置 ###

解压，配置core-site.xml、hdfs-site.xml、slaves、删除masters，注意检查目录权限是否为flink。  
传到另外2台。

core-site.xml

<property>
            <name>fs.defaultFS</name>
    		
            <value>hdfs://hadoopha</value>
        </property>
        <property>
            <name>hadoop.tmp.dir</name>
    		
            <value>/data/hadoop/tmp</value>
        </property>
    	<property>
            <name>ha.zookeeper.quorum</name>
    		<value>node1:2181,node2:2181,node3:2181</value>
        </property>
        <property>
            <name>ha.zookeeper.session-timeout.ms</name>
            <value>150000</value>
        </property>

hdfs-site.xml

<property>
    	    
            <name>dfs.namenode.name.dir</name>
            <value>file:///data/hadoop/name</value>
        </property>
        <property>
            <name>dfs.datanode.data.dir</name>
            <value>file:///data/hadoop/data</value>
        </property>
    	
    	 <property>
            <name>dfs.replication</name>
            <value>3</value>
        </property>
    	
        <property>
            <name>dfs.nameservices</name>
            <value>hadoopha</value>
        </property>
    	<property>
            
            <name>dfs.ha.namenodes.hadoopha</name>
            <value>nn1,nn2</value>
        </property>
    	<property>
            
    		<name>dfs.namenode.rpc-address.hadoopha.nn1</name>
            <value>node1:9000</value>
        </property>
    	<property>
            <name>dfs.namenode.http-address.hadoopha.nn1</name>
            <value>node1:50070</value>
        </property>
    	<property>
            
    		<name>dfs.namenode.rpc-address.hadoopha.nn2</name>
            <value>node2:9000</value>
        </property>
    	<property>
            <name>dfs.namenode.http-address.hadoopha.nn2</name>
            <value>node2:50070</value>
        </property>
    	<property>
            <name>dfs.ha.automatic-failover.enabled</name>
            <value>true</value>
        </property>
    	<property>
            <name>dfs.namenode.shared.edits.dir</name>
            <value>qjournal://node1:8485;node2:8485;node3:8485/hadoopha</value>
        </property>
    	  
    	<property>  
    	    <name>dfs.journalnode.edits.dir</name>  
    	    <value>/data/hadoop/journal</value>  
    	</property>  
    	<property>  
    	    <name>dfs.client.failover.proxy.provider.hadoopha</name>  
    	    <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>  
    	</property> 
    	 <property>
            <name>dfs.ha.fencing.methods</name>
            <value>shell(/bin/true)</value>
        </property>
    	<property>
            <name>dfs.ha.fencing.ssh.private-key-files</name>
            <value>/root/.ssh/id_rsa</value>
        </property>
    	
    	<property>
            <name>dfs.ha.fencing.ssh.connect-timeout</name>
            <value>30000</value>
        </property>

[root@node1 ~]# cat /etc/profile.d/my.sh
    export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.201.b09-2.el7_6.x86_64
    export CALSSPATH=$JAVA_HOME/lib/*.*
    export PATH=$PATH:$JAVA_HOME/bin
    
    export HADOOP_HOME=/flink/hadoop-2.6.5
    export PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin

### 初始化 ###

> **注意**:若是不能一次性搞定，需要清空/data/hadoop目录，及zookeeper的相关数据

1）my.sh环境变量，hadoop version验证下，可以重启下服务器  
2）3台启动zookeeper，及 **切换flink** 执行journalnode  
/flink/hadoop-2.6.5/sbin/hadoop-daemon.sh start journalnode  
3）node1:  
hdfs namenode -format  
/flink/hadoop-2.6.5/sbin/hadoop-daemon.sh start namenode  
4) node2:  
hdfs namenode -bootstrapStandby  
5) node1:  
hdfs zkfc -formatZK  
/flink/hadoop-2.6.5/sbin/start-dfs.sh  
6) 最后访问node1:50070、node2:50070

### 自启动 ###

略

## 5.flink ##

### 基本配置 ###

flink用户解压，配置flink-conf.yaml、slaves、masters，  
hadoop的flink-shaded-hadoop-2-uber-2.6.5-7.0.jar放入flink的lib目录  
传到另外2台服务器

./flink-1.8.0/log目录一定要有，否则启动会报错

**flink-conf.yaml**

high-availability: zookeeper
    high-availability.storageDir: hdfs:///hadoopha/ha/
    high-availability.zookeeper.quorum: node1:2181,node2:2181,node3:2181
    state.backend: filesystem
    state.checkpoints.dir: hdfs:///hadoopha/flink-checkpoints

**slaves**

node1
    node2
    node3

**masters**

node1:8081
    node3:8081

### 启动 ###

\[flink@node1 bin\]$ ./start-cluster.sh  
Starting HA cluster with 2 masters.  
Starting standalonesession daemon on host node1.  
Starting standalonesession daemon on host node3.  
Starting taskexecutor daemon on host node1.  
Starting taskexecutor daemon on host node2.  
Starting taskexecutor daemon on host node3.  
\[flink@node1 bin\]$ jps -----验证是否启动正确

访问：http://node1:8081/、http://node3:8081/

### 自启动 ###

[root@node1 ~]# cat /lib/systemd/system/flink-jobmanager.service
    [Unit]
    Description=flink-jobmanager
    [Service]
    Type=forking
    Environment=JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.201.b09-2.el7_6.x86_64
    Environment=HADOOP_CONF_DIR=/flink/hadoop-2.6.5/etc/hadoop
    ExecStart=/flink/flink-1.8.0/bin/jobmanager.sh start
    ExecStop=/flink/flink-1.8.0/bin/jobmanager.sh stop
    PrivateTmp=true
    User=flink
    Group=flink
     
    [Install]
    WantedBy=multi-user.target
    [root@node1 ~]# cat /lib/systemd/system/flink-taskmanager.service
    [Unit]
    Description=flink-taskmanager
    [Service]
    Type=forking
    Environment=JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.201.b09-2.el7_6.x86_64
    Environment=HADOOP_CONF_DIR=/flink/hadoop-2.6.5/etc/hadoop
    ExecStart=/flink/flink-1.8.0/bin/taskmanager.sh start
    ExecStop=/flink/flink-1.8.0/bin/taskmanager.sh stop
    PrivateTmp=true
    User=flink
    Group=flink
     
    [Install]
    WantedBy=multi-user.target

// 3.scp传到node2、node3
    // 4.systemctl daemon-reload

node1、node3: systemctl start flink-jobmanager  
node1、node2、node3: systemctl start flink-taskmanager  
及systemctl enable xxxx

# 最终结果 #

自启动都配置好了后，重启3台电脑，可以发现都正确了，可以上传任务执行jar包