Storm 集群的入门安装

迷南。 2022-01-30 15:23 351阅读 0赞

### 文章目录 ###

*   *  1、前提
     *  2、集群说明
     *  3、安装步骤
     *   *  3.1、下载 storm
         *  3.2、上传、解压（3台）
         *  3.3、创建 data 目录（3台）
         *  3.4、配置 /etc/profile（3台）
         *  3.5、配置 storm（3台）
         *  3.6、启动
         *   *  3.6.1、在 192.168.1.104 启动 nimbus
             *  3.6.2、在 192.168.1.105、106 启动 supervisor
         *  3.7、检查是否启动成功
         *  3.8、查看日志，检查是否启动成功：
         *  3.9、storm UI
         *   *  3.9.1、启动 storm ui
             *  3.9.2、查看 \`storm ui\` 的日志
             *  3.9.3、登录 \`storm ui\`

## 1、前提 ##

Storm 安装前，确保 zookeeper 的单机或集群版已经配置好，并启动成功。

## 2、集群说明 ##

zookeeper ：

<table> 
 <thead> 
  <tr> 
   <th>IP地址</th> 
   <th>名称</th> 
   <th>说明</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>192.168.1.102</td> 
   <td>zookeeper</td> 
   <td>单机版的（如果是集群版的会更好）</td> 
  </tr> 
 </tbody> 
</table>

storm ：

<table> 
 <thead> 
  <tr> 
   <th>IP地址</th> 
   <th>名称</th> 
   <th>说明</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>192.168.1.104</td> 
   <td>nimbus、ui 的服务</td> 
   <td>主节点</td> 
  </tr> 
  <tr> 
   <td>192.168.1.105</td> 
   <td>supervisor</td> 
   <td>工作节点1</td> 
  </tr> 
  <tr> 
   <td>192.168.1.106</td> 
   <td>supervisor</td> 
   <td>工作节点2</td> 
  </tr> 
 </tbody> 
</table>

## 3、安装步骤 ##

### 3.1、下载 storm ###

清华大学镜像地址： [https://mirrors.cnnic.cn/apache/storm/][https_mirrors.cnnic.cn_apache_storm]

我下载的是 apache-storm-0.9.2-incubating.tar.gz

### 3.2、上传、解压（3台） ###

上传 apache-storm-0.9.2-incubating.tar.gz 到 centos7 的 `/mysoft` 中

解压命令：

tar -zxvf  apache-storm-0.9.2-incubating.tar.gz

重命名为 `/apache-storm-0.9.2` ，移动 `/usr/local/` 目录下：

mv apache-storm-0.9.2-incubating /usr/local/apache-storm-0.9.2

### 3.3、创建 data 目录（3台） ###

此目录用于存放 storm 的数据

cd  /usr/local/apache-storm-0.9.2/
    
    /usr/local/apache-storm-0.9.2/data

### 3.4、配置 /etc/profile（3台） ###

将 storm\_home 配置到 `/etc/profile` 上。

vi  /etc/profile

内容：

STORM_HOME=/usr/local/apache-storm-0.9.2/
    
    PATH=$PATH:$STORM_HOME/bin
    
    export PATH STORM_HOME

重新加载 profile：

source /etc/profile

### 3.5、配置 storm（3台） ###

编辑 conf 目录下的 `storm.yaml` 配置文件：

vi  /usr/local/apache-storm-0.9.2/conf/storm.yaml

添加 或修改内容如下：

storm.zookeeper.servers:
         - "192.168.1.102"
    
    
     nimbus.host: "192.168.1.104"
     storm.local.dir: "/usr/local/apache-storm-0.9.2/data"
     ui.port: 18080
     supervisor.slots.ports:
         - 6700
         - 6701
         - 6702
         - 6703

**说明 ：**

**（3台）** 是指此步骤的操作，在3台服务器上完全相同。

### 3.6、启动 ###

#### 3.6.1、在 192.168.1.104 启动 nimbus ####

由于 storm 已经在 `/etc/profile` 中配置全局命令，所以 **启动命令** 可以在任何目录下操作

**启动 nimbus ：**

storm nimbus  &
    
    也可以使用下面的命令：
    nohup bin/storm nimbus >/dev/null 2>&1 &

#### 3.6.2、在 192.168.1.105、106 启动 supervisor ####

**启动 nimbus ：**

storm supervisor  &

### 3.7、检查是否启动成功 ###

192.168.1.104 查看 nimbus 的启动状态 ：

[root@rocketmq-nameserver1 conf]# jps
    5425 Jps
    5252 nimbus

192.168.1.105、106 上 查看 supervisor 的启动状态：

[root@rocketmq-nameserver3 logs]# jps
    4677 supervisor
    6795 Jps

### 3.8、查看日志，检查是否启动成功： ###

**192.168.1.104 查看 nimbus 的日志 ：**

进入 logs 目录：

[root@rocketmq-nameserver1 apache-storm-0.9.2]# cd logs/
    [root@rocketmq-nameserver1 logs]# ls
    access.log  metrics.log  nimbus.log  ui.log

查看 nimbus.log 日志（192.168.1.104 启动的是 nimbus ）

[root@rocketmq-nameserver1 logs]# tail -300f nimbus.log 
    2019-05-19 19:07:28 o.a.z.ZooKeeper [INFO] Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
    2019-05-19 19:07:28 o.a.z.ZooKeeper [INFO] Client environment:host.name=rocketmq-nameserver1
    2019-05-19 19:07:28 o.a.z.ZooKeeper [INFO] Client environment:java.version=1.8.0_201
    2019-05-19 19:07:28 o.a.z.ZooKeeper [INFO] Client environment:java.vendor=Oracle Corporation
    2019-05-19 19:07:28 o.a.z.ZooKeeper [INFO] Client environment:java.home=/mysoft/jdk1.8.0_201/jre
    
    .....（省略）
    
    2019-05-19 19:07:29 o.a.z.ZooKeeper [INFO] Initiating client connection, connectString=192.168.1.102:2181 sessionTimeout=20000 watcher=org.apache.curator.ConnectionState@400d912a
    2019-05-19 19:07:29 o.a.z.ClientCnxn [INFO] Opening socket connection to server 192.168.1.102/192.168.1.102:2181. Will not attempt to authenticate using SASL (unknown error)
    2019-05-19 19:07:30 o.a.z.ClientCnxn [INFO] Socket connection established to 192.168.1.102/192.168.1.102:2181, initiating session
    2019-05-19 19:07:30 o.a.z.ClientCnxn [INFO] Session establishment complete on server 192.168.1.102/192.168.1.102:2181, sessionid = 0x16acfa5816d0000, negotiated timeout = 20000
    2019-05-19 19:07:30 o.a.c.f.s.ConnectionStateManager [INFO] State change: CONNECTED
    2019-05-19 19:07:30 o.a.c.f.s.ConnectionStateManager [WARN] There are no ConnectionStateListeners registered.
    2019-05-19 19:07:30 b.s.zookeeper [INFO] Zookeeper state update: :connected:none
    2019-05-19 19:07:31 o.a.z.ZooKeeper [INFO] Session: 0x16acfa5816d0000 closed
    2019-05-19 19:07:31 o.a.z.ClientCnxn [INFO] EventThread shut down
    2019-05-19 19:07:31 o.a.c.f.i.CuratorFrameworkImpl [INFO] Starting
    2019-05-19 19:07:31 o.a.z.ZooKeeper [INFO] Initiating client connection, connectString=192.168.1.102:2181/storm sessionTimeout=20000 watcher=org.apache.curator.ConnectionState@6ac97b84
    2019-05-19 19:07:31 o.a.z.ClientCnxn [INFO] Opening socket connection to server 192.168.1.102/192.168.1.102:2181. Will not attempt to authenticate using SASL (unknown error)
    2019-05-19 19:07:31 o.a.z.ClientCnxn [INFO] Socket connection established to 192.168.1.102/192.168.1.102:2181, initiating session
    2019-05-19 19:07:31 o.a.z.ClientCnxn [INFO] Session establishment complete on server 192.168.1.102/192.168.1.102:2181, sessionid = 0x16acfa5816d0001, negotiated timeout = 20000
    2019-05-19 19:07:31 o.a.c.f.s.ConnectionStateManager [INFO] State change: CONNECTED
    2019-05-19 19:07:31 o.a.c.f.s.ConnectionStateManager [WARN] There are no ConnectionStateListeners registered.
    2019-05-19 19:07:32 b.s.d.nimbus [INFO] Starting Nimbus server...

**192.168.1.105、106 查看 nimbus 的日志 ：**

[root@rocketmq-nameserver2 apache-storm-0.9.2]# cd logs/
    [root@rocketmq-nameserver2 logs]# ls
    access.log  metrics.log  supervisor.log

查看 supervisor.log 日志：（192.168.1.105 、106 启动的是 supervisor ）

[root@rocketmq-nameserver3 logs]# tail -300f supervisor.log 
    2019-05-19 19:08:38 o.a.z.ZooKeeper [INFO] Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
    2019-05-19 19:08:38 o.a.z.ZooKeeper [INFO] Client environment:host.name=rocketmq-nameserver3
    2019-05-19 19:08:38 o.a.z.ZooKeeper [INFO] Client environment:java.version=1.8.0_201
    2019-05-19 19:08:38 o.a.z.ZooKeeper [INFO] Client environment:java.vendor=Oracle Corporation
    2019-05-19 19:08:38 o.a.z.ZooKeeper [INFO] Client environment:java.home=/mysoft/jdk1.8.0_201/jre
    
    .....（省略）
    
    2019-05-19 19:08:39 o.a.z.ZooKeeper [INFO] Initiating client connection, connectString=192.168.1.102:2181 sessionTimeout=20000 watcher=org.apache.curator.ConnectionState@6579c3d9
    2019-05-19 19:08:39 o.a.z.ClientCnxn [INFO] Opening socket connection to server 192.168.1.102/192.168.1.102:2181. Will not attempt to authenticate using SASL (unknown error)
    2019-05-19 19:08:39 o.a.z.ClientCnxn [INFO] Socket connection established to 192.168.1.102/192.168.1.102:2181, initiating session
    2019-05-19 19:08:40 o.a.z.ClientCnxn [INFO] Session establishment complete on server 192.168.1.102/192.168.1.102:2181, sessionid = 0x16acfa5816d0002, negotiated timeout = 20000
    2019-05-19 19:08:40 o.a.c.f.s.ConnectionStateManager [INFO] State change: CONNECTED
    2019-05-19 19:08:40 o.a.c.f.s.ConnectionStateManager [WARN] There are no ConnectionStateListeners registered.
    2019-05-19 19:08:40 b.s.zookeeper [INFO] Zookeeper state update: :connected:none
    2019-05-19 19:08:41 o.a.z.ZooKeeper [INFO] Session: 0x16acfa5816d0002 closed
    2019-05-19 19:08:41 o.a.c.f.i.CuratorFrameworkImpl [INFO] Starting
    2019-05-19 19:08:41 o.a.z.ClientCnxn [INFO] EventThread shut down
    2019-05-19 19:08:41 o.a.z.ZooKeeper [INFO] Initiating client connection, connectString=192.168.1.102:2181/storm sessionTimeout=20000 watcher=org.apache.curator.ConnectionState@59712875
    2019-05-19 19:08:41 o.a.z.ClientCnxn [INFO] Opening socket connection to server 192.168.1.102/192.168.1.102:2181. Will not attempt to authenticate using SASL (unknown error)
    2019-05-19 19:08:41 o.a.z.ClientCnxn [INFO] Socket connection established to 192.168.1.102/192.168.1.102:2181, initiating session
    2019-05-19 19:08:41 o.a.z.ClientCnxn [INFO] Session establishment complete on server 192.168.1.102/192.168.1.102:2181, sessionid = 0x16acfa5816d0003, negotiated timeout = 20000
    2019-05-19 19:08:41 o.a.c.f.s.ConnectionStateManager [INFO] State change: CONNECTED
    2019-05-19 19:08:41 o.a.c.f.s.ConnectionStateManager [WARN] There are no ConnectionStateListeners registered.
    2019-05-19 19:08:41 b.s.d.supervisor [INFO] Starting supervisor with id 0ee53c84-1eb4-4a4d-b9be-f7f72ba8e9b2 at host rocketmq-nameserver2

### 3.9、storm UI ###

如果前面的步骤检查没有问题了，启动 `storm ui` 。

#### 3.9.1、启动 storm ui ####

storm ui  &
    
    也可以使用下面的命令：
    nohup bin/storm supervisor >/dev/null 2>&1 &

#### 3.9.2、查看 `storm ui` 的日志 ####

[root@rocketmq-nameserver1 apache-storm-0.9.2]# cd logs/
    [root@rocketmq-nameserver1 logs]# ls
    access.log  metrics.log  nimbus.log  ui.log

查看日志：

[root@rocketmq-nameserver1 logs]# tail -500f ui.log 
    2019-05-19 19:18:26 o.m.log [INFO] Logging to Logger[org.mortbay.log] via org.mortbay.log.Slf4jLog
    2019-05-19 19:18:26 o.m.log [INFO] jetty-6.1.26
    2019-05-19 19:18:26 o.m.log [INFO] Started SocketConnector@0.0.0.0:18080
    2019-05-19 19:18:28 b.s.thrift [INFO] Connecting to Nimbus at 192.168.1.104:6627
    2019-05-19 19:18:28 b.s.thrift [INFO] Connecting to Nimbus at 192.168.1.104:6627

从日志中可以看出 UI 的端口号是 `18080` 。

#### 3.9.3、登录 `storm ui` ####

地址：`http://192.168.1.104:18080`

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3hpYW9qaW4yMWNlbg_size_16_color_FFFFFF_t_70]

[https_mirrors.cnnic.cn_apache_storm]: https://mirrors.cnnic.cn/apache/storm/
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3hpYW9qaW4yMWNlbg_size_16_color_FFFFFF_t_70]: /images/20220130/c32a603719054f4ab2ecfd36cd2742a2.png