Hadoop【部署 02】hadoop-3.1.3 单机版YARN（配置、启动停止shell脚本修改及服务验证）

爱被打了一巴掌 2023-09-28 11:42 45阅读 0赞

### 1. 修改配置 ###

进入 $\{HADOOP\_HOME\}/etc/hadoop/ 目录下，修改以下配置：

1.  mapred-site.xml

[root@tcloud ~]# vim /usr/local/hadoop-3.1.3/etc/hadoop/mapred-site.xml

<configuration>
      <property>
    	<name>mapreduce.framework.name</name>
        <value>yarn</value>
      </property>
      
      <property>
    	<name>mapreduce.jobhistory.address</name>
    	<value>tcloud:10020</value>
       </property>
       <property>
    	<name>mapreduce.jobhistory.webapp.address</name>
    	<value>tcloud:19888</value>
       </property>
    </configuration>

1.  yarn-site.xml

[root@tcloud ~]# vim /usr/local/hadoop-3.1.3/etc/hadoop/yarn-site.xml

<configuration>
      <property>
        
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
      </property>
      
      <property>
    	<name>yarn.resourcemanager.hostname</name>
    	<value>tcloud</value>
      </property>
      
      <property>
    	<name>yarn.nodemanager.env-whitelist</name>	
    	<value>JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_MAPRED_HOME</value>
      </property>
      
      
      <property>
    	<name>yarn.scheduler.minimum-allocation-mb</name>
    	<value>128</value>
      </property>
      <property>
    	<name>yarn.scheduler.maximum-allocation-mb</name>
    	<value>256</value>
      </property>
      <property>
    	<name>yarn.nodemanager.resource.memory-mb</name>
    	<value>256</value>
      </property>
      <property>
    	<name>yarn.nodemanager.pmem-check-enabled</name>
    	<value>false</value>
      </property>
      <property>
    	<name>yarn.nodemanager.vmem-check-enabled</name>
    	<value>false</value>
      </property>
      
      
      <property>
        <name>yarn.log-aggregation-enable</name>
        <value>true</value>
      </property>
    
      
      <property>  
        <name>yarn.log.server.url</name>  
        <value>http://tcloud:19888/jobhistory/logs</value>
      </property>
    
      
      <property>
        <name>yarn.log-aggregation.retain-seconds</name>
        <value>604800</value>
      </property>
    </configuration>

### 2.修改启动和停止shell脚本 ###

[root@tcloud hadoop]# vim /usr/local/hadoop-3.1.3/sbin/start-yarn.sh
    [root@tcloud hadoop]# vim /usr/local/hadoop-3.1.3/sbin/stop-yarn.sh

start-yarn.sh，stop-yarn.sh 这两个文件顶部添加以下参数：

YARN_RESOURCEMANAGER_USER=root
    HDFS_DATANODE_SECURE_USER=yarn
    YARN_NODEMANAGER_USER=root

### 3. 启动服务 ###

进入 $\{HADOOP\_HOME\}/sbin/ 目录下，启动 YARN：

[root@tcloud sbin]# ./start-yarn.sh

### 4. 验证是否启动成功 ###

方式一：执行 jps 命令查看 NodeManager 和 ResourceManager 服务是否已经启动：

[root@tcloud sbin]# jps
    1701 NameNode
    1848 DataNode
    7512 Jps
    7198 NodeManager
    7055 ResourceManager
    2095 SecondaryNameNode

方式二：查看 Web UI 界面，端口号为 8088  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3l1YW56aGVuZ21l_size_16_color_FFFFFF_t_70_pic_center]

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3l1YW56aGVuZ21l_size_16_color_FFFFFF_t_70_pic_center]: https://img-blog.csdnimg.cn/48449793b470461088805dccb5a20053.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3l1YW56aGVuZ21l,size_16,color_FFFFFF,t_70#pic_center