Hive 配置mapred.tasktracker.map.tasks.maximum/hive.exec.reducers.max /每一个tasktracker同时运行的map任务数/最大map数

小灰灰 2021-09-25 02:56 303阅读 0赞

set  mapred.tasktracker.map.tasks.maximum ; -- 2 ,默认，每一个tasktracker**同时运行**的map任务数为2

The maximum number of map tasks that will be run  
simultaneously by a task tracker.

set  mapred.tasktracker.reduce.tasks.maximum ; -- 2 ,默认，每一个tasktracker**同时运行**的reduce任务数为2

The maximum number of reduce tasks that will be run  
simultaneously by a task tracker.

set hive.exec.reducers.max ； --  is undefined，设置该Job的最大reducer数量

*  Default Value: 999 prior to Hive 0.14.0; 1009 in Hive 0.14.0 and later
 *  Added In: Hive 0.2.0; default changed in 0.14.0 with HIVE-7158 (and HIVE-7917) Maximum number of reducers that will be used. If the one specified in the configuration property mapred.reduce.tasks is negative, Hive will use this as the maximum number of reducers when automatically determining the number of reducers.

set hive.exec.mappers.max ；--  is undefined，设置该Job的最大map数量

[https://www.aboutyun.com/thread-7530-1-1.html][https_www.aboutyun.com_thread-7530-1-1.html]

[https://cloud.tencent.com/developer/article/1403285][https_cloud.tencent.com_developer_article_1403285] 很好

[https://zhuanlan.zhihu.com/p/47037815][https_zhuanlan.zhihu.com_p_47037815] 很好

Hive2.0.0

[https_www.aboutyun.com_thread-7530-1-1.html]: https://www.aboutyun.com/thread-7530-1-1.html
[https_cloud.tencent.com_developer_article_1403285]: https://cloud.tencent.com/developer/article/1403285
[https_zhuanlan.zhihu.com_p_47037815]: https://zhuanlan.zhihu.com/p/47037815