Flink 文档初稿--给实施用的

淡淡的烟草味﹌ 2023-06-21 12:21 14阅读 0赞

*  Flink集群部署

1.  下载 flink-1.7.2 解压安装
2.  修改配置文件

vim conf/flink-conf.yaml

这些参数都是必须的，可以动态调节资源大小

参数具体为：

> jobmanager.rpc.address: node1.hadoop
> 
> jobmanager.rpc.port: 6123
> 
> jobmanager.heap.size: 1024m
> 
> taskmanager.heap.size: 1024m
> 
> taskmanager.numberOfTaskSlots: 2
> 
> parallelism.default: 2
> 
> state.checkpoints.num-retained: 20
> 
> rest.port: 8081
> 
> classloader.resolve-order: parent-first

修改配置文件：

1.  vim maste

node1.hadoop:8081

1.  vim slaves

node3.hadoop

node2.hadoop

1.  分发到其他节点
2.  启动 ./bin/start-cluster.sh

*  提交任务到yarn命令：

1.  第一提交

1.  如果是重启任务，必须制定checkpoint 路径

/zywa/flink/flink-1.7.2/bin/flink run  -m yarn-cluster  -ynm FlinkMatchTopicApplication\_app -yn 2  -ys 2 -p 4 -yjm 2048m -ytm 2048m -c application.storm.FlinkMatchTopicApplication\_develop -s hdfs://node1.hadoop:9000//flink/checkpoint/bc6ccee6e4c0a32aef3255ec700c2f89/chk-9/\_metadata  ./flink1.9.5Test.jar

*  查看程序是否正常运行：
 *  ![20191212165621339.png][]

1.  查看yarn任务是否存在：![20191212165639421.png][]

1.  点击跳转：![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMxODY2Nzkz_size_16_color_FFFFFF_t_70][]

1.  跳转到页面并查看申请资源是否正确：![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMxODY2Nzkz_size_16_color_FFFFFF_t_70 1][]

1.  查看正在运行的任务：![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMxODY2Nzkz_size_16_color_FFFFFF_t_70 2][]

[20191212165621339.png]: https://img-blog.csdnimg.cn/20191212165621339.png
[20191212165639421.png]: https://img-blog.csdnimg.cn/20191212165639421.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMxODY2Nzkz_size_16_color_FFFFFF_t_70]: https://img-blog.csdnimg.cn/20191212165648101.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMxODY2Nzkz,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMxODY2Nzkz_size_16_color_FFFFFF_t_70 1]: https://img-blog.csdnimg.cn/2019121216565998.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMxODY2Nzkz,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMxODY2Nzkz_size_16_color_FFFFFF_t_70 2]: https://img-blog.csdnimg.cn/20191212165707498.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzMxODY2Nzkz,size_16,color_FFFFFF,t_70