[prometheus]Step10-prometheus使用grafana的alert告警监控docker容器

桃扇骨 2023-02-15 14:58 21阅读 0赞

### prometheus 使用grafana的alert告警监控docker容器 ###

接上一篇。由于grafana只支持图表告警，所以想要使用它的alert告警，需要专门创建一个视图。

##### 剩余可用内存 #####

![format_png][]

![format_png 1][]

![format_png 2][]

##### 内存使用率 #####

![format_png 3][]

![format_png 4][]

![format_png 5][]

##### 修改参数 #####

点击小齿轮

\[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-bQvtPuGC-1591409337788)(https://gitee.com/jiaminxu/BlogImage/raw/master/img/20200602161152.png)\]

选择Variables 添加如下参数

![format_png 6][]

![format_png 7][]

![format_png 8][]

![format_png 9][]

![format_png 10][]

添加告警规则 添加完成之后 点击Test Rule测试一下

![format_png 11][]

##### 触发告警时 显示如下 #####

![format_png 12][]

![format_png 13][]

#### 配置aler的媒介 ####

##### 邮件 #####

首先备份并修改grafana的配置文件defaults.ini

# 修改
    domain = 自己的ip地址
    
    # 替换smtp配置 我这里用了同事的账号
    [smtp]
    enabled = true
    host = smtp.aliyun.com:465
    user = $name@aliyun.com
    # If the password contains # or ; you have to wrap it with triple quotes. Ex """#password;"""
    password = $password
    cert_file =
    key_file =
    skip_verify = true
    from_address = $name@aliyun.com
    from_name = Grafana
    ehlo_identity =
    
    
    enforce_domain = true

systemctl  restart  grafana
    systemctl  status   grafana

点击Notification channels 并点击 Add channel

![format_png 14][]

配置邮件告警 并测试

![format_png 15][]

确认收到邮件之后 点击保存

![format_png 16][]

去alert界面添加 message Send to处会自动默认

![format_png 17][]

![format_png 18][]

点击view或者go 就会跳转到告警相关的界面

下面配置自己安装的alermanager告警对比一下 感觉还是grafana好看

![format_png 19][]

##### 钉钉告警 #####

![format_png 20][]

![format_png 21][]

测试告警

![format_png 22][]

点开界面就是grafana的监控图

![format_png 23][]

最后使用6个视图 用来做6台docker宿主机的监控 具体如下监控如下

max(container_memory_working_set_bytes{name!="",job=~"docker2"}/container_spec_memory_limit_bytes{name!="",job=~"docker2"})by (name)*100

![format_png 24][]

效果图如下

![format_png 25][]

[format_png]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vamlhbWlueHUvQmxvZ0ltYWdlL3Jhdy9tYXN0ZXIvaW1nLzIwMjAwNjAyMTU1OTE4LnBuZw?x-oss-process=image/format,png
[format_png 1]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vamlhbWlueHUvQmxvZ0ltYWdlL3Jhdy9tYXN0ZXIvaW1nLzIwMjAwNjAyMTYwNzQyLnBuZw?x-oss-process=image/format,png
[format_png 2]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vamlhbWlueHUvQmxvZ0ltYWdlL3Jhdy9tYXN0ZXIvaW1nLzIwMjAwNjAyMTYwNzU2LnBuZw?x-oss-process=image/format,png
[format_png 3]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vamlhbWlueHUvQmxvZ0ltYWdlL3Jhdy9tYXN0ZXIvaW1nLzIwMjAwNjAyMTYwODQ4LnBuZw?x-oss-process=image/format,png
[format_png 4]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vamlhbWlueHUvQmxvZ0ltYWdlL3Jhdy9tYXN0ZXIvaW1nLzIwMjAwNjAyMTYwOTA4LnBuZw?x-oss-process=image/format,png
[format_png 5]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vamlhbWlueHUvQmxvZ0ltYWdlL3Jhdy9tYXN0ZXIvaW1nLzIwMjAwNjAyMTYwOTI1LnBuZw?x-oss-process=image/format,png
[format_png 6]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vamlhbWlueHUvQmxvZ0ltYWdlL3Jhdy9tYXN0ZXIvaW1nLzIwMjAwNjAyMTYxMjI3LnBuZw?x-oss-process=image/format,png
[format_png 7]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vamlhbWlueHUvQmxvZ0ltYWdlL3Jhdy9tYXN0ZXIvaW1nLzIwMjAwNjAyMTYxMzQ0LnBuZw?x-oss-process=image/format,png
[format_png 8]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vamlhbWlueHUvQmxvZ0ltYWdlL3Jhdy9tYXN0ZXIvaW1nLzIwMjAwNjAyMTYxMzU4LnBuZw?x-oss-process=image/format,png
[format_png 9]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vamlhbWlueHUvQmxvZ0ltYWdlL3Jhdy9tYXN0ZXIvaW1nLzIwMjAwNjAyMTYxNDEzLnBuZw?x-oss-process=image/format,png
[format_png 10]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vamlhbWlueHUvQmxvZ0ltYWdlL3Jhdy9tYXN0ZXIvaW1nLzIwMjAwNjAyMTYxNDI4LnBuZw?x-oss-process=image/format,png
[format_png 11]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vamlhbWlueHUvQmxvZ0ltYWdlL3Jhdy9tYXN0ZXIvaW1nLzIwMjAwNjAyMTYxNTA4LnBuZw?x-oss-process=image/format,png
[format_png 12]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vamlhbWlueHUvQmxvZ0ltYWdlL3Jhdy9tYXN0ZXIvaW1nLzIwMjAwNjAyMTU1ODEzLnBuZw?x-oss-process=image/format,png
[format_png 13]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vamlhbWlueHUvQmxvZ0ltYWdlL3Jhdy9tYXN0ZXIvaW1nLzIwMjAwNjAyMTU1ODQxLnBuZw?x-oss-process=image/format,png
[format_png 14]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vamlhbWlueHUvQmxvZ0ltYWdlL3Jhdy9tYXN0ZXIvaW1nLzIwMjAwNjAyMTcwOTIxLnBuZw?x-oss-process=image/format,png
[format_png 15]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vamlhbWlueHUvQmxvZ0ltYWdlL3Jhdy9tYXN0ZXIvaW1nLzIwMjAwNjAyMTcxNjE2LnBuZw?x-oss-process=image/format,png
[format_png 16]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vamlhbWlueHUvQmxvZ0ltYWdlL3Jhdy9tYXN0ZXIvaW1nLzIwMjAwNjAyMTcxNzAxLnBuZw?x-oss-process=image/format,png
[format_png 17]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vamlhbWlueHUvQmxvZ0ltYWdlL3Jhdy9tYXN0ZXIvaW1nLzIwMjAwNjAyMTczODQzLnBuZw?x-oss-process=image/format,png
[format_png 18]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vamlhbWlueHUvQmxvZ0ltYWdlL3Jhdy9tYXN0ZXIvaW1nLzIwMjAwNjAzMTEyNjQxLnBuZw?x-oss-process=image/format,png
[format_png 19]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vamlhbWlueHUvQmxvZ0ltYWdlL3Jhdy9tYXN0ZXIvaW1nLzIwMjAwNjAzMDkzMjIwLnBuZw?x-oss-process=image/format,png
[format_png 20]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vamlhbWlueHUvQmxvZ0ltYWdlL3Jhdy9tYXN0ZXIvaW1nLzIwMjAwNjAzMTEwNzIzLnBuZw?x-oss-process=image/format,png
[format_png 21]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vamlhbWlueHUvQmxvZ0ltYWdlL3Jhdy9tYXN0ZXIvaW1nLzIwMjAwNjAzMTEwNzAzLnBuZw?x-oss-process=image/format,png
[format_png 22]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vamlhbWlueHUvQmxvZ0ltYWdlL3Jhdy9tYXN0ZXIvaW1nLzIwMjAwNjAzMTEyNzAzLnBuZw?x-oss-process=image/format,png
[format_png 23]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vamlhbWlueHUvQmxvZ0ltYWdlL3Jhdy9tYXN0ZXIvaW1nLzIwMjAwNjAzMTEyNzM4LnBuZw?x-oss-process=image/format,png
[format_png 24]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vamlhbWlueHUvQmxvZ0ltYWdlL3Jhdy9tYXN0ZXIvaW1nLzIwMjAwNjA0MTAyMjIwLnBuZw?x-oss-process=image/format,png
[format_png 25]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9naXRlZS5jb20vamlhbWlueHUvQmxvZ0ltYWdlL3Jhdy9tYXN0ZXIvaW1nLzIwMjAwNjA0MTAxODMzLnBuZw?x-oss-process=image/format,png