2D Pose人体关键点实时检测(Python/Android /C++ Demo)

怼烎@ 2023-01-13 06:21 98阅读 0赞

# 2D Pose人体关键点实时检测(Python/Android /C++ Demo) #

1.人体关键点数据集

（1）COCO数据集

（2）MPII数据集

（3）关键点示意图

2.人体关键点Demo（Python版本）

3.人体关键点Demo（C++版本）

4.人体关键点Demo（Android版本）

5.人体关键点检测效果

6.人脸检测+人体检测C++ Android实现

--------------------

人体关键点检测（Human Keypoints Detection）又称为人体姿态估计2D Pose，是计算机视觉中一个相对基础的任务，是人体动作识别、行为分析、人机交互等的前置任务。一般情况下可以将人体关键点检测细分为单人/多人关键点检测、2D/3D关键点检测，同时有算法在完成关键点检测之后还会进行关键点的跟踪，也被称为人体姿态跟踪。

本博客提供2D Pose的Python代码，以及C++版本的推理代码，还提供[Android Demo APP][]，已经集成了轻量化版本的`人体检测模型`和`人体姿态估计模型`，在普通手机可实时检测。

**Repo推荐**：

1.  个人repo(Python版): [https://github.com/PanJinquan/Human-Keypoints-Detection][https_github.com_PanJinquan_Human-Keypoints-Detection]
2.  个人repo(C++版):[https://github.com/PanJinquan/Human-Pose-Estimation-Lite-cpp][https_github.com_PanJinquan_Human-Pose-Estimation-Lite-cpp]
3.  Android Demo支持CPU多线程和GPU加速，可实时检测)(APP Demo，完整项目可微信联系，仅供学习交)：[https://download.csdn.net/download/guyuealian/25674732][Android Demo APP]
4.  《Simple Baselines for Human Pose Estimation and Tracking》[https://github.com/Microsoft/human-pose-estimation.pytorch][https_github.com_Microsoft_human-pose-estimation.pytorch]
5.  《[Deep High-Resolution Representation Learning for Human Pose Estimation][]》[https://github.com/leoxiaobin/deep-high-resolution-net.pytorch][https_github.com_leoxiaobin_deep-high-resolution-net.pytorch]

--------------------

## 1.人体关键点数据集 ##

### （1）COCO数据集 ###

> 下载地址：http://cocodataset.org/
> 
> COCO人体关键点标注，最多标注全身的17个关键点，平均一幅图像2个人，最多有13个人；
> 
> 人体关键点标注，每个人体关键点个数的分布情况，其中11-15这个范围的人体是最多的，有接近70000人，6-10其次，超过40000人，后面依次为16-17,2-5,1.

### （2）MPII数据集 ###

> 下载地址：http://human-pose.mpi-inf.mpg.de/\#download
> 
>  人体关键点标注了全身16个关键点及其是否可见的信息，人数：train有28821，test有11701，有409种人类活动；使用mat的struct格式；行人框使用center和scale标注，人体尺度关于200像素高度。也就是除过了200

### **（3）关键点示意图** ###

<table> 
 <tbody> 
  <tr> 
   <td style="width:60px;">数据集</td> 
   <td style="width:418px;">关键点示意图&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp;</td> 
   <td style="width:342px;">关键点说明</td> 
  </tr> 
  <tr> 
   <td style="width:60px;">COCO</td> 
   <td style="width:418px;"><img alt="" height="641" src="https://img-blog.csdnimg.cn/20210416164539283.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2d1eXVlYWxpYW4=,size_16,color_FFFFFF,t_70" width="329"></td> 
   <td style="width:342px;"> <p># 图像左右翻转时，成对的关键点（训练时用于数据增强）</p> <p>flip_pairs=[[1, 2], [3, 4], [5, 6], [7, 8],[9, 10], [11, 12], [13, 14], [15, 16]]</p> <p></p> <p># 关键点连接线序号（用于绘制图像）</p> <p>skeleton =[[15, 13], [13, 11], [16, 14], [14, 12], [11, 12], [5, 11], [6, 12], [5, 6], [5, 7], [6, 8], [7, 9], [8, 10], [0, 1], [0, 2], [1, 3], [2, 4]]</p> <p></p> <p># 每个关键点序号对应人体关键点的意义</p> <p>"keypoints": { 0: "nose", 1: "left_eye", 2: "right_eye", 3: "left_ear", 4: "right_ear", 5: "left_shoulder", 6: "right_shoulder", 7: "left_elbow", 8: "right_elbow", 9: "left_wrist", 10: "right_wrist", 11: "left_hip", 12: "right_hip", 13: "left_knee", 14: "right_knee", 15: "left_ankle", 16: "right_ankle" }</p> </td> 
  </tr> 
  <tr> 
   <td style="width:60px;">MPII</td> 
   <td style="width:418px;"><img alt="" height="640" src="https://img-blog.csdnimg.cn/20210416165414715.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2d1eXVlYWxpYW4=,size_16,color_FFFFFF,t_70" width="317"></td> 
   <td style="width:342px;"> <p># 图像左右翻转时，成对的关键点（训练时用于数据增强）<br> flip_pairs = [[0, 5], [1, 4], [2, 3], [10, 15], [11, 14], [12, 13]]</p> <p># 关键点连接线序号（用于绘制图像）</p> <p><span style="color:#000000;">skeleton=[[0, 1], [1, 2], [3, 4], [4, 5], [2, 6], [6, 3], [12, 11], [7, 12], [11, 10], [13, 14], [14, 15], [8, 9], [8, 7], [6, 7], [7, 13]]</span></p> <p># 每个关键点序号对应人体关键点的意义<br> "keypoints": {0: "r_ankle", 1: "r_knee", 2: "r_hip", 3: "l_hip", 4: "l_knee", 5: "l_ankle", 6: "pelvis", 7: "thorax", 8: "upper_neck", 9: "head_top", 10: " r_wrist", 11: "r_elbow", 12: "r_shoulder", 13: "l_shoulder", 14: "l_elbow", 15: "l_wrist" }</p> </td> 
  </tr> 
  <tr> 
   <td style="width:60px;">human3.6M</td> 
   <td style="width:418px;"><img alt="" height="574" src="https://img-blog.csdnimg.cn/20210416165708163.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2d1eXVlYWxpYW4=,size_16,color_FFFFFF,t_70" width="549"></td> 
   <td style="width:342px;"></td> 
  </tr> 
  <tr> 
   <td style="width:60px;">kinect</td> 
   <td style="width:418px;"><img alt="" height="428" src="https://img-blog.csdnimg.cn/20210416165742294.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2d1eXVlYWxpYW4=,size_16,color_FFFFFF,t_70" width="429"></td> 
   <td style="width:342px;"> <pre><code>JointType_SpineBase = 0 # 脊柱底
JointType_SpineMid = 1 # 脊柱中间
JointType_Neck = 2 # 脖子
JointType_Head = 3 # 额头
JointType_ShoulderLeft = 4
JointType_ElbowLeft = 5
JointType_WristLeft = 6
JointType_HandLeft = 7
JointType_ShoulderRight = 8
JointType_ElbowRight = 9
JointType_WristRight = 10
JointType_HandRight = 11
JointType_HipLeft = 12
JointType_KneeLeft = 13
JointType_AnkleLeft = 14
JointType_FootLeft = 15
JointType_HipRight = 16
JointType_KneeRight = 17
JointType_AnkleRight = 18
JointType_FootRight = 19
JointType_SpineShoulder = 20
JointType_HandTipLeft = 21
JointType_ThumbLeft = 22
JointType_HandTipRight = 23
JointType_ThumbRight = 24
JointType_Count = 25</code></pre> </td> 
  </tr> 
 </tbody> 
</table>

## 2.人体关键点Demo（Python版本） ##

git clone https://github.com/PanJinquan/Human-Keypoints-Detection

目前仅仅提供推理代码，相关训练代码和数据，可以跟博主交流一起学习，可以支持自定义人体关键点进行训练。

**人体关键点**和**手部关键点**原理差不多的，只要有相关数据即可训练模型。

运行demo：

python demo.py

## 3.人体关键点Demo（C++版本） ##

git clone https://github.com/PanJinquan/Human-Pose-Estimation-Lite-cpp

运行demo：

bash build.sh

这是轻量化版本的人体姿态估计(2D Pose)C++推理代码，推理框架使用[TNN][]

*  轻量化模型是基于MobileNet V2的改进版本
 *  使用COCO的数据集进行训练，也可以支持MPII数据
 *  支持OpenCL模型推理加速，在普通手机可实时检测
 *  该仓库并未集成`人体检测模型`，Pose检测输入是原图，使用人体检测框并进行裁剪，Pose检测效果会更好
 *  关于轻量化版本的人体检测检测模型，可参考[Object-Detection-Lite-cpp][]
 *  注意：纯C++版本速度比较慢，需要配置OpenCL方可实时检测
 *  仅用于学习交流，并未进行过多的性能优化
 *  模型训练代码暂时未提供

## 4.人体关键点Demo（Android版本） ##

*  Android Demo已经集成了轻量化版本的`人体检测模型`和`人体姿态估计模型`，在普通手机可实时检测
 *   Android Demo,支持CPU多线程和GPU加速，可实时检测(APP Demo，完整项目可微信联系，仅供学习交)：[Android人体检测和人体关键点检测APP.zip-互联网文档类资源-CSDN下载Android人体检测和人体关键点检测APP，《2DPose人体关键点实时检测(Python/An更多下载资源、学习资料请访问CSDN下载频道.![favicon32.ico][]https://download.csdn.net/download/guyuealian/25674732][Android Demo APP]

<table style="width:288px;"> 
 <thead> 
  <tr> 
   <th style="width:285px;"><a href="https://github.com/PanJinquan/Human-Pose-Estimation-Lite-cpp/blob/master/data/app" title="Android Demo" rel="nofollow">Android Demo</a>&nbsp;CPU:70ms,GPU:50ms</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td style="text-align:center;width:285px;"><img alt="" src="https://img-blog.csdnimg.cn/20210723091643896.gif"></td> 
  </tr> 
 </tbody> 
</table>

## 5.人体关键点检测效果 ##

## 6.人脸检测+人体检测C++ Android实现 ##

人脸检测+人体检测C++ Android实现，可参考另一篇博客：

[人脸检测+人体检测C++ Android实现\_pan\_jinquan的博客-CSDN博客人脸检测+人体检测C++ Android实现本博客将实现C++版本的人脸检测，人脸关键点检测，人体检测，人脸+人体检测，推理框架采用TNN，在普通Android手机，CPU和GPU都可以达到实时检测的效果人脸检测+人脸关键点检测+人体检测Android Demo APP(非源码，仅供学习交流)链接: https://pan.baidu.com/s/1By43I1DbMa0gBPLObtPZMQ 提取码: msnr尊重原创，转载请注明出处：https://panjinquan.blog..![favicon32.ico][]https://panjinquan.blog.csdn.net/article/details/120688804][C_ Android_pan_jinquan_-CSDN_C_ Android_C_TNN_Android_CPU_GPU_Android Demo APP_ https_pan.baidu.com_s_1By43I1DbMa0gBPLObtPZMQ _ msnr_https_panjinquan.blog.._favicon32.ico_https_panjinquan.blog.csdn.net_article_details_120688804]

![watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBAcGFuX2ppbnF1YW4_size_20_color_FFFFFF_t_70_g_se_x_16][]

[Android Demo APP]: https://download.csdn.net/download/guyuealian/25674732
[https_github.com_PanJinquan_Human-Keypoints-Detection]: https://github.com/PanJinquan/Human-Keypoints-Detection
[https_github.com_PanJinquan_Human-Pose-Estimation-Lite-cpp]: https://github.com/PanJinquan/Human-Pose-Estimation-Lite-cpp
[https_github.com_Microsoft_human-pose-estimation.pytorch]: https://github.com/Microsoft/human-pose-estimation.pytorch
[Deep High-Resolution Representation Learning for Human Pose Estimation]: https://arxiv.org/abs/1902.09212
[https_github.com_leoxiaobin_deep-high-resolution-net.pytorch]: https://github.com/leoxiaobin/deep-high-resolution-net.pytorch
[TNN]: https://github.com/Tencent/TNN
[Object-Detection-Lite-cpp]: https://github.com/PanJinquan/Object-Detection-Lite-cpp
[favicon32.ico]: /images/20221022/c0201b41b9c34b8dba92b389621c23ab.png
[C_ Android_pan_jinquan_-CSDN_C_ Android_C_TNN_Android_CPU_GPU_Android Demo APP_ https_pan.baidu.com_s_1By43I1DbMa0gBPLObtPZMQ _ msnr_https_panjinquan.blog.._favicon32.ico_https_panjinquan.blog.csdn.net_article_details_120688804]: https://panjinquan.blog.csdn.net/article/details/120688804
[watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBAcGFuX2ppbnF1YW4_size_20_color_FFFFFF_t_70_g_se_x_16]: /images/20221022/fa6004090b1248e7ad14d94d94de9e11.png