RGBD 转换成点云 open3d

约定不等于承诺〃 2022-10-07 11:49 353阅读 0赞

深度图转为点云说白了其实就是坐标系的变换：图像坐标系－－>世界坐标系。变换的约束条件就是相机内参，公式很简单：

![20210618001757374.png][]

,其中x,y,z是点云坐标系，x',y'是图像坐标系，Ｄ为深度值。

在进行上述转换之前必须对x',y'进行undistort（反畸变）运算，即便如此最终得到的点云数据还是存在误差的。棋盘标定出的内参本身是带有误差的，它是一种近似逼近值，也就是说它不能真实的映射相机内部结构。

作者：崔世界  
链接：https://www.zhihu.com/question/268488496/answer/353439189  
来源：知乎

网上一个版本的转换：

[http://ziyedy.top/page/python-depth-points.html][http_ziyedy.top_page_python-depth-points.html]

def depth2pc(depth_img):
        """
        深度图转点云数据
        图像坐标系 -> 世界坐标系 
        :param depth_img: 深度图
        :return: 点云数据 N*3
        """
        
        # 相机内参
        cam_fx = 1120.12
        cam_fy = 1120.12
        cam_cx = 640.5
        cam_cy = 360.5
        factor = 1
    
    	# 逐点处理，此过程可以使用numpy优化
        m, n = depth_img.shape
        point_cloud = []
        for v in range(m):
            for u in range(n):
                if depth_img[v, u] == 0:
                    continue
                depth = depth_img[v, u]
                p_z = depth / factor
                p_x = (u - cam_cx) * p_z / cam_fx
                p_y = (v - cam_cy) * p_z / cam_fy
                point_cloud.append([p_x, p_y, p_z])
    
        point_cloud = np.array(point_cloud)
    
        return point_cloud

最近做实验，想实现深度估计到三维点云再到网格的生成，第一步做出深度图结合RGB图生成三维点云

RGB室内图：

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl8zODI5ODM2Mw_size_16_color_FFFFFF_t_70][]

Depth Image：（隐隐约约能过看到点什么）

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl8zODI5ODM2Mw_size_16_color_FFFFFF_t_70 1][]

生成的结果：

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl8zODI5ODM2Mw_size_16_color_FFFFFF_t_70 2][]

转个角度：

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl8zODI5ODM2Mw_size_16_color_FFFFFF_t_70 3][]

结果很粗糙，感觉这个方法不是很理想，也就是通过深度估计得到的深度图转换成点云的想法不太靠谱！

代码如下：（需要安装open3d）

from PIL import Image
    import pandas as pd
    import numpy as np
    from open3d import read_point_cloud, draw_geometries
    import time
     
     
    class point_cloud_generator():
     
        def __init__(self, rgb_file, depth_file, pc_file, focal_length, scalingfactor):
            self.rgb_file = rgb_file
            self.depth_file = depth_file
            self.pc_file = pc_file
            self.focal_length = focal_length
            self.scalingfactor = scalingfactor
            self.rgb = Image.open(rgb_file)
            self.depth = Image.open(depth_file).convert('I')
            self.width = self.rgb.size[0]
            self.height = self.rgb.size[1]
     
        def calculate(self):
            t1=time.time()
            depth = np.asarray(self.depth).T
            self.Z = depth / self.scalingfactor
            X = np.zeros((self.width, self.height))
            Y = np.zeros((self.width, self.height))
            for i in range(self.width):
                X[i, :] = np.full(X.shape[1], i)
     
            self.X = ((X - self.width / 2) * self.Z) / self.focal_length
            for i in range(self.height):
                Y[:, i] = np.full(Y.shape[0], i)
            self.Y = ((Y - self.height / 2) * self.Z) / self.focal_length
     
            df=np.zeros((6,self.width*self.height))
            df[0] = self.X.T.reshape(-1)
            df[1] = -self.Y.T.reshape(-1)
            df[2] = -self.Z.T.reshape(-1)
            img = np.array(self.rgb)
            df[3] = img[:, :, 0:1].reshape(-1)
            df[4] = img[:, :, 1:2].reshape(-1)
            df[5] = img[:, :, 2:3].reshape(-1)
            self.df=df
            t2=time.time()
            print('calcualte 3d point cloud Done.',t2-t1)
     
        def write_ply(self):
            t1=time.time()
            float_formatter = lambda x: "%.4f" % x
            points =[]
            for i in self.df.T:
                points.append("{} {} {} {} {} {} 0\n".format
                              (float_formatter(i[0]), float_formatter(i[1]), float_formatter(i[2]),
                               int(i[3]), int(i[4]), int(i[5])))
     
            file = open(self.pc_file, "w")
            file.write('''ply
            format ascii 1.0
            element vertex %d
            property float x
            property float y
            property float z
            property uchar red
            property uchar green
            property uchar blue
            property uchar alpha
            end_header
            %s
            ''' % (len(points), "".join(points)))
            file.close()
     
            t2=time.time()
            print("Write into .ply file Done.",t2-t1)
     
        def show_point_cloud(self):
            pcd = read_point_cloud(self.pc_file)
            draw_geometries([pcd])
     
    a = point_cloud_generator('01446_colors.png', '01446_depth.png', '01446_3d.ply',
                              focal_length=300, scalingfactor=1000)
    a.calculate()
    a.write_ply()
    a.show_point_cloud()
    df = a.df
    np.save('pc.npy',df)

————————————————  
版权声明：本文为CSDN博主「冷心笑看丽美人」的原创文章，遵循CC 4.0 BY-SA版权协议，转载请附上原文出处链接及本声明。  
原文链接：

[20210618001757374.png]: /images/20221005/116254264a774c3e8f680222e7007956.png
[http_ziyedy.top_page_python-depth-points.html]: http://ziyedy.top/page/python-depth-points.html
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl8zODI5ODM2Mw_size_16_color_FFFFFF_t_70]: /images/20221005/4ad557987d0d47b5bd1807a8f3c53ac5.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl8zODI5ODM2Mw_size_16_color_FFFFFF_t_70 1]: /images/20221005/4e4635a472f24e97b1fe66f798a83bff.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl8zODI5ODM2Mw_size_16_color_FFFFFF_t_70 2]: /images/20221005/331cb0d667c44f3d905abfef82da8f1e.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl8zODI5ODM2Mw_size_16_color_FFFFFF_t_70 3]: /images/20221005/8ecd466c4167416c8da7998c14acabb6.png