RGBD数据集--Princeton Tracking Benchmark

╰+攻爆jí腚メ 2023-02-13 11:52 23阅读 0赞

原文链接：[RGB-D][]

This is an incomplete list of datasets which were captured using a Kinect or similar devices. I initially began it to keep track of semantically labelled datasets, but I have now also included some camera tracking and object pose estimation datasets. I ultimately aim to keep track of all Kinect-style datasets available for researchers to use.

Where possible links have been added to project or personal pages. Where I have not been able to find these I have used a direct link to the data

Please send suggestions for additions and corrections to me at m.firman <at> cs.ucl.ac.uk.

This page is automatically generated from a [YAML file][], and was last updated on 26 November, 2014.

## Turntable data ##

These datasets capture objects under fairly controlled conditions. Bigbird is the most advanced in terms of quality of image data and camera poses, while the RGB-D object dataset is the most extensive.

### RGBD Object dataset ###

**Introduced:** ICRA 2011

**Device:** Kinect v1

**Description:** 300 instances of household objects, in 51 categories. 250,000 frames in total

**Labelling:** Category and instance labelling. Includes auto-generated masks, but no exact 6DOF pose information.

**Download:** [Project page][]

### Bigbird dataset ###

**Introduced:** ICRA 2014

**Device:** Kinect v1 and DSLR

**Description:** 100 household objects

**Labelling:** Instance labelling. Masks, ground truth poses, registered mesh.

**Download:** [Project page][Project page 1]

## Segmentation and pose estimation under controlled conditions ##

These datasets include objects arranged in controlled conditions. Clutter may be present. CAD or meshed models of the objects may or may not be provided. Most provide 6DOF ground truth pose for each object.

### Object segmentation dataset ###

**Introduced:** IROS 2012

**Device:** Kinect v1

**Description:** 111 RGBD images of stacked and occluding objects on table.

**Labelling:** Per-pixel segmentation into objects.

**Download:** [Project page][Project page 2]

### Willow Garage Dataset ###

**Introduced:** 2011

**Device:** Kinect v1

**Description:** Around 160 frames of household objects on a board in controlled environment.

**Labelling:** 6DOF pose for each object, taken from board calibration. Per-pixel labelling.

**Download:** [Project page][Project page 3]

### '3D Model-based Object Recognition and Segmentation in Cluttered Scenes' ###

**Introduced:** IJCV 2009

**Device:** Minolta Vivid 910 (only depth, no RGB!)

**Description:** 50 frames depicting five objects in various occluding poses. No background clutter in any image.

**Labelling:** Pose and per-point labelling information. 3D mesh models of each of the 5 objects.

**Download:** [Project page][Project page 4]

### 'A Global Hypotheses Verifcation Method for 3D Object Recognition' ###

**Introduced:** ECCV 2012

**Device:** Kinect v1

**Description:** 50 Kinect frames, library of 35 objects

**Labelling:** 6DOF GT of each object (unsure how this was gathered). No per-pixel labelling.

**Download:** [Direct link][]

### 'Model Based Training, Detection and Pose Estimation of Texture-Less 3D Objects in Heavily Cluttered Scenes' ###

**Introduced:** ACCV 2012

**Device:** Kinect v1

**Description:** 18,000 Kinect images, library of 15 objects.

**Labelling:** 6DOF pose for each object in each image. No per-pixel labelling.

**Download:** [Project page][Project page 5]

## Kinect data from the real world ##

### RGBD Scenes dataset ###

**Introduced:** ICRA 2011

**Device:** Kinect v1

**Description:** Real indoor scenes, featuring objects from the RGBD object dataset 'arranged' on tables, countertops etc. Video sequences of 8 scenes.

**Labelling:** Per-frame bounding boxes for objects from RGBD object dataset. Other objects not labelled.

**Download:** [Project page][Project page 6]

### RGBD Scenes dataset v2 ###

**Introduced:** ICRA 2014

**Device:** Kinect v1

**Description:** A second set of real indoor scenes featuring objects from the RGBD object dataset. Video sequences of 14 scenes, together with stitched point clouds and camera pose estimations.

**Labelling:** Labelling of points in stitched cloud into one of 9 classes (objects and furniture), plus background.

**Download:** [Project page][Project page 7]

### 'Object Disappearance for Object Discovery' ###

**Introduced:** IROS 2012

**Device:** Kinect v1

**Description:** Three datasets: Small, with still images. Medium, video data from an office environement. Large, video over several rooms. Large dataset has 7 unique objects seen in 397 frames. Data is in ROS bag format.

**Labelling:** Ground truth object segmentations.

**Download:** [Project page][Project page 8]

### 'Object Discovery in 3D scenes via Shape Analysis' ###

**Introduced:** ICRA 2014

**Device:** Kinect v1

**Description:** KinFu meshes of 58 very cluttered indoor scenes.

**Labelling:** Ground truth binary labelling (object/not object) performed on segments proposed by the algorithm, with no labelling on the mesh.

**Download:** [Project page][Project page 9]

### Cornell-RGBD-Dataset ###

**Introduced:** NIPS 2011

**Device:** Kinect v1

**Description:** Multiple RGBD frames from 52 indoor scenes. Stitched point clouds (using RGBDSLAM).

**Labelling:** Per-point object-level labelling on the stitched clouds.

**Download:** [Project page][Project page 10]

### NYU Dataset v1 ###

**Introduced:** ICCV 2011 Workshop on 3D Representation and Recognition

**Device:** Kinect v1

**Description:** Around 51,000 RGBD frames from indoor scenes such as bedrooms and living rooms. Note that the updated NYU v2 dataset is typically used instead of this earlier version.

**Labelling:** Dense multi-class labelling for 2283 frames.

**Download:** [Project page][Project page 11]

### NYU Dataset v2 ###

**Introduced:** ECCV 2012

**Device:** Kinect v1

**Description:** ~408,000 RGBD images from 464 indoor scenes, of a somewhat larger diversity than NYU v1. Per-frame accelerometer data.

**Labelling:** Dense labelling of objects at a class and instance level for 1449 frames. Instance labelling is not carried across scenes. This 1449 subset is the dataset typically used in experiments.

**Download:** [Project page][Project page 12]

### 'Object Detection and Classification from Large-Scale Cluttered Indoor Scans' ###

**Introduced:** Eurographics 2014

**Device:** Faro Lidar scanner

**Description:** Faro lidar scans of ~40 academic offices, with 2-3 scans per office. Each scan is 0.25GB-2GB. Scans include depth and RGB.

**Labelling:** No labelling present. The labelling shown in the exemplar image is their algorithm output.

**Download:** [Project page][Project page 13]

### SUN3D ###

**Introduced:** ICCV 2013

**Device:** Kinect v1

**Description:** Videos of indoor scenes, registered into point clouds.

**Labelling:** Polygons of semantic class and instance labels on frames propagated through video.

**Download:** [Project page][Project page 14]

### B3DO: Berkeley 3-D Object Dataset ###

**Introduced:** ICCV Workshop on Consumer Depth Cameras in Computer Vision 2011

**Device:** Kinect v1

**Description:** Aim is to crowdsource collection of Kinect data, to be included in future releases. Version 1 has 849 images, from 75 scenes.

**Labelling:** Bounding box labelling at a class level.

**Download:** [Project page][Project page 15]

## SLAM, registration and camera pose estimation ##

### TUM Benchmark Dataset ###

**Introduced:** IROS 2012

**Device:** Kinect v1

**Description:** Many different scenes and scenarios for tracking and mapping, including reconstruction, robot kidnap etc.

**Labelling:** 6DOF ground truth from motion capture system with 10 cameras.

**Download:** [Project page][Project page 16]

### Microsoft 7-scenes dataset ###

**Introduced:** CVPR 2013

**Device:** Kinect v1

**Description:** Kinect video from 7 indoor scenes.

**Labelling:** 6DOF 'ground truth' from Kinect Fusion.

**Download:** [Project page][Project page 17]

### IROS 2011 Paper Kinect Dataset ###

**Introduced:** IROS 2011

**Device:** Kinect v1

**Description:** Lab-based setup. The aim seems to be to track the motion of camera.

**Labelling:** 6DOF ground truth from Vicon system

**Download:** [Project page][Project page 18]

### 'When Can We Use KinectFusion for Ground Truth Acquisition?' ###

**Introduced:** Workshop on Color-Depth Camera Fusion in Robotics, IROS 2012

**Device:** Kinect v1

**Description:** A set of 57 scenes, captured from natural environments and from artificial shapes. Each scene has a 3D mesh, volumetric data and registered depth maps.

**Labelling:** Frame-to-frame transformations as computed from KinectFusion. The 'office' and 'statue' scenes have LiDAR ground truth.

**Download:** [Project page][Project page 19]

### DAFT Dataset ###

**Introduced:** ICPR 2012

**Device:** Kinect v1

**Description:** A few short sequences of different planar scenes captured under various camera motions. Used to demonstrate repeatability of feature points under transformations.

**Labelling:** Camera motion type. 2D homographies between the planar scene in different images.

**Download:** [Project page][Project page 20]

### ICL-NUIM Dataset ###

**Introduced:** ICRA 2014

**Device:** Kinect v1 (synthesised)

**Description:** Eight synthetic RGBD video sequences: four from a office scene and four from a living room scene. Simulated camera trajectories are taken from a Kintinuous output from a sensor being moved around a real-world room.

**Labelling:** Camera trajectories for each video. Geometry of the living room scene as an .obj file.

**Download:** [Project page][Project page 21]

### 'Automatic Registration of RGB-D Scans via Salient Directions' ###

**Introduced:** ICCV 2013

**Device:** RGBD Laser scanner

**Description:** Several laser scans taken from each of a European church, city and castle scenes.

**Labelling:** Results of the authors' registration algorithm.

**Download:** [Project page][Project page 22]

### Stanford 3D Scene Dataset ###

**Introduced:** SIGGRAPH 2013

**Device:** Xtion Pro Live (Kinect v1 equivalent)

**Description:** RGBD videos of six indoor and outdoor scenes, together with a dense reconstruction of each scene.

**Labelling:** Estimated camera pose for each frame. No ground truth pose, so not ideal for quantitative evaluation.

**Download:** [Project page][Project page 23]

## Tracking ##

See also some of the human datasets for body and face tracking.

### Princeton Tracking Benchmark ###

**Introduced:** ICCV 2013

**Device:** Kinect v1

**Description:** 100 RGBD videos of moving objects such as humans, balls and cars.

**Labelling:** Per-frame bounding box covering target object only.

**Download:** [Project page][Project page 24]

## Datasets involving humans: Body and hands ##

### Cornell Activity Datasets: CAD-60 and CAD-120 ###

**Introduced:** PAIR 2011/IJRR 2013

**Device:** Kinect v1

**Description:** Videos of humans performing activities

**Labelling:** Each video given at least one label, such as eating, opening or working on computer. Skeleton joint position and orientation labelled on each frame.

**Download:** [Project page][Project page 25]

### RGB-D Person Re-identification Dataset ###

**Introduced:** First International Workshop on Re-Identification 2012

**Device:** Kinect v1

**Description:** Front and back poses of 79 people walking forward in different poses.

**Labelling:** In addition to the per-person label, the dataset provides foreground masks, skeletons, 3D meshes and an estimate of the floor.

**Download:** [Project page][Project page 26]

### Sheffield KInect Gesture (SKIG) Dataset ###

**Introduced:** IJCAI 2013

**Device:** Kinect v1

**Description:** Total of 1080 Kinect videos of six people performing one of 10 hand gesture sequences, such as 'triangle' or 'comehere'. Sequences captured under a variety of illumination and background conditions.

**Labelling:** The gesture being performed in each sequence.

**Download:** [Project page][Project page 27]

### RGB-D People Dataset ###

**Introduced:** IROS 2011

**Device:** Kinect v1

**Description:** 3000+ frames of people walking and standing in a university hallway, captured from three Kinects.

**Labelling:** Per-frame bounding box annotations of individual people, together with a \`visibility' measure.

**Download:** [Project page][Project page 28]

### 50 Salads ###

**Introduced:** UbiComp 2013

**Device:** Kinect v1

**Description:** Over 4 hours of video of 25 people preparing 2 mixed salads each

**Labelling:** Accelerometer data from sensors attached to cooking utensils, and labelling of steps in the recipes.

**Download:** [Project page][Project page 29]

### Microsoft Research Cambridge-12 Kinect gesture data set ###

**Introduced:** CHI 2012

**Device:** Kinect v1

**Description:** 594 sequences and 719,359 frames of 30 people performing 12 gestures.

**Labelling:** Gesture performed in each video sequence, plus motion tracking of human joint locations.

**Download:** [Project page][Project page 30]

### UR Fall Detection Dataset ###

**Introduced:** Computer Vision Theory and Applications 2014

**Device:** Kinect v1

**Description:** Videos of people falling over. Consists of 60 sequences recorded with two Kinects.

**Labelling:** Accelerometer data from device attached to subject.

**Download:** [Project page][Project page 31]

### RGBD-HuDaAct ###

**Introduced:** ICCV Workshops 2011

**Device:** Kinect v1

**Description:** 30 different humans each performing the same 12 activities, e.g. 'eat a meal'. Also include a random 'background' activity. All performed in a lab environment. Around 5,000,000 frames in total.

**Labelling:** Which activity being performed in each sequence.

**Download:** [Project page][Project page 32]

### Human3.6M ###

**Introduced:** PAMI 2014

**Device:** SwissRanger time-of-flight (+ 2D cameras)

**Description:** 11 different humans performing 17 different activities. Data comes from four calibrated video cameras, 1 time-of-flight camera and (static) 3D laser scans of the actors.

**Labelling:** 2D and 3D human joint positions, obtained from a Vicon motion capture system.

**Download:** [Project page][Project page 33]

## Datasets involving humans: Head and face ##

### Biwi Kinect Head Pose Database ###

**Introduced:** IJCV 2013

**Device:** Kinect v1

**Description:** 15K images of 20 different people moving their heads in different directions.

**Labelling:** 3D position of the head and its rotation, acquired using 'faceshift' software.

**Download:** [Project page][Project page 34]

### Eurecom Kinect Face Dataset ###

**Introduced:** ACCV Workshop on Computer Vision with Local Binary Pattern Variants 2012

**Device:** Kinect v1

**Description:** Images of faces captured under laboritory conditions, with different levels of occlusion and illumination, and with different facial expressions.

**Labelling:** In addition to occlusion and expression type, each image is manually labelled with the position of six facial landmarks.

**Download:** [Project page][Project page 35]

### 3D Mask Attack Dataset ###

**Introduced:** Biometrics: Theory, Applications and Systems 2013

**Device:** Kinect v1

**Description:** 76500 frames of 17 different people, facing the camera against a plain background. Two sets of the data are captured on the real subjects two weeks apart, while the final set consists of a single person wearing a fake face mask of the 17 different people.

**Labelling:** Which user is in each frame. Which images are real and which are spoofed. Manually labelled eye positions.

**Download:** [Project page][Project page 36]

### Biwi 3D Audiovisual Corpus of Affective Communication - B3D(AC)^2 ###

**Introduced:** IEEE Transactions on Multimedia 2010

**Device:** Custom active stereo setup

**Description:** Simultaneous audio and visual recordings of 1109 sentences spoken by 14 different people. Each sentence spoken neutrally and with an emotion. Depth images converted to 3D mesh.

**Labelling:** Perceived emotions for each recording. Audio labelled with phonemes.

**Download:** [Project page][Project page 37]

### ETH Face Pose Range Image Data Set ###

**Introduced:** CVPR 2008

**Device:** Custom active stereo setup

**Description:** 10,545 images of 20 different people turning their head.

**Labelling:** Nose potition and coordinate frame at the nose.

**Download:** [Project page][Project page 38]

[RGB-D]: http://www.cnblogs.com/alexanderkun/p/4593124.htm
[YAML file]: http://www0.cs.ucl.ac.uk/staff/M.Firman/RGBDdatasets/rgbddatset.yaml
[Project page]: http://rgbd-dataset.cs.washington.edu/
[Project page 1]: http://rll.berkeley.edu/bigbird/
[Project page 2]: http://www.acin.tuwien.ac.at/?id=289
[Project page 3]: http://www.acin.tuwien.ac.at/forschung/v4r/mitarbeiterprojekte/willow/
[Project page 4]: http://www.csse.uwa.edu.au/~ajmal/recognition.html
[Direct link]: http://users.acin.tuwien.ac.at/aaldoma/datasets/ECCV.zip
[Project page 5]: http://campar.in.tum.de/Main/StefanHinterstoisser
[Project page 6]: http://rgbd-dataset.cs.washington.edu/dataset/rgbd-scenes/
[Project page 7]: http://rgbd-dataset.cs.washington.edu/dataset/rgbd-scenes-v2/
[Project page 8]: http://wiki.ros.org/Papers/IROS2012_Mason_Marthi_Parr
[Project page 9]: http://cs.stanford.edu/people/karpathy/discovery/
[Project page 10]: http://pr.cs.cornell.edu/sceneunderstanding/data/data.php
[Project page 11]: http://cs.nyu.edu/~silberman/datasets/nyu_depth_v1.html
[Project page 12]: http://cs.nyu.edu/~silberman/datasets/nyu_depth_v2.html
[Project page 13]: http://www.ifi.uzh.ch/vmml/publications/ObjDetandClas.html
[Project page 14]: http://sun3d.cs.princeton.edu/
[Project page 15]: http://kinectdata.com/
[Project page 16]: http://vision.in.tum.de/data/datasets/rgbd-dataset
[Project page 17]: http://research.microsoft.com/en-us/projects/7-scenes/
[Project page 18]: http://projects.asl.ethz.ch/datasets/doku.php?id=Kinect:iros2011Kinect
[Project page 19]: http://hci.iwr.uni-heidelberg.de//Benchmarks/document/kinectFusionCapture/
[Project page 20]: http://ias.cs.tum.edu/people/gossow/rgbd
[Project page 21]: http://www.doc.ic.ac.uk/~ahanda/VaFRIC/iclnuim.html
[Project page 22]: http://www.cvg.ethz.ch/research/saldir-rgbd-registration/
[Project page 23]: http://web.stanford.edu/~qianyizh/projects/scenedata.html
[Project page 24]: http://tracking.cs.princeton.edu/dataset.html
[Project page 25]: http://pr.cs.cornell.edu/humanactivities/data.php
[Project page 26]: http://www.iit.it/en/datasets-and-code/datasets/rgbdid.html
[Project page 27]: http://lshao.staff.shef.ac.uk/data/SheffieldKinectGesture.htm
[Project page 28]: http://www2.informatik.uni-freiburg.de/~spinello/RGBD-dataset.html
[Project page 29]: http://cvip.computing.dundee.ac.uk/datasets/foodpreparation/50salads/
[Project page 30]: http://research.microsoft.com/en-us/um/cambridge/projects/msrc12/
[Project page 31]: http://fenix.univ.rzeszow.pl/~mkepski/ds/uf.html
[Project page 32]: http://adsc.illinois.edu/sites/default/files/files/ADSC-RGBD-dataset-download-instructions.pdf
[Project page 33]: http://vision.imar.ro/human3.6m
[Project page 34]: http://www.vision.ee.ethz.ch/~gfanelli/head_pose/head_forest.html#db
[Project page 35]: http://rgb-d.eurecom.fr/
[Project page 36]: https://www.idiap.ch/dataset/3dmad
[Project page 37]: http://www.vision.ee.ethz.ch/datasets/b3dac2.en.html
[Project page 38]: http://www.vision.ee.ethz.ch/datasets/headposeCVPR08/