InScope: A New Real-world 3D Infrastructure-side Collaborative Perception Dataset for Open Traffic Scenarios
This is the official implementation of InScope dataset. "InScope: A New Real-world 3D Infrastructure-side Collaborative Perception Dataset for Open Traffic Scenarios".
Xiaofei Zhang , Yining Li , Jinping Wang , Xiangyi Qin , Ying Shen , Zhengping Fan , Xiaojun Tan†
The ground truth of sequence 0000.
Due to project restrictions, the InScope dataset is made conditionally public. If you need to use the InScope dataset, please fill in the following ./assets/InScope_Dataset_Release_Agreement.docx file and email your full name and affiliation to the contact person. We ask for your information only to ensure the dataset is used for non-commercial purposes.
After downloading the data, please put the data in the following structure:
├── InScope-Sec, InScope_Pri, and InScope datasets
│ ├── ImageSets
| |── train.txt
| |── test.txt
| |── val.txt
│ ├── labels
| |── 000000.txt
| |── 000001.txt
| |── 000002.txt
| |── ...
│ ├── points
| |── 000000.npy
| |── 000001.npy
| |── 000002.npy
| |── ...
├── InScope_track
│ ├── label_02
| |── 0000.txt
| |── 0001.txt
| |── 0002.txt
| |── ...
│ ├── points
| |── 0000
| |── 000000.bin
| |── 000001.bin
| |── 000002.bin
| |── ...
| |── 0001
| |── 0002
| |── ...
│ ├── evaluate_tracking.seqmap
│ ├── evaluate_tracking.seqmap.test
│ ├── evaluate_tracking.seqmap.training
│ ├── evaluate_tracking.seqmap.val
To facilitate researchers' use and understanding, we adapted the InScope dataset to the OpenPCDet framework and provided the corresponding dataset configuration file ./InScope.config
For detection training & inference, you can find instructions in detection_code/openpcdet/README_InScope.md in detail.
All the checkpoints are released in link in the tabels below, you can save them in codes/ckpts/ .
Results of 3D object detection based on the InScope dataset Methods Car AP@0.7 Pedestrian AP@0.5 Cyclist AP@0.5 Truck AP@0.7 mAP40 FPS Download Link PointRCNN 71.75 68.13 62.91 94.50 74.32 4.58 [URL ] 3DSSD 68.00 13.88 36.58 95.08 53.38 11.35 [URL ] SECOND 72.82 47.95 59.91 95.98 69.17 20.58 [URL ] Pointpillar 78.04 35.34 58.46 95.86 66.93 24.51 [URL ] PV-RCNN 75.05 48.37 56.31 94.52 68.56 4.35 [URL ] PV-RCNN++ 80.55 53.31 70.92 95.92 75.18 14.66 [URL ] CenterPoint 77.24 70.45 74.74 96.12 79.64 30.49 [URL ] CenterPoint_RCNN 78.33 71.13 75.23 96.48 80.29 6.55 [URL ]
Results of 3D object detection based on the InScope-Sec, InScope_Pri, and InScope datasets Detection result based on the InScope-Sec Only Methods Car AP@0.7 Pedestrian AP@0.5 Cyclist AP@0.5 Truck AP@0.7 mAP40 FPS Download Link PointRCNN 14.12 23.66 20.62 45.36 25.94 22.94 [URL ] Pointpillar 44.77 33.18 31.42 82.52 47.97 87.72 [URL ] PV-RCNN++ 43.49 34.60 39.94 76.04 48.52 16.67 [URL ] CenterPoint 35.92 37.40 38.24 68.78 45.08 107.53 [URL ]
Detection result based on the InScope_Pri Only Methods Car AP@0.7 Pedestrian AP@0.5 Cyclist AP@0.5 Truck AP@0.7 mAP40 FPS Download Link PointRCNN 61.14 88.80 61.99 48.96 65.22 4.67 [URL ] Pointpillar 67.34 23.82 43.51 91.59 56.57 25.25 [URL ] PV-RCNN++ 72.59 45.26 61.21 91.02 67.52 13.81 [URL ] CenterPoint 61.31 49.62 52.73 82.02 61.42 33.90 [URL ]
Detection result based on the Early Fusion (InScope) Mechanism Methods Car AP@0.7 Pedestrian AP@0.5 Cyclist AP@0.5 Truck AP@0.7 mAP40 FPS Download Link PointRCNN 71.75 68.13 62.91 94.50 74.32 4.58 [URL ] Pointpillar 78.04 35.34 58.46 95.86 66.93 24.33 [URL ] PV-RCNN++ 80.55 53.31 70.92 95.92 75.18 12.45 [URL ] CenterPoint 77.24 70.45 74.74 96.12 79.64 30.49 [URL ]
Detection result based on the Late Fusion Mechanism Methods Car AP@0.7 Pedestrian AP@0.5 Cyclist AP@0.5 Truck AP@0.7 mAP40 FPS Download Link PointRCNN 62.69 61.31 52.31 90.93 66.81 1.32 [URL ]+[URL ] Pointpillar 68.65 31.81 49.92 93.48 60.96 1.81 [URL ]+[URL ] PV-RCNN++ 68.01 53.47 56.95 92.65 67.77 1.21 [URL ]+[URL ] CenterPoint 58.13 50.03 56.01 85.65 62.45 6.40 [URL ]+[URL ]
Detection result based on the Middle Fusion Mechanism Methods Car AP@0.7 Pedestrian AP@0.5 Cyclist AP@0.5 Truck AP@0.7 mAP40 FPS Download Link Point-RCNN - - - - - - Pointpillar - - - - - - PV-RCNN++ 73.78 52.06 62.06 91.89 69.95 13.02 [URL ] CenterPoint 52.74 38.95 51.19 81.73 56.15 15.85 [URL ]
Results of data domain transfer on the car class Source→Target DAIR-V2X-I→KITTI ONCE→KITTI InScope→KITTI InScope→DAIR-V2X-I DAIR-V2X-I→InScope mAP40 mAP40 mAP40 mAP40 AP40 Source Domain 37.98[URL ] 41.65[URL ] 52.97[URL ] 31.05[URL ] 32.16[URL ] SN 44.80[URL ] 49.34[URL ] 61.87[URL ] 31.81[URL ] 33.25[URL ] ST3D 65.35[URL ] 58.19[URL ] 74.63[URL ] 48.98[URL ] 37.03[URL ] Target Domain 81.63[URL ] 81.63[URL ] 81.63[URL ] 81.41[URL ] 71.75[URL ]
3D Multiobject tracking results on the car, pedestrian, cyclist, and truck. Tracking result of the AD3DMOT based on the InScope dataset on the car class (IoU threshold = 0.5/0.7) Detector sAMOTA↑ MOTA↑ IDSW↓ FRAG↓ PointRCNN 74.81/60.34 63.25/44.45 12/6 595/1834 Pointpillar 82.23/64.98 68.85/46.82 56/44 391/2166 PVRCNN++ 81.63/68.71 67.56/50.72 83/39 386/1560 Centerpoint 78.76/61.25 61.02/40.98 27/15 367/1720
Tracking result of the AD3DMOT based on the InScope-Pri dataset on the car class (IoU threshold = 0.5/0.7) Detector sAMOTA↑ MOTA↑ IDSW↓ FRAG↓ PointRCNN 61.14/44.91 55.04/35.34 42/31 1319/2406 Pointpillar 74.02/51.81 66.89/37.84 154/63 1820/3138 PVRCNN++ 73.47/57.82 54.98/37.94 378/99 914/1524 Centerpoint 76.01/49.32 61.89/31.07 103/49 717/2151
Tracking result of the AD3DMOT based on the InScope dataset on the pedestrian class (IoU threshold = 0.25/0.5) Detector sAMOTA↑ MOTA↑ IDSW↓ FRAG↓ PointRCNN 59.89/56.59 39.73/37.06 1/1 6/22 Pointpillar 32.09/27.42 27.79/25.36 0/0 4/24 PVRCNN++ 31.39/28.54 27.71/25.75 3/3 10/20 Centerpoint 67.38/62.03 63.48/59.30 5/4 8/35
Tracking result of the AD3DMOT based on the InScope-Pri dataset on the pedestrian class (IoU threshold = 0.25/0.5) Detector sAMOTA↑ MOTA↑ IDSW↓ FRAG↓ PointRCNN 78.76/72.65 67.61/60.94 1/1 189/241 Pointpillar 78.14/72.78 68.68/61.43 7/6 130/321 PVRCNN++ 73.76/67.67 58.18/51.61 25/1 2121/205 Centerpoint 75.37/64.27 65.03/53.43 10/7 298/500
Tracking result of the AD3DMOT based on the InScope dataset on the cyclist class (IoU threshold = 0.25/0.5) Detector sAMOTA↑ MOTA IDSW↓ FRAG↓ PointRCNN 60.97/50.27 41.56/33.77 10/13 99/272 Pointpillar 49.96/33.75 33.82/22.33 3/13 64/379 PVRCNN++ 63.00/52.65 43.22/34.12 126/82 177/349 Centerpoint 68.78/57.50 45.42/37.58 6/16 70/267
Tracking result of the AD3DMOT based on the InScope-Pri dataset on the cyclist class (IoU threshold = 0.25/0.5) Detector sAMOTA↑ MOTA↑ IDSW↓ FRAG↓ PointRCNN 38.31/25.57 27.68/18.74 31/27 302/595 Pointpillar 27.90/9.46 19.41/5.58 22/12 272/275 PVRCNN++ 23.27/17.06 12.37/10.44 48/32 151/140 Centerpoint 55.81/34.88 38.70/19.55 46/19 198/613
Tracking result of the AD3DMOT based on the InScope dataset on the truck class (IoU threshold = 0.5/0.7) Detector sAMOTA↑ MOTA↑ IDSW↓ FRAG↓ PointRCNN 82.53/78.67 73.34/68.20 3/2 124/181 Pointpillar 82.18/76.79 75.26/70.33 9/8 80/182 PVRCNN++ 81.50/77.20 69.15/64.53 9/8 76/141 Centerpoint 81.44/76.11 71.89/65.85 7/7 70/207
Tracking result of the AD3DMOT based on the InScope-Pri dataset on the truck class (IoU threshold = 0.5/0.7) Detector sAMOTA↑ MOTA↑ IDSW↓ FRAG↓ PointRCNN 78.76/72.65 67.61/60.94 1/1 189/241 Pointpillar 78.14/72.78 68.68/61.43 7/6 130/321 PVRCNN++ 73.76/67.67 58.18/51.61 25/1 2121/205 Centerpoint 75.37/64.27 65.03/53.43 10/7 298/500
The code and configuration of 3DMOT on the InScope dataset will be released.
If you find InScope useful in your research or applications, please consider giving us a star 🌟.
