RayDN

Ray Denoising: Depth-aware Hard Negative Sampling for Multi-view 3D Object Detection

RayDN.mp4

Introduction

This repository is an official implementation of our ECCV 2024 paper Ray Denoising: Depth-aware Hard Negative Sampling for Multi-view 3D Object Detection. This repository contains Pytorch training code, evaluation code and pre-trained models.

Framework

Getting Started

Our code is built based on StreamPETR. Please follow StreamPETR to setup enviroment and prepare data step by step.

Training and Inference

You can train the model following:

tools/dist_train.sh projects/configs/RayDN/raydn_eva02_800_bs2_seq_24e.py 8 

You can evaluate the detection model following:

tools/dist_test.sh projects/configs/RayDN/raydn_eva02_800_bs2_seq_24e.py work_dirs/raydn_eva02_800_bs2_seq_24e/latest.pth 8 --eval bbox

Results on NuScenes Val Set.

Model	Setting	Pretrain	Lr Schd	NDS	mAP	Config	Download
RayDN	R50 - 428q	NuImg	60ep	56.1	47.1	config	ckpt
RayDN	EVA02-L - 900q	EVA02	24ep	62.4	54.1	config	ckpt

Acknowledgements

We thank these great works and open-source codebases: MMDetection3d, StreamPETR, DETR3D, PETR.

Citation

If you find RayDN is useful in your research or applications, please consider giving us a star 🌟 and citing it by the following BibTeX entry.

@article{liu2024ray,
title={Ray Denoising: Depth-aware Hard Negative Sampling for Multi-view 3D Object Detection},
author={Liu, Feng and Huang, Tengteng and Zhang, Qianjing and Yao, Haotian and Zhang, Chi and Wan, Fang and Ye, Qixiang and Zhou, Yanzhao},
journal={arXiv preprint arXiv:2402.03634},
year={2024}
}