Zongxin Ye+ · Wenyu Li+ · Sidun Liu · Peng Qiao · Yong Dou
arXiv | Project Page | Code
This repository contains code for "AbsGS: Recovering Fine Details for 3D Gaussian Splatting" (ACM MM 2024).
We reveal that the original adaptive density control strategy in 3D Gaussian Splatting (3D-GS) has the flaw of gradient collision which results in degradation, and propose homodirectional gradient as the guidance for densification. (a) Our method recovers fine details and achieves higher quality novel view synthesis results. SSIM, PSNR, LPIPS are inset. (b) Our proposed method yields more reasonable distribution of Gaussion points with comparable number of Gaussians and memory consumption with 3D-GS. (c) By adopting our method, the large Gaussians in over-reconstructed regions that lead to blur are eliminated.
The repository contains submodules, thus please check it out with
git clone git@github.com:TY424/AbsGS.git --recursive
# if you have an environment used for 3dgs, use it# if not, create a new environment
conda env create --file environment.yml
conda activate Absgs
cd submodules/
python ./diff-gaussian-rasterization-abs/setup.py install
python ./simple-knn/setup.py install# Train
python train.py -s <path to COLMAP or NeRF Synthetic dataset>
-m <output path>
--eval # Train with train/test split
# Generate renderings
python render.py -s <path to COLMAP or NeRF Synthetic dataset> -m <output path>
# Compute error metrics on renderings
python metrics.py -m <path to trained model> # This script specifies the routine used in our evaluation
python full_eval.py -m360 <mipnerf360 folder> -tat <tanks and temples folder> -db <deep blending folder>
Based on AbsGS, we have made some minor modifications to improve performance in large scenarios. None of these modifications were utilized in our paper.
We noticed that the visualization results of point clouds in COLMAP are inconsistent with those in the .ply file, as COLMAP filters them based on reprojection errors and track length during visualization. Therefore, we modified the read_points3D_binary function. In addition, pruning is performed initially based on the size of the radius to remove noise.
Taking inspiration from the pruning strategy based on max_radii2d in 3DGS, we conduct pruning based on the contribution (max_weight) of the Gaussian during rendering , where max_weight represents the maximum weight of the Gaussian participated in all rendering processes.
In fact, the pruning strategy based on max_radii2d does not work for 3DGS, and we haven't fixed this bug.
Sci-Art scene from UrbanScene3D dataset
w/ prune: uses Initial pruning and weight-based pruning
Percentage of scene extent (0--1) a point must exceed to be splite, 0.001 by default.
Limit that decides if points should be cloned based on 2D position gradient, 0.0002 by default.
Limit that decides if points should be splite based on homodirectional gradient, 0.0004 by default.
Whether to periodically reduce opacity, True by default.
How frequently to reduce opacity, 3_000 by default.
Whether to prune based on weights, False by default.
Gaussians with weights less than this threshold will be pruned, 0.5 by default.
Iteration where pruning by weight stops, 15_000 by default.
If you find our code or paper useful, please consider citing:
@misc{ye2024absgs,
title={AbsGS: Recovering Fine Details for 3D Gaussian Splatting}, author={Zongxin Ye and Wenyu Li and Sidun Liu and Peng Qiao and Yong Dou},
year={2024},
eprint={2404.10484},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
@Article{kerbl3Dgaussians,
author = {Kerbl, Bernhard and Kopanas, Georgios and Leimk{\"u}hler, Thomas and Drettakis, George},
title = {3D Gaussian Splatting for Real-Time Radiance Field Rendering},
journal = {ACM Transactions on Graphics},
number = {4},
volume = {42},
month = {July},
year = {2023},
url = {https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/}
}