HiNeuS

High-fidelity Neural Surface Relightable Toolkit for 3DRealCar Assets

Built on High-fidelity Neural Surface Mitigating Low-texture and Reflective Ambiguity

---

---

Quick Start

Installation

# Clone the repository
git clone https://github.com/wangyida/hineus.git
cd HiNeuS
# Install dependencies
pip install -r requirements.txt
# Install additional dependencies
nvdiffrast: https://nvlabs.github.io/nvdiffrast/#installation
raytracing: https://github.com/ashawkey/raytracing

Note: Ensure you have NVIDIA GPU with CUDA support for optimal performance.

---

1. Geometric Learning

Training

Train neural surface representations for both synthetic and real-world data:

# NeRF synthetic data
python run_training.py \
--cfg configs/shape/nerf/general.yaml \
object=drums \
dataset_dir=${your-path}/dataset/nerf_synthetic
# COLMAP real-world data
python run_training.py \
--cfg configs/shape/real/general.yaml \
object=sedan \
dataset_dir=${your-path}/dataset/real \
name={optional-custom-folder-name}

Outputs:

• Intermediate visualizations: data/train_vis/
• Trained models: data/model/

Mesh Extraction

Extract high-quality meshes from trained neural representations:

python extract_mesh.py \
--cfg configs/shape/real/general.yaml \
object=sedan \
dataset_dir=${your-path}/dataset/real \
name={optional-custom-folder-name}

Outputs:

• Extracted meshes: data/meshes/

2. Material Estimation

Training

Estimate BRDF materials from reconstructed meshes:

# Estimate BRDF for sedan from COLMAP project
python run_training.py \
--cfg configs/material/real/general.yaml \
object=sedan \
dataset_dir=${your-path}/dataset/real \
mesh=${your-mesh-path}.ply

Outputs:

• Training visualizations: data/train_vis/
• Material models: data/model/

Extract Materials

Extract and save material properties:

# Synthetic data example
python extract_materials.py --cfg configs/material/syn/bell.yaml
# Real-world data example
python extract_materials.py --cfg configs/material/real/bear.yaml

Outputs:

• Extracted materials: data/materials/

3. Relighting

Create stunning relighting effects using HDRI environment maps:

# Relight with transparency
python relight.py \
--blender <path-to-your-blender> \
--name bell-neon \
--mesh data/meshes/bell_shape-300000.ply \
--material data/materials/bell_material-100000 \
--hdr data/hdr/neon_photostudio_4k.exr \
--trans
# Relight without transparency
python relight.py \
--blender <path-to-your-blender> \
--name bear-neon \
--mesh data/meshes/bear_shape-300000.ply \
--material data/materials/bear_material-100000 \
--hdr data/hdr/neon_photostudio_4k.exr

Outputs:

• Relighting results: data/relight/{name}/

---

Code Architecture

HiNeuS follows a clean, modular architecture designed for extensibility and maintainability:

HiNeuS
├── network/ # Neural network architectures
│ ├── shape/ # Shape representation networks
│ └── material/ # Material estimation networks
├── dataset/ # Data loading & preprocessing
├── train/ # Training loops & validation
├── utils/ # Core utilities
│ ├── geometry/ # Geometric operations
│ └── image/ # Image processing
├── colmap/ # COLMAP integration
└── config/ # Configuration files

Acknowledgments

We extend our gratitude to the foundational work of these excellent projects:

Special thanks to the 3DRealCar dataset contributors for providing valuable real-world assets for our research.

---

---

Citation

If you find HiNeuS useful in your research, please cite our papers:

@inproceedings{wang2025hineus,
title={HiNeuS: High-fidelity Neural Surface Mitigating Low-texture and Reflective Ambiguity},
author={Wang, Yida and Zhang, Xueyang and Zhan, Kun and Jia, Peng and Lang, Xianpeng},
booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
year={2025}
}
@inproceedings{du20253drealcar,
title={3drealcar: An in-the-wild rgb-d car dataset with 360-degree views},
author={Du, Xiaobiao and Wang, Yida and Sun, Haiyang and Wu, Zhuojie and Sheng, Hongwei and Wang, Shuyun and Ying, Jiaying and Lu, Ming and Zhu, Tianqing and Zhan, Kun and others},
booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
pages={26488--26498},
year={2025}
}

---

Contact & Support

• Project Website:https://wangyida.github.io/posts/hineus/
• Discussions:GitHub Discussions
• Issues:GitHub Issues

---

If you find this project helpful, please consider giving it a star!