Skip to content

RecolorNeRF: Layer Decomposed Radiance Fields for Efficient Color Editing of 3D Scenes

Notifications You must be signed in to change notification settings

yuehaowang/RecolorNeRF

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

8 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

RecolorNeRF

PyTorch implementation of paper "RecolorNeRF: Layer Decomposed Radiance Fields for Efficient Color Editing of 3D Scenes" by Bingchen Gong*, Yuehao Wang*, Xiaoguang Han, and Qi Dou.

A novel user-friendly color editing approach for neural radiance fields.

[Paper] [Project Website] [Selected Results]

recolornerf_teaser.mp4

News

  • Sep 18, 2023: The camera-ready version is uploaded to Arxiv.
  • Jul 16, 2023: Our paper is accepted by ACM MM 2023! 🎉
  • Feb 05, 2023: Initial release of the code.

Installation

Tested on Ubuntu 18.04 with PyTorch 1.12.1 and CUDA 11.1.

Type the commands below to set up the running environment.

conda create -n recolornerf python=3.8
conda activate recolornerf
# PyTorch (may need to adapt to your environment)
pip install torch torchvision
# PyTorch3D
conda install pytorch3d -c pytorch3d
# Essentials
pip install tqdm scikit-image opencv-python configargparse lpips imageio-ffmpeg kornia Pillow lpips tensorboard trimesh
conda install -c conda-forge einops
# Palette extraction
conda install -c conda-forge scipy
conda install -c conda-forge cython
CVXOPT_BUILD_GLPK=1 pip install cvxopt

Data

Support Datasets

Palette Initialization

Download our customized initial palettes to data_palette/ for reproducing our layer decomposition results.

For scenes other than the provided ones, you can use the jupyter notebook tools/get_palette.ipynb to generate and customize new palettes.

Training

Type the command below to train a RecolorNeRF model:

python run_recolornerf.py --config configs/chair.txt

We provide our configurations for 15 scenes in the configs/ directory. Remember to change the datadir option to your dataset path. For more options, please refer to the utils/opt.py file.

Recoloring

We create a jupyter notebook tools/color_edit.ipynb for recoloring an optimized RecolorNeRF in a quasi-interactive way. In this jupyter notebook, some simple GUI widgets (like color pickers) and visualization & rendering scripts are provided.

Citation

If you find our code or paper is helpful, please consider citing:

@article{gong2023recolornerf,
  title={RecolorNeRF: Layer Decomposed Radiance Fields for Efficient Color Editing of 3D Scenes},
  author={Gong, Bingchen and Wang, Yuehao and Han, Xiaoguang and Dou, Qi},
  journal={arXiv preprint arXiv:2301.07958},
  year={2023}
}

Acknowledgement