Shape Generation and Completion Through Point-Voxel Diffusion

Project | Paper

Implementation of Shape Generation and Completion Through Point-Voxel Diffusion

Linqi Zhou, Yilun Du, Jiajun Wu

Requirements:

Make sure the following environments are installed.

python==3.6
pytorch==1.4.0
torchvision==0.5.0
cudatoolkit==10.1
matplotlib==2.2.5
tqdm==4.32.1
open3d==0.9.0
trimesh=3.7.12
scipy==1.5.1

Install PyTorchEMD by

cd metrics/PyTorchEMD
python setup.py install
cp build/**/emd_cuda.cpython-36m-x86_64-linux-gnu.so .

The code was tested on Unbuntu with Titan RTX.

Data

For generation, we use ShapeNet point cloud, which can be downloaded here.

For completion, we use ShapeNet rendering provided by GenRe. We provide script convert_cam_params.py to process the provided data.

For training the model on shape completion, we need camera parameters for each view which are not directly available. To obtain these, simply run

$ python convert_cam_params.py --dataroot DATA_DIR --mitsuba_xml_root XML_DIR

which will create ..._cam_params.npz in each provided data folder for each view.

Pretrained models

Pretrained models can be downloaded here.

Training:

$ python train_generation.py --category car|chair|airplane

Please refer to the python file for optimal training parameters.

Testing:

$ python train_generation.py --category car|chair|airplane --model MODEL_PATH

Results

Some generation and completion results are as follows.

Multimodal completion on a ShapeNet chair.

Multimodal completion on PartNet.

Multimodal completion on two Redwood 3DScan chairs.

Reference

@inproceedings{Zhou_2021_ICCV,
    author    = {Zhou, Linqi and Du, Yilun and Wu, Jiajun},
    title     = {3D Shape Generation and Completion Through Point-Voxel Diffusion},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    month     = {October},
    year      = {2021},
    pages     = {5826-5835}
}

Acknowledgement

For any questions related to codes and experiment setting, please contact Linqi Zhou and Yilun Du.