An Effective Loss Function for Generating 3D Models from Single 2D Image without Rendering
Papers with code | Paper
University of Novi Sad University of Cambridge
Citation
Besides AIAI 2021, our paper is in a Springer's book entitled "Artificial Intelligence Applications and Innovations": link
Please, cite our paper if you find this code useful for your research.
@article{zubic2021effective,
title={An Effective Loss Function for Generating 3D Models from Single 2D Image without Rendering},
author={Zubi{\'c}, Nikola and Li{\`o}, Pietro},
journal={arXiv preprint arXiv:2103.03390},
year={2021}
}
Prerequisites
-
Download code:
Git clone the code with the following command:git clone https://github.com/NikolaZubic/2dimageto3dmodel.git -
Open the project with Conda Environment (Python 3.7)
-
Install packages:
conda install pytorch torchvision torchaudio cudatoolkit=11.0 -c pytorchThen git clone Kaolin library in the root (2dimageto3dmodel) folder with the following commit and run the following commands:
cd kaolin python setup.py install pip install --no-dependencies nuscenes-devkit opencv-python-headless scikit-learn joblib pyquaternion cachetools pip install packaging
Run the program
Run the following commands from the root/code/ (2dimageto3dmodel/code/) directory:
python main.py --dataset cub --batch_size 16 --weights pretrained_weights_cub --save_results
for the CUB Birds Dataset.
python main.py --dataset p3d --batch_size 16 --weights pretrained_weights_p3d --save_results
for the Pascal 3D+ Dataset.
The results will be saved at 2dimageto3dmodel/code/results/ path.
Continue training
To continue the training process:
Run the following commands (without --save_results) from the root/code/ (2dimageto3dmodel/code/) directory:
python main.py --dataset cub --batch_size 16 --weights pretrained_weights_cub
for the CUB Birds Dataset.
python main.py --dataset p3d --batch_size 16 --weights pretrained_weights_p3d
for the Pascal 3D+ Dataset.
License
MIT
Acknowledgment
This idea has been built based on the architecture of Insafutdinov & Dosovitskiy.
Poisson Surface Reconstruction was used for Point Cloud to 3D Mesh transformation.
The GAN architecture (used for texture mapping) is a mixture of Xian's TextureGAN and Li's GAN.
224 Jan 04, 2023
2 Sep 27, 2022
89 Jan 08, 2023
2 Feb 16, 2022
72 Dec 10, 2022
12 Feb 14, 2022
5 Sep 18, 2021
47 Nov 01, 2022
203 Dec 29, 2022
2 Jan 18, 2022
315 Dec 31, 2022
59 Feb 25, 2022
2 Aug 08, 2022
593 Jan 03, 2023
13 Oct 28, 2022
6 Sep 09, 2021
22 Jul 11, 2022
2.2k Dec 31, 2022
19 May 04, 2022
30 Nov 08, 2022