[ICCV2021] Learning to Regress Bodies from Images using Differentiable Semantic Rendering

Getting Started

DSR has been implemented and tested on Ubuntu 18.04 with python 3.6.

Clone the repo:

git clone https://github.com/saidwivedi/DSR.git

Install the requirements using conda:

# conda
source install_conda.sh

Preparation of Data

For evaluation, you need to download the pretrained DSR model and SMPL body models. Run the command following command

source prepare_data.sh

For both evaluation and training, we use data processing techinque similar to SPIN. Kindly refer to their repo for more details.

Evaluation

For evaluating on 3DPW, MPI-INF-3DHP and Human3.6M, run the following command

# Change the val_ds configuration for different datasets
python train.py --cfg configs/dsr_eval.cfg

Training

For training, we use the off-the-self human parsing model Graphonomy to get pseudo ground truth clothing segmentation. Use the Universal Model of Graphonomy to generate the segmentation mask. After preparing the data, run the following command

# For more details on different parameters, refer to dsr/core/config.py
python train.py --cfg configs/dsr_train.cfg

Citation

@inproceedings{Dwivedi_DSR_2021,
  title = {Learning To Regress Bodies From Images Using Differentiable Semantic Rendering},
  author = {Dwivedi, Sai Kumar and Athanasiou, Nikos and Kocabas, Muhammed and Black, Michael J.},
  booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
  pages = {11250-11259},
  month = {October},
  year = {2021}
}

License

This code is available for non-commercial scientific research purposes as defined in the LICENSE file. By downloading and using this code you agree to the terms in the LICENSE. Third-party datasets and software are subject to their respective licenses.

References

Major part of the code is borrowed from PARE.

Contact

For questions, please contact [email protected]

For commercial licensing (and all related questions for business applications), please contact [email protected].