SymmetryNet: Learning to Predict Reflectional and Rotational Symmetries of 3D Shapes from Single-View RGB-D Images

Overview

SymmetryNet

SymmetryNet: Learning to Predict Reflectional and Rotational Symmetries of 3D Shapes from Single-View RGB-D Images

ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia 2020)

Created by Yifei Shi, Junwen Huang, Hongjia Zhang, Xin Xu, Szymon Rusinkiewicz and Kai Xu

teaser

This repository includes:

  • tools: the training scripts and evaluation scripts
    • tools/train_shapenet.py: the training script for shapenet dataset
    • tools/train_ycb.py: the training script for ycb dataset
    • tools/train_scannet.py: the training script for scannet dataset
    • tools/evaluation: the evaluation scripts
      • evaluation/eval_ref_shapenet.py: the evaluation script for reflectional symmetry on shapenet dataset
      • evaluation/eval_ref_ycb.py: the evaluation script for reflectional symmetry on ycb dataset
      • evaluation/eval_ref_scannet.py: the evaluation script for reflectional symmetry on scannet dataset
      • evaluation/eval_rot_shapenet.py: the evaluation script for rotational symmetry on shapenet dataset
      • evaluation/eval_rot_ycb.py: the evaluation script for rotational symmetry on ycb dataset
      • evaluation/eval_rot_scannet.py: the evaluation script for rotational symmetry on scannet dataset
  • lib: the core Python library for networks and loss
    • lib/loss.py: symmetrynet loss caculation for both reflectional and rotational symmetries,the loss items are listed at the end of the text
    • lib/network.py: network architecture
    • lib/tools.py: functions for the operation of rotation and reflection
    • lib/verification.py: verification of the rotational and reflectional symmetries
  • datasets: the dataloader and training/testing lists
    • datasets/shapenet/dataset.py: the training dataloader for shapnet dataset
    • datasets/shapenet/dataset_eval.py: the evaluation dataloader for shapnet dataset
      • datasets/shapenet/dataset_config/*.txt: training and testing splits for shapenet dataset, the testing splits includ holdout view/instance/category
    • datasets/ycb/dataset.py: the training dataloader for ycb dataset
    • datasets/ycb/dataset_eval.py: the evaluation dataloader for ycb dataset
      • datasets/ycb/dataset_config/*.txt: training and testing splits for shapenet dataset,the training/testing splits fallow the ycb defult settings
    • datasets/shapenet/dataset.py: the training dataloader for scannet dataset
    • datasets/shapenet/dataset_eval.py: the evaluation dataloader for scannet dataset
      • datasets/scannet/dataset_config/*.txt: training and testing splits for scannet dataset,the testing splits includ holdout view/scene

Environments

pytorch>=0.4.1 python >=3.6

Datasets

  • ShapeNet dataset

    • shapenetcore: this folder saves the models and their ground truth symmetries for each instance
    • rendered_data: this folder saves the rgbd images that we rendered for each instance, including their ground truth pose and camera intrinsic matrix, etc.
    • name_list.txt: this file saves the correspondence between the name of instances and their ID in this project(the names are too long to identify)
  • YCB dataset

    • models: this folder saves the ground truth model symmetry for each instance
    • data: this folder saves the rgbd videos and the ground truth poses and camera information
    • classes.txt: this file saves the correspondence between the name of YCB objects and their *.xyz models
    • symmetries.txt: this file saves all the ground truth symmetries for ycb object models

Training

To train the network with the default parameter on shapenet dataset, run

python tools/train_shapenet.py --dataset_root= your/folder/to/shapnet/dataset

To train the network with the default parameter on ycb dataset, run

python tools/train_ycb.py --dataset_root= your/folder/to/ycb/dataset

To train the network with the default parameter on scannet dataset, run

python tools/train_scannet.py --dataset_root= your/folder/to/scannet/dataset

Evaluation

To evaluate the model with our metric on shapenet, for reflectional symmetry, run

python tools/evaluation/eval_ref_shapenet.py

for rotational symmetry, run

python tools/evaluation/eval_rot_shapenet.py

To evaluate the model with our metric on ycb, for reflectional symmetry, run

python tools/evaluation/eval_ref_ycb.py

for rotational symmetry, run

python tools/evaluation/eval_rot_ycb.py

To evaluate the model with our metric on scannet, for reflectional symmetry, run

python tools/evaluation/eval_ref_scannet.py

for rotational symmetry, run

python tools/evaluation/eval_rot_scannet.py

Pretrained model & data download

The pretrained models and data can be found at here (dropbox) and here (baidu yunpan, password: symm).

LAVT: Language-Aware Vision Transformer for Referring Image Segmentation

LAVT: Language-Aware Vision Transformer for Referring Image Segmentation Where we are ? 12.27 目前和原论文仍有1%左右得差距,但已经力压很多SOTA了 ckpt__448_epoch_25.pth mIoU

zichengsaber 60 Dec 11, 2022
RL and distillation in CARLA using a factorized world model

World on Rails Learning to drive from a world on rails Dian Chen, Vladlen Koltun, Philipp Krähenbühl, arXiv techical report (arXiv 2105.00636) This re

Dian Chen 131 Dec 16, 2022
A more easy-to-use implementation of KPConv

A more easy-to-use implementation of KPConv This repo contains a more easy-to-use implementation of KPConv based on PyTorch. Introduction KPConv is a

Zheng Qin 35 Dec 14, 2022
Generative Adversarial Text-to-Image Synthesis

###Generative Adversarial Text-to-Image Synthesis Scott Reed, Zeynep Akata, Xinchen Yan, Lajanugen Logeswaran, Bernt Schiele, Honglak Lee This is the

Scott Ellison Reed 883 Dec 31, 2022
PixelPyramids: Exact Inference Models from Lossless Image Pyramids (ICCV 2021)

PixelPyramids: Exact Inference Models from Lossless Image Pyramids This repository contains the PyTorch implementation of the paper PixelPyramids: Exa

Visual Inference Lab @TU Darmstadt 8 Dec 11, 2022
Object detection using yolo-tiny model and opencv used as backend

Object detection Algorithm used : Yolo algorithm Backend : opencv Library required: opencv = 4.5.4-dev' Quick Overview about structure 1) main.py Load

2 Jul 06, 2022
DeepFaceLab fork which provides IPython Notebook to use DFL with Google Colab

DFL-Colab — DeepFaceLab fork for Google Colab This project provides you IPython Notebook to use DeepFaceLab with Google Colaboratory. You can create y

779 Jan 05, 2023
Benchmarking the robustness of Spatial-Temporal Models

Benchmarking the robustness of Spatial-Temporal Models This repositery contains the code for the paper Benchmarking the Robustness of Spatial-Temporal

Yi Chenyu Ian 15 Dec 16, 2022
Deep Latent Force Models

Deep Latent Force Models This repository contains a PyTorch implementation of the deep latent force model (DLFM), presented in the paper, Compositiona

Tom McDonald 5 Oct 26, 2022
DeepLab is a state-of-art deep learning system for semantic image segmentation built on top of Caffe.

DeepLab Introduction DeepLab is a state-of-art deep learning system for semantic image segmentation built on top of Caffe. It combines densely-compute

Ali 234 Nov 14, 2022
Newt - a Gaussian process library in JAX.

Newt __ \/_ (' \`\ _\, \ \\/ /`\/\ \\ \ \\

AaltoML 0 Nov 02, 2021
This repository contains an implementation of ConvMixer for the ICLR 2022 submission "Patches Are All You Need?".

Patches Are All You Need? 🤷 This repository contains an implementation of ConvMixer for the ICLR 2022 submission "Patches Are All You Need?". Code ov

ICLR 2022 Author 934 Dec 30, 2022
The implementation of 'Image synthesis via semantic composition'.

Image synthesis via semantic synthesis [Project Page] by Yi Wang, Lu Qi, Ying-Cong Chen, Xiangyu Zhang, Jiaya Jia. Introduction This repository gives

DV Lab 71 Jan 06, 2023
A PyTorch implementation of EfficientDet.

A PyTorch impl of EfficientDet faithful to the original Google impl w/ ported weights

Ross Wightman 1.4k Jan 07, 2023
Strongly local p-norm-cut algorithms for semi-supervised learning and local graph clustering

Strongly local p-norm-cut algorithms for semi-supervised learning and local graph clustering

Meng Liu 2 Jul 19, 2022
CS_Final_Metal_surface_detection - This is a final project for CoderSchool Machine Learning bootcamp on 29/12/2021.

CS_Final_Metal_surface_detection This is a final project for CoderSchool Machine Learning bootcamp on 29/12/2021. The project is based on the dataset

Cuong Vo 1 Dec 29, 2021
Discord Multi Tool that focuses on design and easy usage

Multi-Tool-v1.0 Discord Multi Tool that focuses on design and easy usage Delete webhook Block all friends Spam webhook Modify webhook Webhook info Tok

Lodi#0001 24 May 23, 2022
Computer-Vision-Paper-Reviews - Computer Vision Paper Reviews with Key Summary along Papers & Codes

Computer-Vision-Paper-Reviews Computer Vision Paper Reviews with Key Summary along Papers & Codes. Jonathan Choi 2021 50+ Papers across Computer Visio

Jonathan Choi 2 Mar 17, 2022
make ASCII Art by Deep Learning

DeepAA This is convolutional neural networks generating ASCII art. This repository is under construction. This work is accepted by NIPS 2017 Workshop,

OsciiArt 1.4k Dec 28, 2022
Generalized Matrix Means for Semi-Supervised Learning with Multilayer Graphs

Generalized Matrix Means for Semi-Supervised Learning with Multilayer Graphs MATLAB implementation of the paper: P. Mercado, F. Tudisco, and M. Hein,

Pedro Mercado 6 May 26, 2022