Implementation of Geometric Vector Perceptron, a simple circuit for 3d rotation equivariance for learning over large biomolecules, in Pytorch. Idea proposed and accepted at ICLR 2021

Last update: Nov 24, 2022

Overview

Geometric Vector Perceptron

Implementation of Geometric Vector Perceptron, a simple circuit with 3d rotation equivariance for learning over large biomolecules, in Pytorch. The repository may also contain experimentation to see if this could be easily extended to self-attention.

Install

$ pip install geometric-vector-perceptron

Functionality

GVP: Implementing the basic geometric vector perceptron.
GVPDropout: Adapted dropout for GVP in MPNN context
GVPLayerNorm: Adapted LayerNorm for GVP in MPNN context
GVP_MPNN: Adapted instance of Message Passing class from torch-geometric package. Still not tested.

Usage

import torch
from geometric_vector_perceptron import GVP

model = GVP(
    dim_vectors_in = 1024,
    dim_feats_in = 512,
    dim_vectors_out = 256,
    dim_feats_out = 512
)

feats, vectors = (torch.randn(1, 512), torch.randn(1, 1024, 3))

feats_out, vectors_out = model( (feats, vectors) ) # (1, 256), (1, 512, 3)

With the specialized dropout and layernorm as described in the paper

import torch
from torch import nn
from geometric_vector_perceptron import GVP, GVPDropout, GVPLayerNorm

model = GVP(
    dim_vectors_in = 1024,
    dim_feats_in = 512,
    dim_vectors_out = 256,
    dim_feats_out = 512
)

dropout = GVPDropout(0.2)
norm = GVPLayerNorm(512)

feats, vectors = (torch.randn(1, 512), torch.randn(1, 1024, 3))

feats, vectors = model( (feats, vectors) )
feats, vectors = dropout(feats, vectors)
feats, vectors = norm(feats, vectors)  # (1, 256), (1, 512, 3)

TF implementation:

The original implementation in TF by the paper authors can be found here: https://github.com/drorlab/gvp/

Citations

@inproceedings{
    anonymous2021learning,
    title={Learning from Protein Structure with Geometric Vector Perceptrons},
    author={Anonymous},
    booktitle={Submitted to International Conference on Learning Representations},
    year={2021},
    url={https://openreview.net/forum?id=1YLJDvSx6J4},
    note={under review}
}

Comments

Adds MPNN wrapper

Could you double check it?

I've tried to follow the original implementation ( https://github.com/drorlab/gvp/ see src/models.py ) but it's weird since they seem to mix the edge_attrs and the node features and return a mix of them, but later they seem to discard them so :man_shrugging: (whereas i only return node features - bc idk how to modify edge_attrs in an easy way -> the aggregation is always node-based)

opened by hypnopump 3
Fixes edge issues and distinguishes between covalent and proximity bonds
Gets all covalent bonds for a protein conformation (different than proximity)

Reduces unused computations

Fixes edges (now undirected graph)

Adapts denoising example accordingly (not in reconstruction example yet)
opened by hypnopump 0

PyTorch reimplementation of the Smooth ReLU activation function proposed in the paper "Real World Large Scale Recommendation Systems Reproducibility and Smooth Activations" [arXiv 2022].

Smooth ReLU in PyTorch Unofficial PyTorch reimplementation of the Smooth ReLU (SmeLU) activation function proposed in the paper Real World Large Scale

10 Jan 2, 2023

Seach Losses of our paper 'Loss Function Discovery for Object Detection via Convergence-Simulation Driven Search', accepted by ICLR 2021.

CSE-Autoloss Designing proper loss functions for vision tasks has been a long-standing research direction to advance the capability of existing models

54 Dec 17, 2022

Neuralnetwork - Basic Multilayer Perceptron Neural Network for deep learning

Neural Network Just a basic Neural Network module Usage Example Importing Module

0 Nov 1, 2022

This repo contains the implementation of the algorithm proposed in Off-Belief Learning, ICML 2021.

Off-Belief Learning Introduction This repo contains the implementation of the algorithm proposed in Off-Belief Learning, ICML 2021. Environment Setup

32 Jan 5, 2023

From Perceptron model to Deep Neural Network from scratch in Python.

Neural-Network-Basics Aim of this Repository: From Perceptron model to Deep Neural Network (from scratch) in Python. ** Currently working on a basic N

1 Jan 14, 2022

[ICLR 2021, Spotlight] Large Scale Image Completion via Co-Modulated Generative Adversarial Networks

Large Scale Image Completion via Co-Modulated Generative Adversarial Networks, ICLR 2021 (Spotlight) Demo | Paper [NEW!] Time to play with our interac

373 Jan 2, 2023

A PyTorch implementation of Mugs proposed by our paper "Mugs: A Multi-Granular Self-Supervised Learning Framework".

Mugs: A Multi-Granular Self-Supervised Learning Framework This is a PyTorch implementation of Mugs proposed by our paper "Mugs: A Multi-Granular Self-

62 Nov 8, 2022

An attempt at the implementation of Glom, Geoffrey Hinton's new idea that integrates neural fields, predictive coding, top-down-bottom-up, and attention (consensus between columns)

GLOM - Pytorch (wip) An attempt at the implementation of Glom, Geoffrey Hinton's new idea that integrates neural fields, predictive coding,

173 Dec 14, 2022

Official Pytorch implementation of 6DRepNet: 6D Rotation representation for unconstrained head pose estimation.

6D Rotation Representation for Unconstrained Head Pose Estimation (Pytorch) Paper Thorsten Hempel and Ahmed A. Abdelrahman and Ayoub Al-Hamadi, "6D Ro

284 Dec 23, 2022

Implementation of Geometric Vector Perceptron, a simple circuit for 3d rotation equivariance for learning over large biomolecules, in Pytorch. Idea proposed and accepted at ICLR 2021

Related tags

Overview

Geometric Vector Perceptron

Install

Functionality

Usage

TF implementation:

Citations

You might also like...

PyTorch reimplementation of the Smooth ReLU activation function proposed in the paper "Real World Large Scale Recommendation Systems Reproducibility and Smooth Activations" [arXiv 2022].

Seach Losses of our paper 'Loss Function Discovery for Object Detection via Convergence-Simulation Driven Search', accepted by ICLR 2021.

Neuralnetwork - Basic Multilayer Perceptron Neural Network for deep learning

This repo contains the implementation of the algorithm proposed in Off-Belief Learning, ICML 2021.

From Perceptron model to Deep Neural Network from scratch in Python.

[ICLR 2021, Spotlight] Large Scale Image Completion via Co-Modulated Generative Adversarial Networks

A PyTorch implementation of Mugs proposed by our paper "Mugs: A Multi-Granular Self-Supervised Learning Framework".

An attempt at the implementation of Glom, Geoffrey Hinton's new idea that integrates neural fields, predictive coding, top-down-bottom-up, and attention (consensus between columns)

Official Pytorch implementation of 6DRepNet: 6D Rotation representation for unconstrained head pose estimation.

Comments

Adds MPNN wrapper

Fixes edge issues and distinguishes between covalent and proximity bonds

Releases(0.0.14)

0.0.14(Jun 8, 2021)

0.0.12(Mar 7, 2021)

0.0.11(Mar 5, 2021)

0.0.10(Mar 3, 2021)

0.0.9(Mar 1, 2021)

0.0.8(Feb 24, 2021)

0.0.7(Feb 21, 2021)

0.0.6(Jan 27, 2021)

0.0.5(Jan 24, 2021)

0.0.3(Jan 24, 2021)

0.0.2(Jan 14, 2021)

0.0.1(Jan 13, 2021)

Owner

Phil Wang

Make your AirPlay devices as TTS speakers

Waymo motion prediction challenge 2021: 3rd place solution

FocusFace: Multi-task Contrastive Learning for Masked Face Recognition

Official implementation of CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification

Tensorflow implementation of Swin Transformer model.

A practical ML pipeline for data labeling with experiment tracking using DVC.

Implementation of Kalman Filter in Python

Google Brain - Ventilator Pressure Prediction

Official PyTorch implementation of "Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemble" (NeurIPS'21)

[CVPR'22] Official PyTorch Implementation of Collaborative Transformers for Grounded Situation Recognition

PyTorch implementation of our paper: Decoupling and Recoupling Spatiotemporal Representation for RGB-D-based Motion Recognition

SciKit-Learn Laboratory (SKLL) makes it easy to run machine learning experiments.

Reducing Information Bottleneck for Weakly Supervised Semantic Segmentation (NeurIPS 2021)

Official implementation of "Robust channel-wise illumination estimation"

Official implementation of Rich Semantics Improve Few-Shot Learning (BMVC, 2021)

ONNX Runtime Web demo is an interactive demo portal showing real use cases running ONNX Runtime Web in VueJS.

DTCN IJCAI - Sequential prediction learning framework and algorithm

Receptive Field Block Net for Accurate and Fast Object Detection, ECCV 2018

PyTorch implementation of MSBG hearing loss model and MBSTOI intelligibility metric

Automatic 2D-to-3D Video Conversion with CNNs