Implementation of Geometric Vector Perceptron, a simple circuit for 3d rotation equivariance for learning over large biomolecules, in Pytorch. Idea proposed and accepted at ICLR 2021

Last update: Nov 24, 2022

Overview

Geometric Vector Perceptron

Implementation of Geometric Vector Perceptron, a simple circuit with 3d rotation equivariance for learning over large biomolecules, in Pytorch. The repository may also contain experimentation to see if this could be easily extended to self-attention.

Install

$ pip install geometric-vector-perceptron

Functionality

GVP: Implementing the basic geometric vector perceptron.
GVPDropout: Adapted dropout for GVP in MPNN context
GVPLayerNorm: Adapted LayerNorm for GVP in MPNN context
GVP_MPNN: Adapted instance of Message Passing class from torch-geometric package. Still not tested.

Usage

import torch
from geometric_vector_perceptron import GVP

model = GVP(
    dim_vectors_in = 1024,
    dim_feats_in = 512,
    dim_vectors_out = 256,
    dim_feats_out = 512
)

feats, vectors = (torch.randn(1, 512), torch.randn(1, 1024, 3))

feats_out, vectors_out = model( (feats, vectors) ) # (1, 256), (1, 512, 3)

With the specialized dropout and layernorm as described in the paper

import torch
from torch import nn
from geometric_vector_perceptron import GVP, GVPDropout, GVPLayerNorm

model = GVP(
    dim_vectors_in = 1024,
    dim_feats_in = 512,
    dim_vectors_out = 256,
    dim_feats_out = 512
)

dropout = GVPDropout(0.2)
norm = GVPLayerNorm(512)

feats, vectors = (torch.randn(1, 512), torch.randn(1, 1024, 3))

feats, vectors = model( (feats, vectors) )
feats, vectors = dropout(feats, vectors)
feats, vectors = norm(feats, vectors)  # (1, 256), (1, 512, 3)

TF implementation:

The original implementation in TF by the paper authors can be found here: https://github.com/drorlab/gvp/

Citations

@inproceedings{
    anonymous2021learning,
    title={Learning from Protein Structure with Geometric Vector Perceptrons},
    author={Anonymous},
    booktitle={Submitted to International Conference on Learning Representations},
    year={2021},
    url={https://openreview.net/forum?id=1YLJDvSx6J4},
    note={under review}
}

Comments

Adds MPNN wrapper

Could you double check it?

I've tried to follow the original implementation ( https://github.com/drorlab/gvp/ see src/models.py ) but it's weird since they seem to mix the edge_attrs and the node features and return a mix of them, but later they seem to discard them so :man_shrugging: (whereas i only return node features - bc idk how to modify edge_attrs in an easy way -> the aggregation is always node-based)

opened by hypnopump 3
Fixes edge issues and distinguishes between covalent and proximity bonds
Gets all covalent bonds for a protein conformation (different than proximity)

Reduces unused computations

Fixes edges (now undirected graph)

Adapts denoising example accordingly (not in reconstruction example yet)
opened by hypnopump 0

PyTorch reimplementation of the Smooth ReLU activation function proposed in the paper "Real World Large Scale Recommendation Systems Reproducibility and Smooth Activations" [arXiv 2022].

Smooth ReLU in PyTorch Unofficial PyTorch reimplementation of the Smooth ReLU (SmeLU) activation function proposed in the paper Real World Large Scale

10 Jan 2, 2023

Seach Losses of our paper 'Loss Function Discovery for Object Detection via Convergence-Simulation Driven Search', accepted by ICLR 2021.

CSE-Autoloss Designing proper loss functions for vision tasks has been a long-standing research direction to advance the capability of existing models

54 Dec 17, 2022

Neuralnetwork - Basic Multilayer Perceptron Neural Network for deep learning

Neural Network Just a basic Neural Network module Usage Example Importing Module

0 Nov 1, 2022

This repo contains the implementation of the algorithm proposed in Off-Belief Learning, ICML 2021.

Off-Belief Learning Introduction This repo contains the implementation of the algorithm proposed in Off-Belief Learning, ICML 2021. Environment Setup

32 Jan 5, 2023

From Perceptron model to Deep Neural Network from scratch in Python.

Neural-Network-Basics Aim of this Repository: From Perceptron model to Deep Neural Network (from scratch) in Python. ** Currently working on a basic N

1 Jan 14, 2022

[ICLR 2021, Spotlight] Large Scale Image Completion via Co-Modulated Generative Adversarial Networks

Large Scale Image Completion via Co-Modulated Generative Adversarial Networks, ICLR 2021 (Spotlight) Demo | Paper [NEW!] Time to play with our interac

373 Jan 2, 2023

A PyTorch implementation of Mugs proposed by our paper "Mugs: A Multi-Granular Self-Supervised Learning Framework".

Mugs: A Multi-Granular Self-Supervised Learning Framework This is a PyTorch implementation of Mugs proposed by our paper "Mugs: A Multi-Granular Self-

62 Nov 8, 2022

An attempt at the implementation of Glom, Geoffrey Hinton's new idea that integrates neural fields, predictive coding, top-down-bottom-up, and attention (consensus between columns)

GLOM - Pytorch (wip) An attempt at the implementation of Glom, Geoffrey Hinton's new idea that integrates neural fields, predictive coding,

173 Dec 14, 2022

Official Pytorch implementation of 6DRepNet: 6D Rotation representation for unconstrained head pose estimation.

6D Rotation Representation for Unconstrained Head Pose Estimation (Pytorch) Paper Thorsten Hempel and Ahmed A. Abdelrahman and Ayoub Al-Hamadi, "6D Ro

284 Dec 23, 2022

Implementation of Geometric Vector Perceptron, a simple circuit for 3d rotation equivariance for learning over large biomolecules, in Pytorch. Idea proposed and accepted at ICLR 2021

Related tags

Overview

Geometric Vector Perceptron

Install

Functionality

Usage

TF implementation:

Citations

You might also like...

PyTorch reimplementation of the Smooth ReLU activation function proposed in the paper "Real World Large Scale Recommendation Systems Reproducibility and Smooth Activations" [arXiv 2022].

Seach Losses of our paper 'Loss Function Discovery for Object Detection via Convergence-Simulation Driven Search', accepted by ICLR 2021.

Neuralnetwork - Basic Multilayer Perceptron Neural Network for deep learning

This repo contains the implementation of the algorithm proposed in Off-Belief Learning, ICML 2021.

From Perceptron model to Deep Neural Network from scratch in Python.

[ICLR 2021, Spotlight] Large Scale Image Completion via Co-Modulated Generative Adversarial Networks

A PyTorch implementation of Mugs proposed by our paper "Mugs: A Multi-Granular Self-Supervised Learning Framework".

An attempt at the implementation of Glom, Geoffrey Hinton's new idea that integrates neural fields, predictive coding, top-down-bottom-up, and attention (consensus between columns)

Official Pytorch implementation of 6DRepNet: 6D Rotation representation for unconstrained head pose estimation.

Comments

Adds MPNN wrapper

Fixes edge issues and distinguishes between covalent and proximity bonds

Releases(0.0.14)

0.0.14(Jun 8, 2021)

0.0.12(Mar 7, 2021)

0.0.11(Mar 5, 2021)

0.0.10(Mar 3, 2021)

0.0.9(Mar 1, 2021)

0.0.8(Feb 24, 2021)

0.0.7(Feb 21, 2021)

0.0.6(Jan 27, 2021)

0.0.5(Jan 24, 2021)

0.0.3(Jan 24, 2021)

0.0.2(Jan 14, 2021)

0.0.1(Jan 13, 2021)

Owner

Phil Wang

PyTorch implementation of "Contrast to Divide: self-supervised pre-training for learning with noisy labels"

S2s2net - Sentinel-2 Super-Resolution Segmentation Network

Library to enable Bayesian active learning in your research or labeling work.

DECAF: Deep Extreme Classification with Label Features

A toolkit for controlling Euro Truck Simulator 2 with python to develop self-driving algorithms.

A PyTorch implementation of " EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks."

PyTorchVideo is a deeplearning library with a focus on video understanding work

PyTorch implementation of Rethinking Positional Encoding in Language Pre-training

Code examples and benchmarks from the paper "Understanding Entropy Coding With Asymmetric Numeral Systems (ANS): a Statistician's Perspective"

Neural network chess engine trained on Gary Kasparov's games.

PyTorch Implementation of NCSOFT's FastPitchFormant: Source-filter based Decomposed Modeling for Speech Synthesis

Pytorch Implementation of "Desigining Network Design Spaces", Radosavovic et al. CVPR 2020.

This is the official PyTorch implementation for "Mesa: A Memory-saving Training Framework for Transformers".

Neural models of common sense. 🤖

Meta-Learning Sparse Implicit Neural Representations (NeurIPS 2021)

A very tiny, very simple, and very secure file encryption tool.

This is the first released system towards complex meters` detection and recognition, which is implemented by computer vision techniques.

PyTorch META-DATASET (Few-shot classification benchmark)

Official implementation of Long-Short Transformer in PyTorch.

PrimitiveNet: Primitive Instance Segmentation with Local Primitive Embedding under Adversarial Metric (ICCV 2021)