RITA is a family of autoregressive protein models, developed by LightOn in collaboration with the OATML group at Oxford and the Debora Marks Lab at Harvard.

Last update: Dec 22, 2022

Overview

RITA: a Study on Scaling Up Generative Protein Sequence Models

RITA is a family of autoregressive protein models, developed by a collaboration of Lighton, the OATML group at Oxford, and the Debbie Marks Lab at Harvard.

Model	#Params	d_model	layers	lm loss uniref-100
Small	85M	768	12	2.31
Medium	300M	1024	24	2.01
Large	680M	1536	24	1.82
XLarge	1.2B	2048	24	1.70

Results

For full results see our preprint: https://arxiv.org/abs/2205.05789

Usage

Instantiate a model like so:

from transformers import AutoModel, AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("lightonai/RITA_s, trust_remote_code=True")
tokenizer = AutoTokenizer.from_pretrained("lightonai/RITA_s")

for generation we support pipelines:

from transformers import pipeline
rita_gen = pipeline('text-generation', model=model, tokenizer=tokenizer)
sequences = rita_gen("MAB", max_length=20, do_sample=True, top_k=950, repetition_penalty=1.2, 
                     num_return_sequences=2, eos_token_id=2)
for seq in sequences:
    print(f"seq: {seq['generated_text'].replace(' ', '')}")

Or see example.py

How to cite

@article{hesslow2022rita,
  title={RITA: a Study on Scaling Up Generative Protein Sequence Models},
  author={Hesslow, Daniel and Zanichelli, Niccol{\'o} and Notin, Pascal and Poli, Iacopo and Marks, Debora},
  journal={arXiv preprint arXiv:2205.05789},
  year={2022}
}

RITA is a family of autoregressive protein models, developed by LightOn in collaboration with the OATML group at Oxford and the Debora Marks Lab at Harvard.

Related tags

Overview

RITA: a Study on Scaling Up Generative Protein Sequence Models

Results

Usage

How to cite

Owner

LightOn

Libraries, tools and tasks created and used at DeepMind Robotics.

Hummingbird compiles trained ML models into tensor computation for faster inference.

An Unpaired Sketch-to-Photo Translation Model

SmartSim Infrastructure Library.

THIS IS THE OLD PYMC PROJECT. PLEASE USE PYMC3 INSTEAD:

Neural Scene Graphs for Dynamic Scene (CVPR 2021)

PyStan, a Python interface to Stan, a platform for statistical modeling. Documentation: https://pystan.readthedocs.io

Mask-invariant Face Recognition through Template-level Knowledge Distillation

Pytorch implementation of Feature Pyramid Network (FPN) for Object Detection

In this project we use both Resnet and Self-attention layer for cat, dog and flower classification.

TransFGU: A Top-down Approach to Fine-Grained Unsupervised Semantic Segmentation

Study of human inductive biases in CNNs and Transformers.

Crawl & visualize ICLR papers and reviews

Using fully convolutional networks for semantic segmentation with caffe for the cityscapes dataset

Code for CVPR2019 paper《Unequal Training for Deep Face Recognition with Long Tailed Noisy Data》

Learning infinite-resolution image processing with GAN and RL from unpaired image datasets, using a differentiable photo editing model.

Very large and sparse networks appear often in the wild and present unique algorithmic opportunities and challenges for the practitioner

ViDT: An Efficient and Effective Fully Transformer-based Object Detector

PyTorch code for EMNLP 2021 paper: Don't be Contradicted with Anything! CI-ToD: Towards Benchmarking Consistency for Task-oriented Dialogue System

Tensors and neural networks in Haskell

RITA is a family of autoregressive protein models, developed by LightOn in collaboration with the OATML group at Oxford and the Debora Marks Lab at Harvard.

Related tags

Overview

RITA: a Study on Scaling Up Generative Protein Sequence Models

Results

Usage

How to cite

Owner

LightOn

Libraries, tools and tasks created and used at DeepMind Robotics.

Hummingbird compiles trained ML models into tensor computation for faster inference.

An Unpaired Sketch-to-Photo Translation Model

SmartSim Infrastructure Library.

THIS IS THE **OLD** PYMC PROJECT. PLEASE USE PYMC3 INSTEAD:

Neural Scene Graphs for Dynamic Scene (CVPR 2021)

PyStan, a Python interface to Stan, a platform for statistical modeling. Documentation: https://pystan.readthedocs.io

Mask-invariant Face Recognition through Template-level Knowledge Distillation

Pytorch implementation of Feature Pyramid Network (FPN) for Object Detection

In this project we use both Resnet and Self-attention layer for cat, dog and flower classification.

TransFGU: A Top-down Approach to Fine-Grained Unsupervised Semantic Segmentation

Study of human inductive biases in CNNs and Transformers.

Crawl & visualize ICLR papers and reviews

Using fully convolutional networks for semantic segmentation with caffe for the cityscapes dataset

Code for CVPR2019 paper《Unequal Training for Deep Face Recognition with Long Tailed Noisy Data》

Learning infinite-resolution image processing with GAN and RL from unpaired image datasets, using a differentiable photo editing model.

Very large and sparse networks appear often in the wild and present unique algorithmic opportunities and challenges for the practitioner

ViDT: An Efficient and Effective Fully Transformer-based Object Detector

PyTorch code for EMNLP 2021 paper: Don't be Contradicted with Anything! CI-ToD: Towards Benchmarking Consistency for Task-oriented Dialogue System

Tensors and neural networks in Haskell

THIS IS THE OLD PYMC PROJECT. PLEASE USE PYMC3 INSTEAD: