Reproduce results and replicate training fo T0 (Multitask Prompted Training Enables Zero-Shot Task Generalization)

Last update: Dec 27, 2022

Related tags

Overview

T-Zero

This repository serves primarily as codebase and instructions for training, evaluation and inference of T0.

T0 is the model developed in Multitask Prompted Training Enables Zero-Shot Task Generalization. In this paper, we demonstrate that massive multitask prompted fine-tuning is extremely effective to obtain task zero-shot generalization. T0 outperforms or matches GPT-3 while being 16x smaller.

While the codebase in this repository mainly reproduces and replicates the training and evaluation of T0, it will be useful for future research on massively multitask fine-tuning.

Training: reproducing (or replicating) the massively multitask fine-tuning
Evaluation: reproducing the main results reported in the paper
Inference: running inference with T0

Citation

If you find this resource useful, please cite the paper introducing T0:

@misc{sanh2021multitask,
      title={Multitask Prompted Training Enables Zero-Shot Task Generalization},
      author={Victor Sanh and Albert Webson and Colin Raffel and Stephen H. Bach and Lintang Sutawika and Zaid Alyafeai and Antoine Chaffin and Arnaud Stiegler and Teven Le Scao and Arun Raja and Manan Dey and M Saiful Bari and Canwen Xu and Urmish Thakker and Shanya Sharma Sharma and Eliza Szczechla and Taewoon Kim and Gunjan Chhablani and Nihal Nayak and Debajyoti Datta and Jonathan Chang and Mike Tian-Jian Jiang and Han Wang and Matteo Manica and Sheng Shen and Zheng Xin Yong and Harshit Pandey and Rachel Bawden and Thomas Wang and Trishala Neeraj and Jos Rozen and Abheesht Sharma and Andrea Santilli and Thibault Fevry and Jason Alan Fries and Ryan Teehan and Stella Biderman and Leo Gao and Tali Bers and Thomas Wolf and Alexander M. Rush},
      year={2021},
      eprint={2110.08207},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}

Reproduce results and replicate training fo T0 (Multitask Prompted Training Enables Zero-Shot Task Generalization)

Related tags

Overview

T-Zero

Contents

Citation

Owner

BigScience Workshop

Example how to deploy deep learning model with aiohttp.

Decompose to Adapt: Cross-domain Object Detection via Feature Disentanglement

VideoGPT: Video Generation using VQ-VAE and Transformers

FairyTailor: Multimodal Generative Framework for Storytelling

Temporally Coherent GAN SIGGRAPH project.

Implement the Pareto Optimizer and pcgrad to make a self-adaptive loss for multi-task

This is the open-source reference implementation of the SIGGRAPH 2021 paper Intersection-free Rigid Body Dynamics.

Codes for SIGIR'22 Paper 'On-Device Next-Item Recommendation with Self-Supervised Knowledge Distillation'

The implementation of ICASSP 2020 paper "Pixel-level self-paced learning for super-resolution"

Qimera: Data-free Quantization with Synthetic Boundary Supporting Samples

Code of 3D Shape Variational Autoencoder Latent Disentanglement via Mini-Batch Feature Swapping for Bodies and Faces

Machine Learning Platform for Kubernetes

A GOOD REPRESENTATION DETECTS NOISY LABELS

object detection; robust detection; ACM MM21 grand challenge; Security AI Challenger Phase VII

SPCL: A New Framework for Domain Adaptive Semantic Segmentation via Semantic Prototype-based Contrastive Learning

Canonical Capsules: Unsupervised Capsules in Canonical Pose (NeurIPS 2021)

Deep Reinforcement Learning for Multiplayer Online Battle Arena

We have made you a wrapper you can't refuse

Official repository for "Deep Recurrent Neural Network with Multi-scale Bi-directional Propagation for Video Deblurring".

Author Disambiguation using Knowledge Graph Embeddings with Literals