ASFormer: Transformer for Action Segmentation

This repo provides training & inference code for BMVC 2021 paper: ASFormer: Transformer for Action Segmentation.

Enviroment

Pytorch == 1.1.0, torchvision == 0.3.0, python == 3.6, CUDA=10.1

Reproduce our results

1. Download the dataset data.zip at (https://mega.nz/#!O6wXlSTS!wcEoDT4Ctq5HRq_hV-aWeVF1_JB3cacQBQqOLjCIbc8) or (https://zenodo.org/record/3625992#.Xiv9jGhKhPY). 
2. Unzip the data.zip file to the current folder. There are three datasets in the ./data folder, i.e. ./data/breakfast, ./data/50salads, ./data/gtea
3. Download the pre-trained models at (https://pan.baidu.com/s/1zf-d-7eYqK-IxroBKTxDfg). There are pretrained models for three datasets, i.e. ./models/50salads, ./models/breakfast, ./models/gtea
4. Run python main.py --action=predict --dataset=50salads/gtea/breakfast --split=1/2/3/4/5 to generate predicted results for each split.
5. Run python eval.py --dataset=50salads/gtea/breakfast --split=0/1/2/3/4/5 to evaluate the performance. **NOTE**: split=0 will evaulate the average results for all splits, It needs to be done after you complete all split predictions.

Train your own model

Also, you can retrain the model by yourself with following command.

python main.py --action=train --dataset=50salads/gtea/breakfast --split=1/2/3/4/5

The training process is very stable in our experiments. It convergences very fast and is not sensitive to the number of training epochs.

Demo for using ASFormer as your backbone

In our paper, we replace the original TCN-based backbone model MS-TCN in ASRF with our ASFormer. The new model achieves even higher results on the 50salads dataset than the original ASRF. Code is Here.

If you find our repo useful, please give us a star and cite

@inproceedings{chinayi_ASformer,  
	author={Fangqiu Yi and Hongyu Wen and Tingting Jiang}, 
	booktitle={The British Machine Vision Conference (BMVC)},   
	title={ASFormer: Transformer for Action Segmentation},
	year={2021},  
}

Feel free to raise a issue if you got trouble with our code.

Official repo for BMVC2021 paper ASFormer: Transformer for Action Segmentation

Related tags

Overview

ASFormer: Transformer for Action Segmentation

Enviroment

Reproduce our results

Train your own model

Demo for using ASFormer as your backbone

Owner

FCOS: Fully Convolutional One-Stage Object Detection (ICCV'19)

Codes to pre-train T5 (Text-to-Text Transfer Transformer) models pre-trained on Japanese web texts

🌎 The Modern Declarative Data Flow Framework for the AI Empowered Generation.

Implementation of the paper "Language-agnostic representation learning of source code from structure and context".

Implementation of average- and worst-case robust flatness measures for adversarial training.

Fine-grained Post-training for Improving Retrieval-based Dialogue Systems - NAACL 2021

Pytorch Implementation of the paper "Cross-domain Correspondence Learning for Exemplar-based Image Translation"

Official PyTorch implementation of "The Center of Attention: Center-Keypoint Grouping via Attention for Multi-Person Pose Estimation" (ICCV 21).

Indoor Panorama Planar 3D Reconstruction via Divide and Conquer

Implementation of the paper "Fine-Tuning Transformers: Vocabulary Transfer"

Stochastic Scene-Aware Motion Prediction

Non-Homogeneous Poisson Process Intensity Modeling and Estimation using Measure Transport

A code generator from ONNX to PyTorch code

A simple Rock-Paper-Scissors game using CV in python

《DeepViT: Towards Deeper Vision Transformer》(2021)

Intel® Neural Compressor is an open-source Python library running on Intel CPUs and GPUs

A benchmark dataset for emulating atmospheric radiative transfer in weather and climate models with machine learning (NeurIPS 2021 Datasets and Benchmarks Track)

PyTorch implementation of federated learning framework based on the acceleration of global momentum

Hierarchical Metadata-Aware Document Categorization under Weak Supervision (WSDM'21)

Neural HMMs are all you need (for high-quality attention-free TTS)