Mae segmentation - Reproduction of semantic segmentation using masked autoencoder (mae)

Last update: Dec 17, 2022

Overview

ADE20k Semantic segmentation with MAE

Getting started

Install the mmsegmentation library and some required packages.

pip install mmcv-full==1.3.0 mmsegmentation==0.11.0
pip install scipy timm==0.3.2

Install apex for mixed-precision training

git clone https://github.com/NVIDIA/apex
cd apex
pip install -v --disable-pip-version-check --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./

Follow the guide in mmseg to prepare the ADE20k dataset.

Fine-tuning for Reproducing Results of MAE ViT-Base

Command:

tools/dist_train.sh configs/mae/upernet_mae_base_12_512_slide_160k_ade20k.py 8 --seed 0  --options model.pretrained=https://dl.fbaipublicfiles.com/mae/pretrain/mae_pretrain_vit_base.pth

Expected results log(paper results: 48.1 mIoU):

+--------+-------+-------+-------+
| Scope  | mIoU  | mAcc  | aAcc  |
+--------+-------+-------+-------+
| global | 48.15 | 58.99 | 83.05 |
+--------+-------+-------+-------+

Evaluation

Command format:

tools/dist_test.sh  <CONFIG_PATH> <CHECKPOINT_PATH> <NUM_GPUS> --eval mIoU

Acknowledgment

This code is built using the mmsegmentation library, Timm library, the Swin repository, XCiT, SETR, BEiT and the MAE repository.

Mae segmentation - Reproduction of semantic segmentation using masked autoencoder (mae)

Related tags

Overview

ADE20k Semantic segmentation with MAE

Getting started

Fine-tuning for Reproducing Results of MAE ViT-Base

Evaluation

Acknowledgment

Owner

A wrapper around SageMaker ML Lineage Tracking extending ML Lineage to end-to-end ML lifecycles, including additional capabilities around Feature Store groups, queries, and other relevant artifacts.

[ICLR 2021] Heteroskedastic and Imbalanced Deep Learning with Adaptive Regularization

Autoregressive Predictive Coding: An unsupervised autoregressive model for speech representation learning

2021 credit card consuming recommendation

EfficientDet (Scalable and Efficient Object Detection) implementation in Keras and Tensorflow

A stock generator that assess a list of stocks and returns the best stocks for investing and money allocations based on users choices of volatility, duration and number of stocks

A collection of Google research projects related to Federated Learning and Federated Analytics.

StarGAN - Official PyTorch Implementation (CVPR 2018)

Safe Control for Black-box Dynamical Systems via Neural Barrier Certificates

An end-to-end machine learning web app to predict rugby scores (Pandas, SQLite, Keras, Flask, Docker)

A project which aims to protect your privacy using inexpensive hardware and easily modifiable software

Unofficial implementation of the ImageNet, CIFAR 10 and SVHN Augmentation Policies learned by AutoAugment using pillow

Official Pytorch Implementation of Unsupervised Image Denoising with Frequency Domain Knowledge

Machine learning evaluation metrics, implemented in Python, R, Haskell, and MATLAB / Octave

Uni-Fold: Training your own deep protein-folding models

The Body Part Regression (BPR) model translates the anatomy in a radiologic volume into a machine-interpretable form.

Invertible conditional GANs for image editing

DTCN IJCAI - Sequential prediction learning framework and algorithm

My course projects for the 2021 Spring Machine Learning course at the National Taiwan University (NTU)

Ontologysim: a Owlready2 library for applied production simulation