Generalized Decision Transformer for Offline Hindsight Information Matching

If you use this codebase for your research, please cite the paper:

@article{furuta2021generalized,
  title={Generalized Decision Transformer for Offline Hindsight Information Matching},
  author={Hiroki Furuta and Yutaka Matsuo and Shixiang Shane Gu},
  journal={arXiv preprint arXiv:2111.10364},
  year={2021}
}

Installation

Experiments require MuJoCo. Follow the instructions in the mujoco-py repo to install. Then, dependencies can be installed with the following command:

conda env create -f conda_env.yml

Downloading datasets

Datasets are stored in the data directory. Install the D4RL repo, following the instructions there. Then, run the following script in order to download the datasets and save them in our format:

python download_d4rl_datasets.py

Run experiments

Run train_cdt.py to train Categorical DT:

python train_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'reward' --save_model True

python train_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'xvel' --save_model True

Run eval_cdt.py to eval CDT using saved weights:

python eval_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'reward' --save_rollout True
python eval_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'xvel' --save_rollout True

For Bi-directional DT, run train_bdt.py & eval_bdtf.py

python train_bdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --z_dim 16 --save_model True
python eval_bdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --z_dim 16 --save_rollout True

Reference

This repository is developed on top of original Decision Transformer.

Generalized Decision Transformer for Offline Hindsight Information Matching

Related tags

Overview

Generalized Decision Transformer for Offline Hindsight Information Matching

Installation

Downloading datasets

Run experiments

Reference

Owner

Hiroki Furuta

Byte-based multilingual transformer TTS for low-resource/few-shot language adaptation.

An open-source Kazakh named entity recognition dataset (KazNERD), annotation guidelines, and baseline NER models.

Cervix ROI Segmentation Using U-NET

Pytorch implementation of paper "Efficient Nearest Neighbor Language Models" (EMNLP 2021)

This is a tensorflow-based rotation detection benchmark, also called AlphaRotate.

A basic implementation of Layer-wise Relevance Propagation (LRP) in PyTorch.

The BCNet related data and inference model.

A framework for GPU based high-performance medical image processing and visualization

Easy genetic ancestry predictions in Python

TuckER: Tensor Factorization for Knowledge Graph Completion

CrossMLP - The repository offers the official implementation of our BMVC 2021 paper (oral) in PyTorch.

Implementations for the ICLR-2021 paper: SEED: Self-supervised Distillation For Visual Representation.

MLP-Like Vision Permutator for Visual Recognition (PyTorch)

A setup script to generate ITK Python Wheels

Evolutionary Scale Modeling (esm): Pretrained language models for proteins

A machine learning library for spiking neural networks. Supports training with both torch and jax pipelines, and deployment to neuromorphic hardware.

One-line your code easily but still with the fun of doing so!

Image reconstruction done with untrained neural networks.

Code release for "Detecting Twenty-thousand Classes using Image-level Supervision".

Fine-tuning StyleGAN2 for Cartoon Face Generation