Generalized Decision Transformer for Offline Hindsight Information Matching

If you use this codebase for your research, please cite the paper:

@article{furuta2021generalized,
  title={Generalized Decision Transformer for Offline Hindsight Information Matching},
  author={Hiroki Furuta and Yutaka Matsuo and Shixiang Shane Gu},
  journal={arXiv preprint arXiv:2111.10364},
  year={2021}
}

Installation

Experiments require MuJoCo. Follow the instructions in the mujoco-py repo to install. Then, dependencies can be installed with the following command:

conda env create -f conda_env.yml

Downloading datasets

Datasets are stored in the data directory. Install the D4RL repo, following the instructions there. Then, run the following script in order to download the datasets and save them in our format:

python download_d4rl_datasets.py

Run experiments

Run train_cdt.py to train Categorical DT:

python train_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'reward' --save_model True

python train_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'xvel' --save_model True

Run eval_cdt.py to eval CDT using saved weights:

python eval_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'reward' --save_rollout True
python eval_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'xvel' --save_rollout True

For Bi-directional DT, run train_bdt.py & eval_bdtf.py

python train_bdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --z_dim 16 --save_model True
python eval_bdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --z_dim 16 --save_rollout True

Reference

This repository is developed on top of original Decision Transformer.

Generalized Decision Transformer for Offline Hindsight Information Matching

Related tags

Overview

Generalized Decision Transformer for Offline Hindsight Information Matching

Installation

Downloading datasets

Run experiments

Reference

Owner

Hiroki Furuta

This is the official implementation of Elaborative Rehearsal for Zero-shot Action Recognition (ICCV2021)

Hi Guys, here I am providing examples, which will help you in Lerarning Python

Ego4d dataset repository. Download the dataset, visualize, extract features & example usage of the dataset

Implementation of "Semi-supervised Domain Adaptive Structure Learning"

SmoothGrad implementation in PyTorch

PyTorch implementation for OCT-GAN Neural ODE-based Conditional Tabular GANs (WWW 2021)

Code for our paper Aspect Sentiment Quad Prediction as Paraphrase Generation in EMNLP 2021.

we propose EfficientDerain for high-efficiency single-image deraining

Finetune SSL models for MOS prediction

Retrieval.pytorch - The code we used in [2020 DIGIX]

Awesome Graph Classification - A collection of important graph embedding, classification and representation learning papers with implementations.

DeepLab is a state-of-art deep learning system for semantic image segmentation built on top of Caffe.

General Assembly Capstone: NBA Game Predictor

A clear, concise, simple yet powerful and efficient API for deep learning.

StarGAN v2 - Official PyTorch Implementation (CVPR 2020)

Open source Python module for computer vision

JAX code for the paper "Control-Oriented Model-Based Reinforcement Learning with Implicit Differentiation"

Implementation of ECCV20 paper: the devil is in classification: a simple framework for long-tail object detection and instance segmentation

This repository is the offical Pytorch implementation of ContextPose: Context Modeling in 3D Human Pose Estimation: A Unified Perspective (CVPR 2021).

Data and extra materials for the food safety publications classifier