Generalized Decision Transformer for Offline Hindsight Information Matching

If you use this codebase for your research, please cite the paper:

@article{furuta2021generalized,
  title={Generalized Decision Transformer for Offline Hindsight Information Matching},
  author={Hiroki Furuta and Yutaka Matsuo and Shixiang Shane Gu},
  journal={arXiv preprint arXiv:2111.10364},
  year={2021}
}

Installation

Experiments require MuJoCo. Follow the instructions in the mujoco-py repo to install. Then, dependencies can be installed with the following command:

conda env create -f conda_env.yml

Downloading datasets

Datasets are stored in the data directory. Install the D4RL repo, following the instructions there. Then, run the following script in order to download the datasets and save them in our format:

python download_d4rl_datasets.py

Run experiments

Run train_cdt.py to train Categorical DT:

python train_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'reward' --save_model True

python train_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'xvel' --save_model True

Run eval_cdt.py to eval CDT using saved weights:

python eval_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'reward' --save_rollout True
python eval_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'xvel' --save_rollout True

For Bi-directional DT, run train_bdt.py & eval_bdtf.py

python train_bdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --z_dim 16 --save_model True
python eval_bdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --z_dim 16 --save_rollout True

Reference

This repository is developed on top of original Decision Transformer.

Generalized Decision Transformer for Offline Hindsight Information Matching

Related tags

Overview

Generalized Decision Transformer for Offline Hindsight Information Matching

Installation

Downloading datasets

Run experiments

Reference

Owner

Hiroki Furuta

Baseline and template code for node21 detection track

A repository for benchmarking neural vocoders by their quality and speed.

Reverse engineering Rosetta 2 in M1 Mac

AAI supports interdisciplinary research to help better understand human, animal, and artificial cognition.

Exploring Relational Context for Multi-Task Dense Prediction [ICCV 2021]

Real-time LIDAR-based Urban Road and Sidewalk detection for Autonomous Vehicles 🚗

Official repo for QHack—the quantum machine learning hackathon

Import Python modules from dicts and JSON formatted documents.

A library for researching neural networks compression and acceleration methods.

This repository contains the source code of Auto-Lambda and baselines from the paper, Auto-Lambda: Disentangling Dynamic Task Relationships.

A voice recognition assistant similar to amazon alexa, siri and google assistant.

Rewrite ultralytics/yolov5 v6.0 opencv inference code based on numpy, no need to rely on pytorch

Simple tool to combine(merge) onnx models. Simple Network Combine Tool for ONNX.

A forwarding MPI implementation that can use any other MPI implementation via an MPI ABI

Library extending Jupyter notebooks to integrate with Apache TinkerPop and RDF SPARQL.

TensorFlow implementation of Deep Reinforcement Learning papers

Implementation of light baking system for ray tracing based on Activision's UberBake

[UNMAINTAINED] Automated machine learning for analytics & production

Multi-Modal Machine Learning toolkit based on PyTorch.

PyTorch implementation of "Optimization Planning for 3D ConvNets"