Co-GAIL: Learning Diverse Strategies for Human-Robot Collaboration

Related tags

Deep Learningcogail
Overview

CoGAIL

Table of Content

Overview

This repository is the implementation code of the paper "Co-GAIL: Learning Diverse Strategies for Human-Robot Collaboration"(arXiv, Project, Video) by Wang et al. at Stanford Vision and Learning Lab. In this repo, we provide our full implementation code of training and evaluation.

Installation

  • python 3.6+
conda create -n cogail python=3.6
conda activate cogail
  • iGibson 1.0 variant version for co-gail. For more details of iGibson installation please refer to Link
git clone https://github.com/j96w/iGibson.git --recursive
cd iGibson
git checkout cogail
python -m pip install -e .

Please also download the assets of iGibson (models of the objects, 3D scenes, etc.) follow the instruction. The data should be located at your_installation_path/igibson/data/. After downloaded the dataset, copy the modified robot and humanoid mesh file to this location as follows

cd urdfs
cp fetch.urdf your_installation_path/igibson/data/assets/models/fetch/.
cp camera.urdf your_installation_path/igibson/data/assets/models/grippers/basic_gripper/.
cp -r humanoid_hri your_installation_path/igibson/data/assets/models/.
  • other requirements
cd cogail
python -m pip install -r requirements.txt

Dataset

You can download the collected human-human collaboration demonstrations for Link. The demos for cogail_exp1_2dfq is collected by a pair of joysticks on an xbox controller. The demos for cogail_exp2_handover and cogail_exp3_seqmanip are collected with two phones on the teleoperation system RoboTurk. After downloaded the file, simply unzip them at cogail/ as follows

unzip dataset.zip
mv dataset your_installation_path/cogail/dataset

Training

There are three environments (cogail_exp1_2dfq, cogail_exp2_handover, cogail_exp3_seqmanip) implemented in this work. Please specify the choice of environment with --env-name

python scripts/train.py --env-name [cogail_exp1_2dfq / cogail_exp2_handover / cogail_exp3_seqmanip]

Evaluation

Evaluation on unseen human demos (replay evaluation):

python scripts/eval_replay.py --env-name [cogail_exp1_2dfq / cogail_exp2_handover / cogail_exp3_seqmanip]

Trained Checkpoints

You can download the trained checkpoints for all three environments from Link.

Acknowledgement

The cogail_exp1_2dfq is implemented with Pygame. The cogail_exp2_handover and cogail_exp3_seqmanip are implemented in iGibson v1.0.

The demos for robot manipulation in iGibson is collected with RoboTurk.

Code is based on the PyTorch GAIL implementation by ikostrikov (https://github.com/ikostrikov/pytorch-a2c-ppo-acktr-gail.git).

Citations

Please cite Co-GAIL if you use this repository in your publications:

@article{wang2021co,
  title={Co-GAIL: Learning Diverse Strategies for Human-Robot Collaboration},
  author={Wang, Chen and P{\'e}rez-D'Arpino, Claudia and Xu, Danfei and Fei-Fei, Li and Liu, C Karen and Savarese, Silvio},
  journal={arXiv preprint arXiv:2108.06038},
  year={2021}
}

License

Licensed under the MIT License

Owner
Jeremy Wang
Ph.D. student, Stanford
Jeremy Wang
Speech-Emotion-Analyzer - The neural network model is capable of detecting five different male/female emotions from audio speeches. (Deep Learning, NLP, Python)

Speech Emotion Analyzer The idea behind creating this project was to build a machine learning model that could detect emotions from the speech we have

Mitesh Puthran 965 Dec 24, 2022
PyTorch code of my ICDAR 2021 paper Vision Transformer for Fast and Efficient Scene Text Recognition (ViTSTR)

Vision Transformer for Fast and Efficient Scene Text Recognition (ICDAR 2021) ViTSTR is a simple single-stage model that uses a pre-trained Vision Tra

Rowel Atienza 198 Dec 27, 2022
This is the official implementation of the paper "Object Propagation via Inter-Frame Attentions for Temporally Stable Video Instance Segmentation".

ObjProp Introduction This is the official implementation of the paper "Object Propagation via Inter-Frame Attentions for Temporally Stable Video Insta

Anirudh S Chakravarthy 6 May 03, 2022
The 2nd Version Of Slothybot

SlothyBot Go to this website: "https://bitly.com/SlothyBot" The 2nd Version Of Slothybot. The Bot Has Many Features, Such As: Moderation Commands; Kic

Slothy 0 Jun 01, 2022
The PyTorch re-implement of a 3D CNN Tracker to extract coronary artery centerlines with state-of-the-art (SOTA) performance. (paper: 'Coronary artery centerline extraction in cardiac CT angiography using a CNN-based orientation classifier')

The PyTorch re-implement of a 3D CNN Tracker to extract coronary artery centerlines with state-of-the-art (SOTA) performance. (paper: 'Coronary artery centerline extraction in cardiac CT angiography

James 135 Dec 23, 2022
How to use TensorLayer

How to use TensorLayer While research in Deep Learning continues to improve the world, we use a bunch of tricks to implement algorithms with TensorLay

zhangrui 349 Dec 07, 2022
Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.

Pretrained Language Model This repository provides the latest pretrained language models and its related optimization techniques developed by Huawei N

HUAWEI Noah's Ark Lab 2.6k Jan 01, 2023
Distributed Evolutionary Algorithms in Python

DEAP DEAP is a novel evolutionary computation framework for rapid prototyping and testing of ideas. It seeks to make algorithms explicit and data stru

Distributed Evolutionary Algorithms in Python 4.9k Jan 05, 2023
Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks

MGANs Training & Testing code (torch), pre-trained models and supplementary materials for "Precomputed Real-Time Texture Synthesis with Markovian Gene

290 Nov 15, 2022
Torchserve server using a YoloV5 model running on docker with GPU and static batch inference to perform production ready inference.

Yolov5 running on TorchServe (GPU compatible) ! This is a dockerfile to run TorchServe for Yolo v5 object detection model. (TorchServe (PyTorch librar

82 Nov 29, 2022
Implementation for the "Surface Reconstruction from 3D Line Segments" paper.

Surface Reconstruction from 3D Line Segments Surface reconstruction from 3d line segments. Langlois, P. A., Boulch, A., & Marlet, R. In 2019 Internati

85 Jan 04, 2023
The codes and models in 'Gaze Estimation using Transformer'.

GazeTR We provide the code of GazeTR-Hybrid in "Gaze Estimation using Transformer". We recommend you to use data processing codes provided in GazeHub.

65 Dec 27, 2022
'Solving the sampling problem of the Sycamore quantum supremacy circuits

solve_sycamore This repo contains data, contraction code, and contraction order for the paper ''Solving the sampling problem of the Sycamore quantum s

Feng Pan 29 Nov 28, 2022
Official code for 'Pixel-wise Energy-biased Abstention Learning for Anomaly Segmentationon Complex Urban Driving Scenes'

PEBAL This repo contains the Pytorch implementation of our paper: Pixel-wise Energy-biased Abstention Learning for Anomaly Segmentation on Complex Urb

Yu Tian 117 Jan 03, 2023
v objective diffusion inference code for JAX.

v-diffusion-jax v objective diffusion inference code for JAX, by Katherine Crowson (@RiversHaveWings) and Chainbreakers AI (@jd_pressman). The models

Katherine Crowson 186 Dec 21, 2022
InsTrim: Lightweight Instrumentation for Coverage-guided Fuzzing

InsTrim The paper: InsTrim: Lightweight Instrumentation for Coverage-guided Fuzzing Build Prerequisite llvm-8.0-dev clang-8.0 cmake = 3.2 Make git cl

75 Dec 23, 2022
Genshin-assets - 👧 Public documentation & static assets for Genshin Impact data.

genshin-assets This repo provides easy access to the Genshin Impact assets, primarily for use on static sites. Sources Genshin Optimizer - An Artifact

Zerite Development 5 Nov 22, 2022
My implementation of Image Inpainting - A deep learning Inpainting model

Image Inpainting What is Image Inpainting Image inpainting is a restorative process that allows for the fixing or removal of unwanted parts within ima

Joshua V Evans 1 Dec 12, 2021
PyDeepFakeDet is an integrated and scalable tool for Deepfake detection.

PyDeepFakeDet An integrated and scalable library for Deepfake detection research. Introduction PyDeepFakeDet is an integrated and scalable Deepfake de

Junke, Wang 49 Dec 11, 2022
A PyTorch implementation of the continual learning experiments with deep neural networks

Brain-Inspired Replay A PyTorch implementation of the continual learning experiments with deep neural networks described in the following paper: Brain

182 Dec 27, 2022