Code for our paper "MG-GAN: A Multi-Generator Model Preventing Out-of-Distribution Samples in Pedestrian Trajectory Prediction" published at ICCV 2021.

Last update: Jan 05, 2023

Related tags

Deep Learning MG-GAN

Overview

MG-GAN: A Multi-Generator Model Preventing Out-of-Distribution Samples in Pedestrian Trajectory Prediction

This repository contains the code for the paper

MG-GAN: A Multi-Generator Model Preventing Out-of-Distribution Samples in Pedestrian Trajectory Prediction
Patrick Dendorfer*, Sven Elflein*, Laura Leal-Taixé (* equal contribution)
International Conference on Computer Vision (ICCV), 2021

Motivation

The distribution over future trajectories of pedestrians is often multi-modal and does not have connected support (a).

We found that single generator GANs introduce out-of-distribution (OOD) samples in this case due to GANs mapping the continuous latent variable z with a continuous function (b). These OOD samples might introduce unforseen behavior in real world applications, such as autonomous driving.

To resolve this problem, we propose to learn the target distribution in a piecewise manner using multiple generators, effectively preventing OOD samples (c).

Model

Our model consists of four key components: Encoding modules, Attention modules, and our novel contribution PM-Network learning a distribution over multiple Generators.

Setup

First, setup Python environment

conda create -f environment.yml -n mggan
conda activate mggan

Then, download the datasets (data.zip) from here and unzip in the root of this repository

unzip data.zip

which will create a folder ./data/datasets.

Training

Models can be trained using the script mggan/model/train.py using the following command

python mggan/models/pinet_multi_generator/train.py --name <name_of_experiment> --num_gens <number_of_generators>  --dataset <dataset_name> --epochs 50

This generates a output folder in ./logs/<name_of_experiment> with Tensorboard logs and the model checkpoints. You can use tensorboard --logdir ./logs/<name_of_experiment> to monitor the training process.

Evaluation

For evaluation of metrics (ADE, FDE, Precison, Recall) for k=1 to k=20 predictions, use

python scripts/evaluate.py --model_path <path_to_model_directory>  --output_folder <folder_to_store_result_csv>

One can use --eval-set <dataset_name> to evaluate models on other test sets than the dataset the model was trained on. This is useful to evaluate the BIWI models on the Garden of Forking Paths dataset (gofp) for which we report results in the paper.

Pre-trained models

We provide pre-trained models for MG-GAN with 2-8 generators together with the training configurations, on the BIWI datasets and Stanford Drone dataset (SDD) here.

Citation

If our work is useful to you, please consider citing

@inproceedings{dendorfer2021iccv,
  title={MG-GAN: A Multi-Generator Model Preventing Out-of-Distribution Samples in Pedestrian Trajectory Prediction}, 
  author={Dendorfer, Patrick and Elflein, Sven and Leal-Taixé, Laura},
  month={October}
  year={2021},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
  }

The implementation of the algorithm in the paper "Safe Deep Semi-Supervised Learning for Unseen-Class Unlabeled Data" published in ICML 2020.

DS3L This is the code for paper "Safe Deep Semi-Supervised Learning for Unseen-Class Unlabeled Data" published in ICML 2020. Setups The code is implem

36 Oct 19, 2022

This is the repo for the paper `SumGNN: Multi-typed Drug Interaction Prediction via Efficient Knowledge Graph Summarization'. (published in Bioinformatics'21)

SumGNN: Multi-typed Drug Interaction Prediction via Efficient Knowledge Graph Summarization This is the code for our paper ``SumGNN: Multi-typed Drug

58 Dec 21, 2022

Code for ICCV 2021 paper "Distilling Holistic Knowledge with Graph Neural Networks"

HKD Code for ICCV 2021 paper "Distilling Holistic Knowledge with Graph Neural Networks" cifia-100 result The implementation of compared methods are ba

30 Dec 18, 2022

code for ICCV 2021 paper 'Generalized Source-free Domain Adaptation'

G-SFDA Code (based on pytorch 1.3) for our ICCV 2021 paper 'Generalized Source-free Domain Adaptation'. [project] [paper]. Dataset preparing Download

84 Dec 26, 2022

Code for ICCV 2021 paper: ARAPReg: An As-Rigid-As Possible Regularization Loss for Learning Deformable Shape Generators..

ARAPReg Code for ICCV 2021 paper: ARAPReg: An As-Rigid-As Possible Regularization Loss for Learning Deformable Shape Generators.. Installation The cod

132 Nov 28, 2022

Comments

request to visualizer

Hello author！ I admire your work and would like to reproduce your results. There is a small requirement here that needs to trouble you. Do you have a visual code, which has shown the effect in your paper. Thanks again for your work and contributions！

opened by 12num 0
Question regarding Garden of Forking Path Dataset

Hello,

I see there are more scenes in the test set (ETH, Hotel, and ZARA1) than the train set (ETH) in your pre-processed dataset of GOFP. Could you kindly elaborate on why it is that?

Thanks, Sourav Das

opened by SodaCoder 0
Question about ETH&UCY Dataset

Hi, I notice that trajectories in some datasets are not consistent with provided in Social GAN. May I ask how do you preprocess your data? It will be helpful to conduct my experiments in a fair environment. Thanks!

opened by HRHLALALA 1
Reproducible MG-GAN code for the FPD dataset

Hello Patrick, Sven,

This is Sourav Das, a 1st year Ph.D. student at the University of Padova, Italy.

This Github repository has the reproducible implementation for the datasets: ETH, Hotel, Social_Stanford_Synthetic, Stanford, Univ, Zara1, Zara2, and GOFP.

I would like to reproduce the results on FPD datasets also. Could you kindly share with me the code with support for the FPD dataset?

Here is my Github: https://github.com/SodaCoder

Thanks in advance,

opened by SodaCoder 1

Releases(1.0)

1.0(Aug 14, 2021)

data.zip contains the datasets, checkpoints.zip contains pre-trained models for BIWI and Stanford Drone datasets.
Source code(tar.gz)
Source code(zip)
checkpoints.zip(296.92 MB)
data.zip(82.24 MB)

Code for our paper "MG-GAN: A Multi-Generator Model Preventing Out-of-Distribution Samples in Pedestrian Trajectory Prediction" published at ICCV 2021.

Related tags

Overview

MG-GAN: A Multi-Generator Model Preventing Out-of-Distribution Samples in Pedestrian Trajectory Prediction

Motivation

Model

Setup

Training

Evaluation

Pre-trained models

Citation

You might also like...

The implementation of the algorithm in the paper "Safe Deep Semi-Supervised Learning for Unseen-Class Unlabeled Data" published in ICML 2020.

This is the repo for the paper `SumGNN: Multi-typed Drug Interaction Prediction via Efficient Knowledge Graph Summarization'. (published in Bioinformatics'21)

Code for ICCV 2021 paper "Distilling Holistic Knowledge with Graph Neural Networks"

code for ICCV 2021 paper 'Generalized Source-free Domain Adaptation'

Code for ICCV 2021 paper: ARAPReg: An As-Rigid-As Possible Regularization Loss for Learning Deformable Shape Generators..

Code for the ICCV 2021 paper "Pixel Difference Networks for Efficient Edge Detection" (Oral).

Sync2Gen Code for ICCV 2021 paper: Scene Synthesis via Uncertainty-Driven Attribute Synchronization

Code for the paper "Spatio-temporal Self-Supervised Representation Learning for 3D Point Clouds" (ICCV 2021)

Code release for ICCV 2021 paper "Anticipative Video Transformer"

Comments

request to visualizer

Question regarding Garden of Forking Path Dataset

Question about ETH&UCY Dataset

Reproducible MG-GAN code for the FPD dataset

Releases(1.0)

1.0(Aug 14, 2021)

Owner

Sven

The PyTorch implementation of DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision.

Repository for the semantic WMI loss

Implementation of Bottleneck Transformer in Pytorch

(Python, R, C/C++) Isolation Forest and variations such as SCiForest and EIF, with some additions (outlier detection + similarity + NA imputation)

The MATH Dataset

Source code and dataset of the paper "Contrastive Adaptive Propagation Graph Neural Networks forEfficient Graph Learning"

Banglore House Prediction Using Flask Server (Python)

FuseDream: Training-Free Text-to-Image Generationwith Improved CLIP+GAN Space OptimizationFuseDream: Training-Free Text-to-Image Generationwith Improved CLIP+GAN Space Optimization

Official repository for Jia, Raghunathan, Göksel, and Liang, "Certified Robustness to Adversarial Word Substitutions" (EMNLP 2019)

Multi-Scale Aligned Distillation for Low-Resolution Detection (CVPR2021)

Understanding and Overcoming the Challenges of Efficient Transformer Quantization

Reinforcement Learning via Supervised Learning

🚀 An end-to-end ML applications using PyTorch, W&B, FastAPI, Docker, Streamlit and Heroku

Deep Face Recognition in PyTorch

A complete speech segmentation system using Kaldi and x-vectors for voice activity detection (VAD) and speaker diarisation.

Aydin is a user-friendly, feature-rich, and fast image denoising tool

Checking fibonacci - Generating the Fibonacci sequence is a classic recursive problem

A simple rest api that classifies pneumonia infection weather it is Normal, Pneumonia Virus or Pneumonia Bacteria from a chest-x-ray image.

PaSST: Efficient Training of Audio Transformers with Patchout

LineBoard - Python+React+MySQL-白板即時系統改善人群行為