Code for our paper "MG-GAN: A Multi-Generator Model Preventing Out-of-Distribution Samples in Pedestrian Trajectory Prediction" published at ICCV 2021.

Last update: Jan 05, 2023

Related tags

Deep Learning MG-GAN

Overview

MG-GAN: A Multi-Generator Model Preventing Out-of-Distribution Samples in Pedestrian Trajectory Prediction

This repository contains the code for the paper

MG-GAN: A Multi-Generator Model Preventing Out-of-Distribution Samples in Pedestrian Trajectory Prediction
Patrick Dendorfer*, Sven Elflein*, Laura Leal-Taixé (* equal contribution)
International Conference on Computer Vision (ICCV), 2021

Motivation

The distribution over future trajectories of pedestrians is often multi-modal and does not have connected support (a).

We found that single generator GANs introduce out-of-distribution (OOD) samples in this case due to GANs mapping the continuous latent variable z with a continuous function (b). These OOD samples might introduce unforseen behavior in real world applications, such as autonomous driving.

To resolve this problem, we propose to learn the target distribution in a piecewise manner using multiple generators, effectively preventing OOD samples (c).

Model

Our model consists of four key components: Encoding modules, Attention modules, and our novel contribution PM-Network learning a distribution over multiple Generators.

Setup

First, setup Python environment

conda create -f environment.yml -n mggan
conda activate mggan

Then, download the datasets (data.zip) from here and unzip in the root of this repository

unzip data.zip

which will create a folder ./data/datasets.

Training

Models can be trained using the script mggan/model/train.py using the following command

python mggan/models/pinet_multi_generator/train.py --name <name_of_experiment> --num_gens <number_of_generators>  --dataset <dataset_name> --epochs 50

This generates a output folder in ./logs/<name_of_experiment> with Tensorboard logs and the model checkpoints. You can use tensorboard --logdir ./logs/<name_of_experiment> to monitor the training process.

Evaluation

For evaluation of metrics (ADE, FDE, Precison, Recall) for k=1 to k=20 predictions, use

python scripts/evaluate.py --model_path <path_to_model_directory>  --output_folder <folder_to_store_result_csv>

One can use --eval-set <dataset_name> to evaluate models on other test sets than the dataset the model was trained on. This is useful to evaluate the BIWI models on the Garden of Forking Paths dataset (gofp) for which we report results in the paper.

Pre-trained models

We provide pre-trained models for MG-GAN with 2-8 generators together with the training configurations, on the BIWI datasets and Stanford Drone dataset (SDD) here.

Citation

If our work is useful to you, please consider citing

@inproceedings{dendorfer2021iccv,
  title={MG-GAN: A Multi-Generator Model Preventing Out-of-Distribution Samples in Pedestrian Trajectory Prediction}, 
  author={Dendorfer, Patrick and Elflein, Sven and Leal-Taixé, Laura},
  month={October}
  year={2021},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
  }

The implementation of the algorithm in the paper "Safe Deep Semi-Supervised Learning for Unseen-Class Unlabeled Data" published in ICML 2020.

DS3L This is the code for paper "Safe Deep Semi-Supervised Learning for Unseen-Class Unlabeled Data" published in ICML 2020. Setups The code is implem

36 Oct 19, 2022

This is the repo for the paper `SumGNN: Multi-typed Drug Interaction Prediction via Efficient Knowledge Graph Summarization'. (published in Bioinformatics'21)

SumGNN: Multi-typed Drug Interaction Prediction via Efficient Knowledge Graph Summarization This is the code for our paper ``SumGNN: Multi-typed Drug

58 Dec 21, 2022

Code for ICCV 2021 paper "Distilling Holistic Knowledge with Graph Neural Networks"

HKD Code for ICCV 2021 paper "Distilling Holistic Knowledge with Graph Neural Networks" cifia-100 result The implementation of compared methods are ba

30 Dec 18, 2022

code for ICCV 2021 paper 'Generalized Source-free Domain Adaptation'

G-SFDA Code (based on pytorch 1.3) for our ICCV 2021 paper 'Generalized Source-free Domain Adaptation'. [project] [paper]. Dataset preparing Download

84 Dec 26, 2022

Code for ICCV 2021 paper: ARAPReg: An As-Rigid-As Possible Regularization Loss for Learning Deformable Shape Generators..

ARAPReg Code for ICCV 2021 paper: ARAPReg: An As-Rigid-As Possible Regularization Loss for Learning Deformable Shape Generators.. Installation The cod

132 Nov 28, 2022

Comments

request to visualizer

Hello author！ I admire your work and would like to reproduce your results. There is a small requirement here that needs to trouble you. Do you have a visual code, which has shown the effect in your paper. Thanks again for your work and contributions！

opened by 12num 0
Question regarding Garden of Forking Path Dataset

Hello,

I see there are more scenes in the test set (ETH, Hotel, and ZARA1) than the train set (ETH) in your pre-processed dataset of GOFP. Could you kindly elaborate on why it is that?

Thanks, Sourav Das

opened by SodaCoder 0
Question about ETH&UCY Dataset

Hi, I notice that trajectories in some datasets are not consistent with provided in Social GAN. May I ask how do you preprocess your data? It will be helpful to conduct my experiments in a fair environment. Thanks!

opened by HRHLALALA 1
Reproducible MG-GAN code for the FPD dataset

Hello Patrick, Sven,

This is Sourav Das, a 1st year Ph.D. student at the University of Padova, Italy.

This Github repository has the reproducible implementation for the datasets: ETH, Hotel, Social_Stanford_Synthetic, Stanford, Univ, Zara1, Zara2, and GOFP.

I would like to reproduce the results on FPD datasets also. Could you kindly share with me the code with support for the FPD dataset?

Here is my Github: https://github.com/SodaCoder

Thanks in advance,

opened by SodaCoder 1

Releases(1.0)

1.0(Aug 14, 2021)

data.zip contains the datasets, checkpoints.zip contains pre-trained models for BIWI and Stanford Drone datasets.
Source code(tar.gz)
Source code(zip)
checkpoints.zip(296.92 MB)
data.zip(82.24 MB)

Code for our paper "MG-GAN: A Multi-Generator Model Preventing Out-of-Distribution Samples in Pedestrian Trajectory Prediction" published at ICCV 2021.

Related tags

Overview

MG-GAN: A Multi-Generator Model Preventing Out-of-Distribution Samples in Pedestrian Trajectory Prediction

Motivation

Model

Setup

Training

Evaluation

Pre-trained models

Citation

You might also like...

The implementation of the algorithm in the paper "Safe Deep Semi-Supervised Learning for Unseen-Class Unlabeled Data" published in ICML 2020.

This is the repo for the paper `SumGNN: Multi-typed Drug Interaction Prediction via Efficient Knowledge Graph Summarization'. (published in Bioinformatics'21)

Code for ICCV 2021 paper "Distilling Holistic Knowledge with Graph Neural Networks"

code for ICCV 2021 paper 'Generalized Source-free Domain Adaptation'

Code for ICCV 2021 paper: ARAPReg: An As-Rigid-As Possible Regularization Loss for Learning Deformable Shape Generators..

Code for the ICCV 2021 paper "Pixel Difference Networks for Efficient Edge Detection" (Oral).

Sync2Gen Code for ICCV 2021 paper: Scene Synthesis via Uncertainty-Driven Attribute Synchronization

Code for the paper "Spatio-temporal Self-Supervised Representation Learning for 3D Point Clouds" (ICCV 2021)

Code release for ICCV 2021 paper "Anticipative Video Transformer"

Comments

request to visualizer

Question regarding Garden of Forking Path Dataset

Question about ETH&UCY Dataset

Reproducible MG-GAN code for the FPD dataset

Releases(1.0)

1.0(Aug 14, 2021)

Owner

Sven

Leaderboard, taxonomy, and curated list of few-shot object detection papers.

A Python library for unevenly-spaced time series analysis

CoReD: Generalizing Fake Media Detection with Continual Representation using Distillation (ACMMM'21 Oral Paper)

Official PyTorch implementation of Less is More: Pay Less Attention in Vision Transformers.

Reinforcement learning for self-driving in a 3D simulation

PSGAN running with ncnn⚡妆容迁移/仿妆⚡Imitation Makeup/Makeup Transfer⚡

PyTorch implementation for "HyperSPNs: Compact and Expressive Probabilistic Circuits", NeurIPS 2021

[ICRA2021] Reconstructing Interactive 3D Scene by Panoptic Mapping and CAD Model Alignment

Official Repo for Ground-aware Monocular 3D Object Detection for Autonomous Driving

PiRapGenerator - Make anyone rap the digits of pi

The King is Naked: on the Notion of Robustness for Natural Language Processing

StyleGAN2 with adaptive discriminator augmentation (ADA) - Official TensorFlow implementation

Addition of pseudotorsion caclulation eta, theta, eta', and theta' to barnaba package

SenseNet is a sensorimotor and touch simulator for deep reinforcement learning research

Evaluation framework for testing segmentation networks in PyTorch

The Python3 import playground

Taichi Course Homework Template

Ranking Models in Unlabeled New Environments （iccv21）

This program creates a formatted excel file which highlights the undervalued stock according to Graham's number.

Supervised forecasting of sequential data in Python.