TorchGRL is the source code for our paper Graph Convolution-Based Deep Reinforcement Learning for Multi-Agent Decision-Making in Mixed Traffic Environments for IV 2022.

Related tags

Deep LearningTorchGRL
Overview

TorchGRL

TorchGRL is the source code for our paper Graph Convolution-Based Deep Reinforcement Learning for Multi-Agent Decision-Making in Mixed Traffic Environments for IV 2022.TorchGRL is a modular simulation framework that integrates different GRL algorithms and SUMO simulation platform to realize the simulation of multi-agents decision-making algorithms in mixed traffic environment. You can adjust the test scenarios and the implemented GRL algorithm according to your needs.


Preparation

Before starting to carry out some relevant works on our framework, some preparations are required to be done.

Hardware

Our framework is developed based on a laptop, and the specific configuration is as follows:

  • Operating system: Ubuntu 20.04
  • RAM: 32 GB
  • CPU: Intel (R) Core (TM) i9-10980HK CPU @ 2.40GHz
  • GPU: RTX 2070

It should be noted that our program must be reproduced under the Ubuntu 20.04 operating system, and we strongly recommend using GPU for training.

Development Environment

Before compiling the code of our framework, you need to install the following development environment:

  • Ubuntu 20.04 with latest GPU driver
  • Pycharm
  • Anaconda
  • CUDA 11.1
  • cudnn-11.1, 8.0.5.39

Installation

Please download our GRL framework repository first:

git clone https://github.com/Jacklinkk/TorchGRL.git

Then enter the root directory of TorchGRL:

cd TorchGRL

and please be sure to run the below commands from /path/to/TorchGRL.

Installation of FLOW

The FLOW library will be firstly installed.

Firstly, enter the flow directory:

cd flow

Then, create a conda environment from flow library:

conda env create -f environment.yml

Activate conda environment:

conda activate TorchGCQ

Install flow from source code:

python setup.py develop

Installation of SUMO

SUMO simulation platform will be installed. Please make sure to run the below commands in the "TorchGRL" virtual environment.

Install via pip:

pip install eclipse-sumo

Setting in Pycharm:

In order to adopt SUMO correctly, you need to define the environment variable of SUMO_HOME in Pycharm. The specific directory is:

/home/…/.conda/envs/TorchGCQ/lib/python3.7/site-packages/sumo

Setting in Ubuntu:

At first, run:

gedit ~/.bashrc

then copy the path name of SUMO_HOME to “~/.bashrc”:

export SUMO_HOME=“/home/…/.conda/envs/TorchGCQ/lib/python3.7/site-packages/sumo”

Finally, run:

source ~/.bashrc

Installation of Pytorch and related libraries

Please make sure to run the below commands in the "TorchGRL" virtual environment.

Installation of Pytorch:

We use Pytorch version 1.9.0 for development under a specific version of CUDA and cudnn.

pip3 install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html

Installation of pytorch geometric:

Pytorch geometric is a Graph Neural Network (GNN) library upon Pytorch

pip install torch-scatter torch-sparse torch-cluster torch-spline-conv torch-geometric -f https://data.pyg.org/whl/torch-1.9.0+cu111.html

Installation of pfrl library

Please make sure to run the below commands in the "TorchGRL" virtual environment.

pfrl is a deep reinforcement learning library that implements various algorithms in Python using PyTorch.

Firstly, enter the pfrl directory:

cd pfrl

Then install from source code:

python setup.py develop

Instruction

flow folder

The flow folder is the root directory of the library after the FLOW library is installed through source code, including interface-related programs between DRL algorithms and SUMO platform.

Flow_Test folder

The Flow_Test folder includes the related programs of the test environment configuration; specifically, T_01.py is the core python program. If the program runs successfully, the environment configuration is successful.

pfrl folder

The pfrl folder is the root directory of the library after the deep reinforcement learning pfrl library is installed through source code, including all DRL related programs. The source program can be modified as needed.

GRLNet folder

The GRLNet folder contains the GRL neural network built in the Pytorch environment. You can modify the source code as needed or add your own neural network.

  • Pytorch_GRL.py constructs the fundamental neural network of GRL algorithms
  • Pytorch_GRL_Dueling.py constructs the dueling network of GRL algorithms

GRL_utils folder

The GRL_utils folder contains basic functions such as model training and testing, data storage, and curve drawing.

  • Train_and_Test.py contains the training and testing functions for the GRL model.
  • Data_Plot_Train.py is the function to plot the training data curve.
  • Data_Process_Test.py is the function to process the test data.
  • Fig folder stores the training data curve.
  • Logging_Training folder stores the training data generated by different GRL algorithms.
  • Logging_Test folder stores the testing data generated by different GRL algorithms.

GRL_Simulation folder

The GRL_Simulation folder is the core of our framework, which contains the core simulation program and some related functional programs.

  • main.py is the main program, containing the definition of FLOW parameters, as well as the controlling (start and end) of the simulation.
  • controller.py is the definition of vehicle control model based on FLOW library.
  • environment.py is the core program to build and initialize the simulation environment of SUMO.
  • network.py defines the road network.
  • registry_custom.py registers the simulation environment of SUMO to the gym library to realize the connection with GRL algorithms.
  • specific_environment.py defines the elements in MDPs, including state representation, action space and reward function.
  • Experiment folder is the core program of co-simulation under different GRL algorithms, including the initialization of the simulation environment, the initialization of the neural network, the training and testing of GRL algorithms, and the preservation of the training and testing results.
  • GRL_Trained_Models folder stores the trained GRL model when the training process ends.

Tutorial

You can simply run "main.py" in Pycharm to simulate the GRL algorithm, and observe the simulation process in SUMO platform. You can generate training plot such as Reward curve:

Verification of other algorithms

If you want to verify other algorithms, you can develop the source code as needed under the "Experiment folder", and don't forget to change the imported python script in "main.py". In addition, you can also construct your own network in GRLNet folder.

Verification of other traffic scenario

If you want to verify other traffic scenario, you can define a new scenario in "network.py". You can refer to the documentation of SUMO for more details .

Owner
XXQQ
XXQQ
Context Decoupling Augmentation for Weakly Supervised Semantic Segmentation

Context Decoupling Augmentation for Weakly Supervised Semantic Segmentation The code of: Context Decoupling Augmentation for Weakly Supervised Semanti

54 Dec 12, 2022
Point detection through multi-instance deep heatmap regression for sutures in endoscopy

Suture detection PyTorch This repo contains the reference implementation of suture detection model in PyTorch for the paper Point detection through mu

artificial intelligence in the area of cardiovascular healthcare 3 Jul 16, 2022
Improving Factual Completeness and Consistency of Image-to-text Radiology Report Generation

Improving Factual Completeness and Consistency of Image-to-text Radiology Report Generation The reference code of Improving Factual Completeness and C

46 Dec 15, 2022
A basic duplicate image detection service using perceptual image hash functions and nearest neighbor search, implemented using faiss, fastapi, and imagehash

Duplicate Image Detection Getting Started Install dependencies pip install -r requirements.txt Run service python main.py Testing Test with pytest How

Matthew Podolak 21 Nov 11, 2022
Official pytorch implementation of paper "Image-to-image Translation via Hierarchical Style Disentanglement".

HiSD: Image-to-image Translation via Hierarchical Style Disentanglement Official pytorch implementation of paper "Image-to-image Translation

364 Dec 14, 2022
Parameterized Explainer for Graph Neural Network

PGExplainer This is a Tensorflow implementation of the paper: Parameterized Explainer for Graph Neural Network https://arxiv.org/abs/2011.04573 NeurIP

Dongsheng Luo 89 Dec 12, 2022
PyTorch implementation of MuseMorphose, a Transformer-based model for music style transfer.

MuseMorphose This repository contains the official implementation of the following paper: Shih-Lun Wu, Yi-Hsuan Yang MuseMorphose: Full-Song and Fine-

Yating Music, Taiwan AI Labs 142 Jan 08, 2023
SatelliteNeRF - PyTorch-based Neural Radiance Fields adapted to satellite domain

SatelliteNeRF PyTorch-based Neural Radiance Fields adapted to satellite domain.

Kai Zhang 46 Nov 20, 2022
StyleTransfer - Open source style transfer project, based on VGG19

StyleTransfer - Open source style transfer project, based on VGG19

Patrick martins de lima 9 Dec 13, 2021
This folder contains the implementation of the multi-relational attribute propagation algorithm.

MrAP This folder contains the implementation of the multi-relational attribute propagation algorithm. It requires the package pytorch-scatter. Please

6 Dec 06, 2022
Code to use Augmented Shapiro Wilks Stopping, as well as code for the paper "Statistically Signifigant Stopping of Neural Network Training"

This codebase is being actively maintained, please create and issue if you have issues using it Basics All data files are included under losses and ea

J K Terry 32 Nov 09, 2021
Segmentation Training Pipeline

Segmentation Training Pipeline This package is a part of Musket ML framework. Reasons to use Segmentation Pipeline Segmentation Pipeline was developed

Musket ML 52 Dec 12, 2022
Research on controller area network Intrusion Detection Systems

Group members information Member 1: Lixue Liang Member 2: Yuet Lee Chan Member 3: Xinruo Zhang Member 4: Yifei Han User Manual Generate Attack Packets

Roche 4 Aug 30, 2022
Translate darknet to tensorflow. Load trained weights, retrain/fine-tune using tensorflow, export constant graph def to mobile devices

Intro Real-time object detection and classification. Paper: version 1, version 2. Read more about YOLO (in darknet) and download weight files here. In

Trieu 6.1k Dec 30, 2022
Compares various time-series feature sets on computational performance, within-set structure, and between-set relationships.

feature-set-comp Compares various time-series feature sets on computational performance, within-set structure, and between-set relationships. Reposito

Trent Henderson 7 May 25, 2022
Official code release for: EditGAN: High-Precision Semantic Image Editing

Official code release for: EditGAN: High-Precision Semantic Image Editing

565 Jan 05, 2023
A Data Annotation Tool for Semantic Segmentation, Object Detection and Lane Line Detection.(In Development Stage)

Data-Annotation-Tool How to Run this Tool? To run this software, follow the steps: git clone https://github.com/Autonomous-Car-Project/Data-Annotation

TiVRA AI 13 Aug 18, 2022
Blender Add-On for slicing meshes with planes

MeshSlicer Blender Add-On for slicing meshes with multiple overlapping planes at once. This is a simple Blender addon to slice a silmple mesh with mul

52 Dec 12, 2022
Image based Human Fall Detection

Here I integrated the YOLOv5 object detection algorithm with my own created dataset which consists of human activity images to achieve low cost, high accuracy, and real-time computing requirements

UTTEJ KUMAR 12 Dec 11, 2022
Simple data balancing baselines for worst-group-accuracy benchmarks.

BalancingGroups Code to replicate the experimental results from Simple data balancing baselines achieve competitive worst-group-accuracy. Replicating

Meta Research 29 Dec 02, 2022