Code for "Solving Graph-based Public Good Games with Tree Search and Imitation Learning"

Overview

Code for "Solving Graph-based Public Good Games with Tree Search and Imitation Learning"

This is the code for the paper Solving Graph-based Public Good Games with Tree Search and Imitation Learning by Victor-Alexandru Darvariu, Stephen Hailes and Mirco Musolesi, presented at NeurIPS 2021. If you use this code, please consider citing:

@inproceedings{darvariu_solving_2021,
  title = {Solving Graph-based Public Good Games with Tree Search and Imitation Learning},
  author = {Darvariu, Victor-Alexandru and Hailes, Stephen and Musolesi, Mirco},
  booktitle = {35th Conference on Neural Information Processing Systems (NeurIPS 2021)},
  year={2021},
}

License

MIT.

Prerequisites

Currently tested on Linux and MacOS (specifically, CentOS 7.4.1708 and Mac OS Big Sur 11.2.3), can also be adapted to Windows through WSL. The host machine requires NVIDIA CUDA toolkit version 9.0 or above (tested with NVIDIA driver version 384.81).

Makes heavy use of Docker, see e.g. here for how to install. Tested with Docker 19.03. The use of Docker largely does away with dependency and setup headaches, making it significantly easier to reproduce the reported results.

Configuration

The Docker setup uses Unix groups to control permissions. You can reuse an existing group that you are a member of, or create a new group groupadd -g GID GNAME and add your user to it usermod -a -G GNAME MYUSERNAME.

Create a file relnet.env at the root of the project (see relnet_example.env) and adjust the paths within: this is where some data generated by the container will be stored. Also specify the group ID and name created / selected above.

Add the following lines to your .bashrc, replacing /home/john/git/relnet with the path where the repository is cloned.

export RN_SOURCE_DIR='/home/john/git/relnet'
set -a
. $RN_SOURCE_DIR/relnet.env
set +a

export PATH=$PATH:$RN_SOURCE_DIR/scripts

Make the scripts executable (e.g. chmod u+x scripts/*) the first time after cloning the repository, and run apply_permissions.sh in order to create and permission the necessary directories.

Managing the containers

Some scripts are provided for convenience. To build the containers (note, this will take a significant amount of time e.g. 2 hours, as some packages are built from source):

update_container.sh

To start them:

manage_container_gpu.sh up
manage_container.sh up

To stop them:

manage_container_gpu.sh stop
manage_container.sh stop

To purge the queue and restart the containers (useful for killing tasks that were launched):

purge_and_restart.sh

Adjusting the number of workers and threads

To take maximum advantage of your machine's capacity, you may want to tweak the number of threads for the GPU and CPU workers. This configuration is provided in projectconfig.py. Additionally, you may want to enforce certain memory limits for your workers to avoid OOM errors. This can be tweaked in docker-compose.yml and manage_container_gpu.sh.

It is also relatively straightforward to add more workers from different machines you control. For this, you will need to mount the volumes on networked-attached storage (i.e., make sure paths provided in relnet.env are network-accessible) and adjust the location of backend and queue in projectconfig.py to a network location instead of localhost. On the other machines, only start the worker container (see e.g. manage_container.sh).

Setting up graph data

Synthetic data will be automatically generated when the experiments are ran and stored to $RN_EXPERIMENT_DIR/stored_graphs.

Accessing the services

There are several services running on the manager node.

  • Jupyter notebook server: http://localhost:8888
  • Flower for queue statistics: http://localhost:5555
  • Tensorboard (currently disabled due to its large memory footprint): http://localhost:6006
  • RabbitMQ management: http://localhost:15672

The first time Jupyter is accessed it will prompt for a token to enable password configuration, it can be grabbed by running docker exec -it relnet-manager /bin/bash -c "jupyter notebook list".

Accessing experiment data and results database

Experiment data and results are stored in part as files (under your configured $RN_EXPERIMENT_DATA_DIR) as well as in a MongoDB database. To access the MongoDB database with a GUI, you can use a MongoDB client such as Robo3T and point it to http://localhost:27017.

Some functionality is provided in relnet/evaluation/storage.py to insert and retrieve data, you can use it in e.g. analysis notebooks.

Running experiments

Experiments are launched from the manager container and processed (in a parallel way) by the workers. The file relnet/evaluation/experiment_conditions.py contains the configuration for the experiments reported in the paper, but you may modify e.g. agents, objective functions, hyperparameters etc. to suit your needs.

Then, you can launch all the experiments as follows:

Part 1: Hyperparameter optimization & evaluation for all aproaches except GIL

run_part1.sh

Part 2: Data collection for GIL using the UCT algorithm

run_part2.sh

Part 3: Training & hyperparameter optimization for GIL

run_part3.sh

Monitoring experiments

  • You can navigate to http://localhost:5555 for the Flower interface which shows the progress of processing tasks in the queue. You may also check logs for both manager and worker at $RN_EXPERIMENT_DATA_DIR/logs.

Reproducing the results

Jupyter notebooks are used to perform the data analysis and produce tables and figures. Navigate to http://localhost:8888, then notebooks folder.

All tables and result figures can be obtained by opening the GGNN_Evaluation.ipynb notebook, selecting the py3-relnet kernel and run all cells. Resulting .pdf figures and .tex tables can be found at $RN_EXPERIMENT_DIR/aggregate. There are additional notebooks provided for analyzing the results of hyperparameter optimization:

  • GGNN_Hyperparam_Optimisation.ipynb for UCT
  • GGNN_Hyperparam_Optimisation_IL.ipynb for GIL

Problems with jupyter kernel

In case the py3-relnet kernel is not found, try reinstalling the kernel by running docker exec -it -u 0 relnet-manager /bin/bash -c "source activate relnet-cenv; python -m ipykernel install --user --name relnet --display-name py3-relnet"

Owner
Victor-Alexandru Darvariu
Doctoral Student at University College London and The Alan Turing Institute.
Victor-Alexandru Darvariu
CN24 is a complete semantic segmentation framework using fully convolutional networks

Build status: master (production branch): develop (development branch): Welcome to the CN24 GitHub repository! CN24 is a complete semantic segmentatio

Computer Vision Group Jena 123 Jul 14, 2022
PyTorch implementation of MoCo: Momentum Contrast for Unsupervised Visual Representation Learning

MoCo: Momentum Contrast for Unsupervised Visual Representation Learning This is a PyTorch implementation of the MoCo paper: @Article{he2019moco, aut

Meta Research 3.7k Jan 02, 2023
Labels4Free: Unsupervised Segmentation using StyleGAN

Labels4Free: Unsupervised Segmentation using StyleGAN ICCV 2021 Figure: Some segmentation masks predicted by Labels4Free Framework on real and synthet

70 Dec 23, 2022
Official Pytorch implementation for Deep Contextual Video Compression, NeurIPS 2021

Introduction Official Pytorch implementation for Deep Contextual Video Compression, NeurIPS 2021 Prerequisites Python 3.8 and conda, get Conda CUDA 11

51 Dec 03, 2022
This is the repository for paper NEEDLE: Towards Non-invertible Backdoor Attack to Deep Learning Models.

This is the repository for paper NEEDLE: Towards Non-invertible Backdoor Attack to Deep Learning Models.

1 Oct 25, 2021
This is the codebase for Diffusion Models Beat GANS on Image Synthesis.

This is the codebase for Diffusion Models Beat GANS on Image Synthesis.

OpenAI 3k Dec 26, 2022
Implementation of character based convolutional neural network

Character Based CNN This repo contains a PyTorch implementation of a character-level convolutional neural network for text classification. The model a

Ahmed BESBES 248 Nov 21, 2022
Mining-the-Social-Web-3rd-Edition - The official online compendium for Mining the Social Web, 3rd Edition (O'Reilly, 2018)

Mining the Social Web, 3rd Edition The official code repository for Mining the Social Web, 3rd Edition (O'Reilly, 2019). The book is available from Am

Mikhail Klassen 838 Jan 01, 2023
Highly comparative time-series analysis

〰️ hctsa 〰️ : highly comparative time-series analysis hctsa is a software package for running highly comparative time-series analysis using Matlab (fu

Ben Fulcher 569 Dec 21, 2022
Implementation of MA-Trace - a general-purpose multi-agent RL algorithm for cooperative environments.

Off-Policy Correction For Multi-Agent Reinforcement Learning This repository is the official implementation of Off-Policy Correction For Multi-Agent R

4 Aug 18, 2022
The final project for "Applying AI to Wearable Device Data" course from "AI for Healthcare" - Udacity.

Motion Compensated Pulse Rate Estimation Overview This project has 2 main parts. Develop a Pulse Rate Algorithm on the given training data. Then Test

Omar Laham 2 Oct 25, 2022
Grammar Induction using a Template Tree Approach

Gitta Gitta ("Grammar Induction using a Template Tree Approach") is a method for inducing context-free grammars. It performs particularly well on data

Thomas Winters 36 Nov 15, 2022
YOLOv5 Series Multi-backbone, Pruning and quantization Compression Tool Box.

YOLOv5-Compression Update News Requirements 环境安装 pip install -r requirements.txt Evaluation metric Visdrone Model mAP ZhangYuan 719 Jan 02, 2023

Pytorch port of Google Research's LEAF Audio paper

leaf-audio-pytorch Pytorch port of Google Research's LEAF Audio paper published at ICLR 2021. This port is not completely finished, but the Leaf() fro

Dennis Fedorishin 80 Oct 31, 2022
AI-Bot - 一个基于watermelon改造的OpenAI-GPT-2的智能机器人

AI-Bot 一个基于watermelon改造的OpenAI-GPT-2的智能机器人 在Binder上直接运行测试 目前有两种实现方式 TF2的GPT-2 TF

9 Nov 16, 2022
A library for graph deep learning research

Documentation | Paper [JMLR] | Tutorials | Benchmarks | Examples DIG: Dive into Graphs is a turnkey library for graph deep learning research. Why DIG?

DIVE Lab, Texas A&M University 1.3k Jan 01, 2023
Voxel Transformer for 3D object detection

Voxel Transformer This is a reproduced repo of Voxel Transformer for 3D object detection. The code is mainly based on OpenPCDet. Introduction We provi

173 Dec 25, 2022
Caffe models in TensorFlow

Caffe to TensorFlow Convert Caffe models to TensorFlow. Usage Run convert.py to convert an existing Caffe model to TensorFlow. Make sure you're using

Saumitro Dasgupta 2.8k Dec 31, 2022
Additional functionality for use with fastai’s medical imaging module

fmi Adding additional functionality to fastai's medical imaging module To learn more about medical imaging using Fastai you can view my blog Install g

14 Oct 31, 2022
A powerful framework for decentralized federated learning with user-defined communication topology

Scatterbrained Decentralized Federated Learning Scatterbrained makes it easy to build federated learning systems. In addition to traditional federated

Johns Hopkins Applied Physics Laboratory 7 Sep 26, 2022