The CARL Benchmark Library
CARL (context adaptive RL) provides highly configurable contextual extensions to several well-known RL environments. It's designed to test your agent's generalization capabilities in all scenarios where intra-task generalization is important.
Benchmarks include:
-
OpenAI gym classic control suite extended with several physics context features like gravity or friction
-
OpenAI gym Box2D BipedalWalker, LunarLander and CarRacing, each with their own modification possibilities like new vehicles to race
-
All Brax locomotion environments with exposed internal features like joint strength or torso mass
-
Super Mario (TOAD-GAN), a procedurally generated jump'n'run game with control over level similarity
-
RNADesign, an environment for RNA design given structure constraints with structures from different datasets to choose from
Installation
We recommend you use a virtual environment (e.g. Anaconda) to install CARL and its dependencies. We recommend and test with python 3.9 under Linux.
First, clone our repository and install the basic requirements:
git clone https://github.com/automl/CARL.git --recursive
cd CARL
pip install .
This will only install the basic classic control environments, which should run on most operating systems. For the full set of environments, use the install options:
pip install -e .[box2d, brax, rna, mario]
These may not be compatible with Windows systems. Box2D environment may need to be installed via conda on MacOS systems:
conda install -c conda-forge gym-box2d
In general, we test on Linux systems, but aim to keep the benchmark compatible with MacOS as much as possible. Mario at this point, however, will not run on any operation system besides Linux
To install the additional requirements for ToadGAN:
javac src/envs/mario/Mario-AI-Framework/**/*.java
If you want to use the RNA design environment:
cd src/envs/rna/learna
make requirements
make data
In case you want to run our experiments or use our training files, also install the experiment dependencies:
pip install -e .[experiments]
Train an Agent
To get started with CARL, you can use our 'train.py' script. It will train a PPO agent on the environment of your choice with custom context variations that are sampled from a standard deviation.
To use MetaCartPole with variations in gravity and friction by 20% compared to the default, run:
python train.py
--env CARLCartPoleEnv
--context_args gravity friction
--default_sample_std_percentage 0.2
--outdir <result_location>
You can use the plotting scripts in src/eval to view the results.
CARL's Contextual Extension
CARL contextually extends the environment by making the context visible and configurable. During training we therefore can encounter different contexts and train for generalization. We exemplarily show how Brax' Fetch is extended and embedded by CARL. Different instiations can be achieved by setting the context features to different values.
Cite Us
@misc{CARL,
author = {C. Benjamins and
T. Eimer and
F. Schubert and
A. Biedenkapp and
B. Rosenhahn and
F. Hutter and
M. Lindauer},
title = {CARL: A Benchmark for Contextual and Adaptive Reinforcement Learning},
howpublished = {https://github.com/automl/CARL},
year = {2021},
month = aug,
}
References
OpenAI gym, Brockman et al., 2016. arXiv preprint arXiv:1606.01540
TOAD-GAN: Coherent Style Level Generation from a Single Example, Awiszus et al., AIIDE 2020
Learning to Design RNA, Runge et al., ICRL 2019
License
CARL falls under the Apache License 2.0 (see file 'LICENSE') as is permitted by all work that we use. This includes CARLMario, which is not based on the Nintendo Game, but on TOAD-GAN and TOAD-GUI running under an MIT license. They in turn make use of the Mario AI framework (https://github.com/amidos2006/Mario-AI-Framework). This is not the original game but a replica, explicitly built for research purposes and includes a copyright notice (https://github.com/amidos2006/Mario-AI-Framework#copyrights ).
34 Nov 09, 2022
686 Nov 24, 2022
554 Dec 30, 2022
2.2k Jan 01, 2023
311 Dec 30, 2022
0 Oct 28, 2022
101 Dec 30, 2022
58 Dec 23, 2022
49 Dec 18, 2022
1 Feb 09, 2022
689 Dec 28, 2022
271 Jan 01, 2023
33 Dec 27, 2022
34 Jan 05, 2023
4 May 31, 2022
41 Dec 24, 2022
24 Nov 26, 2022
211 Dec 25, 2022
9 Nov 29, 2022
615 Dec 25, 2022