The CARL Benchmark Library
CARL (context adaptive RL) provides highly configurable contextual extensions to several well-known RL environments. It's designed to test your agent's generalization capabilities in all scenarios where intra-task generalization is important.
Benchmarks include:
-
OpenAI gym classic control suite extended with several physics context features like gravity or friction
-
OpenAI gym Box2D BipedalWalker, LunarLander and CarRacing, each with their own modification possibilities like new vehicles to race
-
All Brax locomotion environments with exposed internal features like joint strength or torso mass
-
Super Mario (TOAD-GAN), a procedurally generated jump'n'run game with control over level similarity
-
RNADesign, an environment for RNA design given structure constraints with structures from different datasets to choose from
Installation
We recommend you use a virtual environment (e.g. Anaconda) to install CARL and its dependencies. We recommend and test with python 3.9 under Linux.
First, clone our repository and install the basic requirements:
git clone https://github.com/automl/CARL.git --recursive
cd CARL
pip install .
This will only install the basic classic control environments, which should run on most operating systems. For the full set of environments, use the install options:
pip install -e .[box2d, brax, rna, mario]
These may not be compatible with Windows systems. Box2D environment may need to be installed via conda on MacOS systems:
conda install -c conda-forge gym-box2d
In general, we test on Linux systems, but aim to keep the benchmark compatible with MacOS as much as possible. Mario at this point, however, will not run on any operation system besides Linux
To install the additional requirements for ToadGAN:
javac src/envs/mario/Mario-AI-Framework/**/*.java
If you want to use the RNA design environment:
cd src/envs/rna/learna
make requirements
make data
In case you want to run our experiments or use our training files, also install the experiment dependencies:
pip install -e .[experiments]
Train an Agent
To get started with CARL, you can use our 'train.py' script. It will train a PPO agent on the environment of your choice with custom context variations that are sampled from a standard deviation.
To use MetaCartPole with variations in gravity and friction by 20% compared to the default, run:
python train.py
--env CARLCartPoleEnv
--context_args gravity friction
--default_sample_std_percentage 0.2
--outdir <result_location>
You can use the plotting scripts in src/eval to view the results.
CARL's Contextual Extension
CARL contextually extends the environment by making the context visible and configurable. During training we therefore can encounter different contexts and train for generalization. We exemplarily show how Brax' Fetch is extended and embedded by CARL. Different instiations can be achieved by setting the context features to different values.
Cite Us
@misc{CARL,
author = {C. Benjamins and
T. Eimer and
F. Schubert and
A. Biedenkapp and
B. Rosenhahn and
F. Hutter and
M. Lindauer},
title = {CARL: A Benchmark for Contextual and Adaptive Reinforcement Learning},
howpublished = {https://github.com/automl/CARL},
year = {2021},
month = aug,
}
References
OpenAI gym, Brockman et al., 2016. arXiv preprint arXiv:1606.01540
TOAD-GAN: Coherent Style Level Generation from a Single Example, Awiszus et al., AIIDE 2020
Learning to Design RNA, Runge et al., ICRL 2019
License
CARL falls under the Apache License 2.0 (see file 'LICENSE') as is permitted by all work that we use. This includes CARLMario, which is not based on the Nintendo Game, but on TOAD-GAN and TOAD-GUI running under an MIT license. They in turn make use of the Mario AI framework (https://github.com/amidos2006/Mario-AI-Framework). This is not the original game but a replica, explicitly built for research purposes and includes a copyright notice (https://github.com/amidos2006/Mario-AI-Framework#copyrights ).
37 Oct 21, 2022
1 Jun 09, 2022
110 Nov 21, 2022
14 Jul 13, 2022
2k Dec 26, 2022
0 Jan 23, 2022
24 Dec 12, 2022
1 Feb 26, 2022
1 Oct 28, 2022
16 Jul 25, 2022
221 Jan 06, 2023
3 Jul 14, 2021
31 Oct 14, 2022
9 Sep 06, 2022
2.3k Jan 04, 2023
62 Dec 14, 2022
130 Dec 13, 2022
694 Nov 23, 2022
4 Aug 31, 2022
61 Dec 27, 2022