GUI for TOAD-GAN, a PCG-ML algorithm for Token-based Super Mario Bros. Levels.

Overview

If you are using this code in your own project, please cite our paper:

@inproceedings{awiszus2020toadgan,
  title={TOAD-GAN: Coherent Style Level Generation from a Single Example},
  author={Awiszus, Maren and Schubert, Frederik and Rosenhahn, Bodo},
  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment},
  year={2020}
}

TOAD-GUI

TOAD-GUI is a Framework with which Super Mario Bros. levels can be randomly generated, loaded, saved, edited and played using a graphical user interface. Generation is done with pre-trained TOAD-GAN (Token-based, One-shot, Arbitrary Dimension Generative Adversarial Network). For more information on TOAD-GAN, please refer to the paper (arxiv link) and the Github.

TOAD-GUI_linux_example

This project uses the Mario-AI-Framework by Ahmed Khalifa and includes graphics from the game Super Mario Bros. It is not affiliated with or endorsed by Nintendo. The project was built for research purposes only.

AIIDE 2020

Our paper "TOAD-GAN: Coherent Style Level Generation from a Single Example" was accepted for oral presentation at AIIDE 2020! You can find our video presentation on YouTube.

Our code for TOAD-GUI and TOAD-GAN has been accepted for the AIIDE 2020 Artifact Evaluation Track! It will be recognized in the AIIDE 2020 Program.

Getting Started

This section includes the necessary steps to get TOAD-GUI running on your system.

Python

You will need Python 3 and the packages specified in requirements.txt. We recommend setting up a virtual environment with pip and installing the packages there.

$ pip3 install -r requirements.txt -f "https://download.pytorch.org/whl/torch_stable.html"

Make sure you use the pip3 that belongs to your previously defined virtual environment.

The GUI is made with Tkinter, which from Python 3.7 onwards is installed by default. If you don't have it installed because of an older version, follow the instructions here.

Java

TOAD-GUI uses the Mario-AI-Framework to play the generated levels. For the Framework to run, Java 11 (or higher) needs to be installed.

Running TOAD-GUI

Once all prerequisites are installed, TOAD-GUI can be started by running main.py.

$ python main.py

Make sure you are using the python installation you installed the prerequisites into.

TOAD-GUI

When running TOAD-GUI you can:

  • toad folder Open a Folder containing a Generator (TOAD-GAN)
  • level folder Open a (previously saved) level .txt to view and/or play
  • gear toad Generate a level of the size defined in the entries below
  • save button Save the currently loaded level level to a .txt or .png image file
  • play button Play the currently loaded level

NOTE: When a generator is opened, it will not show any files in the dialog window. That is intended behavior for askdirectory() of tkinter. Just navigate to the correct path and click "Open" regardless.

When a level is loaded, right clicking a point in the preview will allow you to change the token at that specific spot. If you resample the level, any changes made will be lost.

The labels at the bottom will display the currently loaded path and information. This program was made mostly by one researcher and is not optimized. Impatiently clicking buttons might crash the program.

Edit Mode

In this mode, parts of a generated level can be resampled with TOAD-GAN. The red bounding box shows the area to be changed, while the yellow bounding box shows which blocks can still be affected by that change. The area of effect depends on the scale which is to be resampled and is a result of the Field of View produced by the convolutional layers. Changes in a lower scale will result in larger changes in the final level.

Use the control panel to set the bounding box. The representation inside the panel shows which pixels in the noise map will be changed.

TOAD-GUI_bbox

Resample the noise map in the chosen scale. The "Noise influence" is a learned parameter that indicates how big the effect of resampling in this scale will be.

TOAD-GUI_sc3

Scale 0 is the first scale and results in the most changes. Note that the tokens outside of the bounding box change. This is because of the field of view from the convolutional layers applied to the noise map.

TOAD-GUI_sc0

You can right click a token you want to change and replace it with another token present in the level. This should be done after resampling, as resampling will regenerate the level from the noise maps which will undo these edits.

TOAD-GUI_edit

TOAD-GAN

If you are interested in training your own Generator, refer to the TOAD-GAN Github and copy the folder of your trained generator into the generators/ folder. You should now be able to open it just like the provided generators.

The necessary files are:

generators.pth
noise_amplitudes.pth
noise_maps.pth
num_layer.pth
reals.pth
token_list.pth

Any other files can be deleted if you want to keep your folders tidy.

NOTE: When a generator is opened, it will not show these files in the dialog window. That is intended behavior for askdirectory() of tkinter. Just navigate to the correct path and click "Open" regardless.

Known Bugs

  • If the level play is quit using the window ('x' button in the corner), an error message regarding py4j will occur. In spite of that, the program should continue running normally.

  • If you have two monitors with different resolutions, the GUI and the Java window might not be displayed in the correct resolution. Try moving the windows to the monitor with the other resolution if you encounter this problem. You can also change the DPI awareness for the program in the beginning of GUI.py.

Built With

  • Tkinter - Python package for building GUIs
  • py4j - Python to Java interface
  • Pillow - Python Image Library for displaying images
  • Pytorch - Deep Learning Framework
  • Maven - Used for building the Mario-AI-Framework

Authors

  • Maren Awiszus - Institut für Informationsverarbeitung, Leibniz University Hanover
  • Frederik Schubert - Institut für Informationsverarbeitung, Leibniz University Hanover

Copyright

This program is not endorsed by Nintendo and is only intended for research purposes. Mario is a Nintendo character which the authors don’t own any rights to. Nintendo is also the sole owner of all the graphical assets in the game.

Owner
Maren A.
Maren A.
Project to create an open-source 6 DoF input device

6DInputs A Project to create open-source 3D printed 6 DoF input devices Note the plural ('6DInputs' and 'devices') in the headings. We would like seve

RepRap Ltd 47 Jul 28, 2022
A Bayesian cognition approach for belief updating of correlation judgement through uncertainty visualizations

Overview Code and supplemental materials for Karduni et al., 2020 IEEE Vis. "A Bayesian cognition approach for belief updating of correlation judgemen

Ryan Wesslen 1 Feb 08, 2022
"Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedback"

This is code repo for our EMNLP 2017 paper "Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedback", which implements the A2C algorithm on top of a neural encoder-

Khanh Nguyen 131 Oct 21, 2022
This code is for our paper "VTGAN: Semi-supervised Retinal Image Synthesis and Disease Prediction using Vision Transformers"

ICCV Workshop 2021 VTGAN This code is for our paper "VTGAN: Semi-supervised Retinal Image Synthesis and Disease Prediction using Vision Transformers"

Sharif Amit Kamran 25 Dec 08, 2022
Learning Dense Representations of Phrases at Scale (Lee et al., 2020)

DensePhrases DensePhrases provides answers to your natural language questions from the entire Wikipedia in real-time. While it efficiently searches th

Princeton Natural Language Processing 540 Dec 30, 2022
PyTorch implementation of the paper: Long-tail Learning via Logit Adjustment

logit-adj-pytorch PyTorch implementation of the paper: Long-tail Learning via Logit Adjustment This code implements the paper: Long-tail Learning via

Chamuditha Jayanga 53 Dec 23, 2022
This is a library for training and applying sparse fine-tunings with torch and transformers.

This is a library for training and applying sparse fine-tunings with torch and transformers. Please refer to our paper Composable Sparse Fine-Tuning f

Cambridge Language Technology Lab 37 Dec 30, 2022
Dynamic View Synthesis from Dynamic Monocular Video

Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer This repository contains code to compute depth from a

Intelligent Systems Lab Org 2.3k Jan 01, 2023
Keras Image Embeddings using Contrastive Loss

Keras-Image-Embeddings-using-Contrastive-Loss Image to Embedding projection in vector space. Implementation in keras and tensorflow for custom data. B

Shravan Anand K 5 Mar 21, 2022
Deep Learning and Reinforcement Learning Library for Scientists and Engineers 🔥

TensorLayer is a novel TensorFlow-based deep learning and reinforcement learning library designed for researchers and engineers. It provides an extens

TensorLayer Community 7.1k Dec 29, 2022
Network Enhancement implementation in pytorch

network_enahncement_pytorch Network Enhancement implementation in pytorch Research paper Network Enhancement: a general method to denoise weighted bio

Yen 1 Nov 12, 2021
Realtime YOLO Monster Detection With Non Maximum Supression

Realtime-YOLO-Monster-Detection-With-Non-Maximum-Supression Table of Contents In

5 Oct 07, 2022
Pytorch implementation of Implicit Behavior Cloning.

Implicit Behavior Cloning - PyTorch (wip) Pytorch implementation of Implicit Behavior Cloning. Install conda create -n ibc python=3.8 pip install -r r

Kevin Zakka 49 Dec 25, 2022
Regularizing Nighttime Weirdness: Efficient Self-supervised Monocular Depth Estimation in the Dark (ICCV 2021)

Regularizing Nighttime Weirdness: Efficient Self-supervised Monocular Depth Estimation in the Dark (ICCV 2021) Kun Wang, Zhenyu Zhang, Zhiqiang Yan, X

kunwang 66 Nov 24, 2022
ScaleNet: A Shallow Architecture for Scale Estimation

ScaleNet: A Shallow Architecture for Scale Estimation Repository for the code of ScaleNet paper: "ScaleNet: A Shallow Architecture for Scale Estimatio

Axel Barroso 34 Nov 09, 2022
GradAttack is a Python library for easy evaluation of privacy risks in public gradients in Federated Learning

GradAttack is a Python library for easy evaluation of privacy risks in public gradients in Federated Learning, as well as corresponding mitigation strategies.

129 Dec 30, 2022
TorchCV: A PyTorch-Based Framework for Deep Learning in Computer Vision

TorchCV: A PyTorch-Based Framework for Deep Learning in Computer Vision @misc{you2019torchcv, author = {Ansheng You and Xiangtai Li and Zhen Zhu a

Donny You 2.2k Jan 06, 2023
Additional environments compatible with OpenAI gym

Decentralized Control of Quadrotor Swarms with End-to-end Deep Reinforcement Learning A codebase for training reinforcement learning policies for quad

Zhehui Huang 40 Dec 06, 2022
Easy-to-use micro-wrappers for Gym and PettingZoo based RL Environments

SuperSuit introduces a collection of small functions which can wrap reinforcement learning environments to do preprocessing ('microwrappers'). We supp

Farama Foundation 357 Jan 06, 2023
This repository is related to an Arabic tutorial, within the tutorial we discuss the common data structure and algorithms and their worst and best case for each, then implement the code using Python.

Data Structure and Algorithms with Python This repository is related to the Arabic tutorial here, within the tutorial we discuss the common data struc

Mohamed Ayman 33 Dec 02, 2022