GUI for TOAD-GAN, a PCG-ML algorithm for Token-based Super Mario Bros. Levels.

Overview

If you are using this code in your own project, please cite our paper:

@inproceedings{awiszus2020toadgan,
  title={TOAD-GAN: Coherent Style Level Generation from a Single Example},
  author={Awiszus, Maren and Schubert, Frederik and Rosenhahn, Bodo},
  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment},
  year={2020}
}

TOAD-GUI

TOAD-GUI is a Framework with which Super Mario Bros. levels can be randomly generated, loaded, saved, edited and played using a graphical user interface. Generation is done with pre-trained TOAD-GAN (Token-based, One-shot, Arbitrary Dimension Generative Adversarial Network). For more information on TOAD-GAN, please refer to the paper (arxiv link) and the Github.

TOAD-GUI_linux_example

This project uses the Mario-AI-Framework by Ahmed Khalifa and includes graphics from the game Super Mario Bros. It is not affiliated with or endorsed by Nintendo. The project was built for research purposes only.

AIIDE 2020

Our paper "TOAD-GAN: Coherent Style Level Generation from a Single Example" was accepted for oral presentation at AIIDE 2020! You can find our video presentation on YouTube.

Our code for TOAD-GUI and TOAD-GAN has been accepted for the AIIDE 2020 Artifact Evaluation Track! It will be recognized in the AIIDE 2020 Program.

Getting Started

This section includes the necessary steps to get TOAD-GUI running on your system.

Python

You will need Python 3 and the packages specified in requirements.txt. We recommend setting up a virtual environment with pip and installing the packages there.

$ pip3 install -r requirements.txt -f "https://download.pytorch.org/whl/torch_stable.html"

Make sure you use the pip3 that belongs to your previously defined virtual environment.

The GUI is made with Tkinter, which from Python 3.7 onwards is installed by default. If you don't have it installed because of an older version, follow the instructions here.

Java

TOAD-GUI uses the Mario-AI-Framework to play the generated levels. For the Framework to run, Java 11 (or higher) needs to be installed.

Running TOAD-GUI

Once all prerequisites are installed, TOAD-GUI can be started by running main.py.

$ python main.py

Make sure you are using the python installation you installed the prerequisites into.

TOAD-GUI

When running TOAD-GUI you can:

  • toad folder Open a Folder containing a Generator (TOAD-GAN)
  • level folder Open a (previously saved) level .txt to view and/or play
  • gear toad Generate a level of the size defined in the entries below
  • save button Save the currently loaded level level to a .txt or .png image file
  • play button Play the currently loaded level

NOTE: When a generator is opened, it will not show any files in the dialog window. That is intended behavior for askdirectory() of tkinter. Just navigate to the correct path and click "Open" regardless.

When a level is loaded, right clicking a point in the preview will allow you to change the token at that specific spot. If you resample the level, any changes made will be lost.

The labels at the bottom will display the currently loaded path and information. This program was made mostly by one researcher and is not optimized. Impatiently clicking buttons might crash the program.

Edit Mode

In this mode, parts of a generated level can be resampled with TOAD-GAN. The red bounding box shows the area to be changed, while the yellow bounding box shows which blocks can still be affected by that change. The area of effect depends on the scale which is to be resampled and is a result of the Field of View produced by the convolutional layers. Changes in a lower scale will result in larger changes in the final level.

Use the control panel to set the bounding box. The representation inside the panel shows which pixels in the noise map will be changed.

TOAD-GUI_bbox

Resample the noise map in the chosen scale. The "Noise influence" is a learned parameter that indicates how big the effect of resampling in this scale will be.

TOAD-GUI_sc3

Scale 0 is the first scale and results in the most changes. Note that the tokens outside of the bounding box change. This is because of the field of view from the convolutional layers applied to the noise map.

TOAD-GUI_sc0

You can right click a token you want to change and replace it with another token present in the level. This should be done after resampling, as resampling will regenerate the level from the noise maps which will undo these edits.

TOAD-GUI_edit

TOAD-GAN

If you are interested in training your own Generator, refer to the TOAD-GAN Github and copy the folder of your trained generator into the generators/ folder. You should now be able to open it just like the provided generators.

The necessary files are:

generators.pth
noise_amplitudes.pth
noise_maps.pth
num_layer.pth
reals.pth
token_list.pth

Any other files can be deleted if you want to keep your folders tidy.

NOTE: When a generator is opened, it will not show these files in the dialog window. That is intended behavior for askdirectory() of tkinter. Just navigate to the correct path and click "Open" regardless.

Known Bugs

  • If the level play is quit using the window ('x' button in the corner), an error message regarding py4j will occur. In spite of that, the program should continue running normally.

  • If you have two monitors with different resolutions, the GUI and the Java window might not be displayed in the correct resolution. Try moving the windows to the monitor with the other resolution if you encounter this problem. You can also change the DPI awareness for the program in the beginning of GUI.py.

Built With

  • Tkinter - Python package for building GUIs
  • py4j - Python to Java interface
  • Pillow - Python Image Library for displaying images
  • Pytorch - Deep Learning Framework
  • Maven - Used for building the Mario-AI-Framework

Authors

  • Maren Awiszus - Institut für Informationsverarbeitung, Leibniz University Hanover
  • Frederik Schubert - Institut für Informationsverarbeitung, Leibniz University Hanover

Copyright

This program is not endorsed by Nintendo and is only intended for research purposes. Mario is a Nintendo character which the authors don’t own any rights to. Nintendo is also the sole owner of all the graphical assets in the game.

Owner
Maren A.
Maren A.
Local trajectory planner based on a multilayer graph framework for autonomous race vehicles.

Graph-Based Local Trajectory Planner The graph-based local trajectory planner is python-based and comes with open interfaces as well as debug, visuali

TUM - Institute of Automotive Technology 160 Jan 04, 2023
TransPrompt - Towards an Automatic Transferable Prompting Framework for Few-shot Text Classification

TransPrompt This code is implement for our EMNLP 2021's paper 《TransPrompt:Towards an Automatic Transferable Prompting Framework for Few-shot Text Cla

WangJianing 23 Dec 21, 2022
PyTorch implementation of the Quasi-Recurrent Neural Network - up to 16 times faster than NVIDIA's cuDNN LSTM

Quasi-Recurrent Neural Network (QRNN) for PyTorch Updated to support multi-GPU environments via DataParallel - see the the multigpu_dataparallel.py ex

Salesforce 1.3k Dec 28, 2022
Hand Gesture Volume Control is AIML based project which uses image processing to control the volume of your Computer.

Hand Gesture Volume Control Modules There are basically three modules Handtracking Program Handtracking Module Volume Control Program Handtracking Pro

VITTAL 1 Jan 12, 2022
Implementation of the paper "Shapley Explanation Networks"

Shapley Explanation Networks Implementation of the paper "Shapley Explanation Networks" at ICLR 2021. Note that this repo heavily uses the experimenta

68 Dec 27, 2022
Measuring Coding Challenge Competence With APPS

Measuring Coding Challenge Competence With APPS This is the repository for Measuring Coding Challenge Competence With APPS by Dan Hendrycks*, Steven B

Dan Hendrycks 218 Dec 27, 2022
Data Consistency for Magnetic Resonance Imaging

Data Consistency for Magnetic Resonance Imaging Data Consistency (DC) is crucial for generalization in multi-modal MRI data and robustness in detectin

Dimitris Karkalousos 19 Dec 12, 2022
IPATool-py: download ipa easily

IPATool-py Python version of IPATool! Installation pip3 install -r requirements.txt Usage Quickstart: download app with specific bundleId into DIR: p

159 Dec 30, 2022
Manipulation OpenAI Gym environments to simulate robots at the STARS lab

Manipulator Learning This repository contains a set of manipulation environments that are compatible with OpenAI Gym and simulated in pybullet. In par

STARS Laboratory 5 Dec 08, 2022
Code for BMVC2021 "MOS: A Low Latency and Lightweight Framework for Face Detection, Landmark Localization, and Head Pose Estimation"

MOS-Multi-Task-Face-Detect Introduction This repo is the official implementation of "MOS: A Low Latency and Lightweight Framework for Face Detection,

104 Dec 08, 2022
Event queue (Equeue) dialect is an MLIR Dialect that models concurrent devices in terms of control and structure.

Event Queue Dialect Event queue (Equeue) dialect is an MLIR Dialect that models concurrent devices in terms of control and structure. Motivation The m

Cornell Capra 23 Dec 08, 2022
An efficient and easy-to-use deep learning model compression framework

TinyNeuralNetwork 简体中文 TinyNeuralNetwork is an efficient and easy-to-use deep learning model compression framework, which contains features like neura

Alibaba 441 Dec 25, 2022
Multiview Neural Surface Reconstruction by Disentangling Geometry and Appearance

Multiview Neural Surface Reconstruction by Disentangling Geometry and Appearance Project Page | Paper | Data This repository contains an implementatio

Lior Yariv 521 Dec 30, 2022
Pytorch implementation of “Recursive Non-Autoregressive Graph-to-Graph Transformer for Dependency Parsing with Iterative Refinement”

Graph-to-Graph Transformers Self-attention models, such as Transformer, have been hugely successful in a wide range of natural language processing (NL

Idiap Research Institute 40 Aug 14, 2022
A program to recognize fruits on pictures or videos using yolov5

Yolov5 Fruits Detector Requirements Either Linux or Windows. We recommend Linux for better performance. Python 3.6+ and PyTorch 1.7+. Installation To

Fateme Zamanian 30 Jan 06, 2023
Best Practices on Recommendation Systems

Recommenders What's New (February 4, 2021) We have a new relase Recommenders 2021.2! It comes with lots of bug fixes, optimizations and 3 new algorith

Microsoft 14.8k Jan 03, 2023
Complex-Valued Neural Networks (CVNN)Complex-Valued Neural Networks (CVNN)

Complex-Valued Neural Networks (CVNN) Done by @NEGU93 - J. Agustin Barrachina Using this library, the only difference with a Tensorflow code is that y

youceF 1 Nov 12, 2021
PixelPyramids: Exact Inference Models from Lossless Image Pyramids (ICCV 2021)

PixelPyramids: Exact Inference Models from Lossless Image Pyramids This repository contains the PyTorch implementation of the paper PixelPyramids: Exa

Visual Inference Lab @TU Darmstadt 8 Dec 11, 2022
Automatic Video Captioning Evaluation Metric --- EMScore

Automatic Video Captioning Evaluation Metric --- EMScore Overview For an illustration, EMScore can be computed as: Installation modify the encode_text

Yaya Shi 17 Nov 28, 2022