GUI for TOAD-GAN, a PCG-ML algorithm for Token-based Super Mario Bros. Levels.

Overview

If you are using this code in your own project, please cite our paper:

@inproceedings{awiszus2020toadgan,
  title={TOAD-GAN: Coherent Style Level Generation from a Single Example},
  author={Awiszus, Maren and Schubert, Frederik and Rosenhahn, Bodo},
  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment},
  year={2020}
}

TOAD-GUI

TOAD-GUI is a Framework with which Super Mario Bros. levels can be randomly generated, loaded, saved, edited and played using a graphical user interface. Generation is done with pre-trained TOAD-GAN (Token-based, One-shot, Arbitrary Dimension Generative Adversarial Network). For more information on TOAD-GAN, please refer to the paper (arxiv link) and the Github.

TOAD-GUI_linux_example

This project uses the Mario-AI-Framework by Ahmed Khalifa and includes graphics from the game Super Mario Bros. It is not affiliated with or endorsed by Nintendo. The project was built for research purposes only.

AIIDE 2020

Our paper "TOAD-GAN: Coherent Style Level Generation from a Single Example" was accepted for oral presentation at AIIDE 2020! You can find our video presentation on YouTube.

Our code for TOAD-GUI and TOAD-GAN has been accepted for the AIIDE 2020 Artifact Evaluation Track! It will be recognized in the AIIDE 2020 Program.

Getting Started

This section includes the necessary steps to get TOAD-GUI running on your system.

Python

You will need Python 3 and the packages specified in requirements.txt. We recommend setting up a virtual environment with pip and installing the packages there.

$ pip3 install -r requirements.txt -f "https://download.pytorch.org/whl/torch_stable.html"

Make sure you use the pip3 that belongs to your previously defined virtual environment.

The GUI is made with Tkinter, which from Python 3.7 onwards is installed by default. If you don't have it installed because of an older version, follow the instructions here.

Java

TOAD-GUI uses the Mario-AI-Framework to play the generated levels. For the Framework to run, Java 11 (or higher) needs to be installed.

Running TOAD-GUI

Once all prerequisites are installed, TOAD-GUI can be started by running main.py.

$ python main.py

Make sure you are using the python installation you installed the prerequisites into.

TOAD-GUI

When running TOAD-GUI you can:

  • toad folder Open a Folder containing a Generator (TOAD-GAN)
  • level folder Open a (previously saved) level .txt to view and/or play
  • gear toad Generate a level of the size defined in the entries below
  • save button Save the currently loaded level level to a .txt or .png image file
  • play button Play the currently loaded level

NOTE: When a generator is opened, it will not show any files in the dialog window. That is intended behavior for askdirectory() of tkinter. Just navigate to the correct path and click "Open" regardless.

When a level is loaded, right clicking a point in the preview will allow you to change the token at that specific spot. If you resample the level, any changes made will be lost.

The labels at the bottom will display the currently loaded path and information. This program was made mostly by one researcher and is not optimized. Impatiently clicking buttons might crash the program.

Edit Mode

In this mode, parts of a generated level can be resampled with TOAD-GAN. The red bounding box shows the area to be changed, while the yellow bounding box shows which blocks can still be affected by that change. The area of effect depends on the scale which is to be resampled and is a result of the Field of View produced by the convolutional layers. Changes in a lower scale will result in larger changes in the final level.

Use the control panel to set the bounding box. The representation inside the panel shows which pixels in the noise map will be changed.

TOAD-GUI_bbox

Resample the noise map in the chosen scale. The "Noise influence" is a learned parameter that indicates how big the effect of resampling in this scale will be.

TOAD-GUI_sc3

Scale 0 is the first scale and results in the most changes. Note that the tokens outside of the bounding box change. This is because of the field of view from the convolutional layers applied to the noise map.

TOAD-GUI_sc0

You can right click a token you want to change and replace it with another token present in the level. This should be done after resampling, as resampling will regenerate the level from the noise maps which will undo these edits.

TOAD-GUI_edit

TOAD-GAN

If you are interested in training your own Generator, refer to the TOAD-GAN Github and copy the folder of your trained generator into the generators/ folder. You should now be able to open it just like the provided generators.

The necessary files are:

generators.pth
noise_amplitudes.pth
noise_maps.pth
num_layer.pth
reals.pth
token_list.pth

Any other files can be deleted if you want to keep your folders tidy.

NOTE: When a generator is opened, it will not show these files in the dialog window. That is intended behavior for askdirectory() of tkinter. Just navigate to the correct path and click "Open" regardless.

Known Bugs

  • If the level play is quit using the window ('x' button in the corner), an error message regarding py4j will occur. In spite of that, the program should continue running normally.

  • If you have two monitors with different resolutions, the GUI and the Java window might not be displayed in the correct resolution. Try moving the windows to the monitor with the other resolution if you encounter this problem. You can also change the DPI awareness for the program in the beginning of GUI.py.

Built With

  • Tkinter - Python package for building GUIs
  • py4j - Python to Java interface
  • Pillow - Python Image Library for displaying images
  • Pytorch - Deep Learning Framework
  • Maven - Used for building the Mario-AI-Framework

Authors

  • Maren Awiszus - Institut für Informationsverarbeitung, Leibniz University Hanover
  • Frederik Schubert - Institut für Informationsverarbeitung, Leibniz University Hanover

Copyright

This program is not endorsed by Nintendo and is only intended for research purposes. Mario is a Nintendo character which the authors don’t own any rights to. Nintendo is also the sole owner of all the graphical assets in the game.

Owner
Maren A.
Maren A.
Tutorials, assignments, and competitions for MIT Deep Learning related courses.

MIT Deep Learning This repository is a collection of tutorials for MIT Deep Learning courses. More added as courses progress. Tutorial: Deep Learning

Lex Fridman 9.5k Jan 07, 2023
SHRIMP: Sparser Random Feature Models via Iterative Magnitude Pruning

SHRIMP: Sparser Random Feature Models via Iterative Magnitude Pruning This repository is the official implementation of "SHRIMP: Sparser Random Featur

Bobby Shi 0 Dec 16, 2021
Research code of ICCV 2021 paper "Mesh Graphormer"

MeshGraphormer ✨ ✨ This is our research code of Mesh Graphormer. Mesh Graphormer is a new transformer-based method for human pose and mesh reconsructi

Microsoft 251 Jan 08, 2023
List some popular DeepFake models e.g. DeepFake, FaceSwap-MarekKowal, IPGAN, FaceShifter, FaceSwap-Nirkin, FSGAN, SimSwap, CihaNet, etc.

deepfake-models List some popular DeepFake models e.g. DeepFake, CihaNet, SimSwap, FaceSwap-MarekKowal, IPGAN, FaceShifter, FaceSwap-Nirkin, FSGAN, Si

Mingcan Xiang 100 Dec 17, 2022
Yolo Traffic Light Detection With Python

Yolo-Traffic-Light-Detection This project is based on detecting the Traffic light. Pretained data is used. This application entertained both real time

Ananta Raj Pant 2 Aug 08, 2022
A minimal solution to hand motion capture from a single color camera at over 100fps. Easy to use, plug to run.

Minimal Hand A minimal solution to hand motion capture from a single color camera at over 100fps. Easy to use, plug to run. This project provides the

Yuxiao Zhou 824 Jan 07, 2023
Pytorch port of Google Research's LEAF Audio paper

leaf-audio-pytorch Pytorch port of Google Research's LEAF Audio paper published at ICLR 2021. This port is not completely finished, but the Leaf() fro

Dennis Fedorishin 80 Oct 31, 2022
Multi-modal Content Creation Model Training Infrastructure including the FACT model (AI Choreographer) implementation.

AI Choreographer: Music Conditioned 3D Dance Generation with AIST++ [ICCV-2021]. Overview This package contains the model implementation and training

Google Research 365 Dec 30, 2022
Article Reranking by Memory-enhanced Key Sentence Matching for Detecting Previously Fact-checked Claims.

MTM This is the official repository of the paper: Article Reranking by Memory-enhanced Key Sentence Matching for Detecting Previously Fact-checked Cla

ICTMCG 13 Sep 17, 2022
Ontologysim: a Owlready2 library for applied production simulation

Ontologysim: a Owlready2 library for applied production simulation Ontologysim is an open-source deep production simulation framework, with an emphasi

10 Nov 30, 2022
Deep learning for Engineers - Physics Informed Deep Learning

SciANN: Neural Networks for Scientific Computations SciANN is a Keras wrapper for scientific computations and physics-informed deep learning. New to S

SciANN 195 Jan 03, 2023
[2021 MultiMedia] CONQUER: Contextual Query-aware Ranking for Video Corpus Moment Retrieval

CONQUER: Contexutal Query-aware Ranking for Video Corpus Moment Retreival PyTorch implementation of CONQUER: Contexutal Query-aware Ranking for Video

Hou zhijian 23 Dec 26, 2022
Fast, flexible and fun neural networks.

Brainstorm Discontinuation Notice Brainstorm is no longer being maintained, so we recommend using one of the many other,available frameworks, such as

IDSIA 1.3k Nov 21, 2022
PyTorch Implementation of ECCV 2020 Spotlight TuiGAN: Learning Versatile Image-to-Image Translation with Two Unpaired Images

TuiGAN-PyTorch Official PyTorch Implementation of "TuiGAN: Learning Versatile Image-to-Image Translation with Two Unpaired Images" (ECCV 2020 Spotligh

181 Dec 09, 2022
ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation

ENet in Caffe Execution times and hardware requirements Network 1024x512 1280x720 Parameters Model size (fp32) ENet 20.4 ms 32.9 ms 0.36 M 1.5 MB SegN

Timo Sämann 561 Jan 04, 2023
PassAPI is a password generator in hash format and fully developed in Python, with the aim of teaching how to handle and build

simple, elegant and safe Introduction PassAPI is a password generator in hash format and fully developed in Python, with the aim of teaching how to ha

Johnsz 2 Mar 02, 2022
The 7th edition of NTIRE: New Trends in Image Restoration and Enhancement workshop will be held on June 2022 in conjunction with CVPR 2022.

NTIRE 2022 - Image Inpainting Challenge Important dates 2022.02.01: Release of train data (input and output images) and validation data (only input) 2

Andrés Romero 37 Nov 27, 2022
Covid19-Forecasting - An interactive website that tracks, models and predicts COVID-19 Cases

Covid-Tracker This is an interactive website that tracks, models and predicts CO

Adam Lahmadi 1 Feb 01, 2022
I decide to sync up this repo and self-critical.pytorch. (The old master is in old master branch for archive)

An Image Captioning codebase This is a codebase for image captioning research. It supports: Self critical training from Self-critical Sequence Trainin

Ruotian(RT) Luo 1.3k Dec 31, 2022
PySLM Python Library for Selective Laser Melting and Additive Manufacturing

PySLM Python Library for Selective Laser Melting and Additive Manufacturing PySLM is a Python library for supporting development of input files used i

Dr Luke Parry 35 Dec 27, 2022