GUI for TOAD-GAN, a PCG-ML algorithm for Token-based Super Mario Bros. Levels.

Overview

If you are using this code in your own project, please cite our paper:

@inproceedings{awiszus2020toadgan,
  title={TOAD-GAN: Coherent Style Level Generation from a Single Example},
  author={Awiszus, Maren and Schubert, Frederik and Rosenhahn, Bodo},
  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment},
  year={2020}
}

TOAD-GUI

TOAD-GUI is a Framework with which Super Mario Bros. levels can be randomly generated, loaded, saved, edited and played using a graphical user interface. Generation is done with pre-trained TOAD-GAN (Token-based, One-shot, Arbitrary Dimension Generative Adversarial Network). For more information on TOAD-GAN, please refer to the paper (arxiv link) and the Github.

TOAD-GUI_linux_example

This project uses the Mario-AI-Framework by Ahmed Khalifa and includes graphics from the game Super Mario Bros. It is not affiliated with or endorsed by Nintendo. The project was built for research purposes only.

AIIDE 2020

Our paper "TOAD-GAN: Coherent Style Level Generation from a Single Example" was accepted for oral presentation at AIIDE 2020! You can find our video presentation on YouTube.

Our code for TOAD-GUI and TOAD-GAN has been accepted for the AIIDE 2020 Artifact Evaluation Track! It will be recognized in the AIIDE 2020 Program.

Getting Started

This section includes the necessary steps to get TOAD-GUI running on your system.

Python

You will need Python 3 and the packages specified in requirements.txt. We recommend setting up a virtual environment with pip and installing the packages there.

$ pip3 install -r requirements.txt -f "https://download.pytorch.org/whl/torch_stable.html"

Make sure you use the pip3 that belongs to your previously defined virtual environment.

The GUI is made with Tkinter, which from Python 3.7 onwards is installed by default. If you don't have it installed because of an older version, follow the instructions here.

Java

TOAD-GUI uses the Mario-AI-Framework to play the generated levels. For the Framework to run, Java 11 (or higher) needs to be installed.

Running TOAD-GUI

Once all prerequisites are installed, TOAD-GUI can be started by running main.py.

$ python main.py

Make sure you are using the python installation you installed the prerequisites into.

TOAD-GUI

When running TOAD-GUI you can:

  • toad folder Open a Folder containing a Generator (TOAD-GAN)
  • level folder Open a (previously saved) level .txt to view and/or play
  • gear toad Generate a level of the size defined in the entries below
  • save button Save the currently loaded level level to a .txt or .png image file
  • play button Play the currently loaded level

NOTE: When a generator is opened, it will not show any files in the dialog window. That is intended behavior for askdirectory() of tkinter. Just navigate to the correct path and click "Open" regardless.

When a level is loaded, right clicking a point in the preview will allow you to change the token at that specific spot. If you resample the level, any changes made will be lost.

The labels at the bottom will display the currently loaded path and information. This program was made mostly by one researcher and is not optimized. Impatiently clicking buttons might crash the program.

Edit Mode

In this mode, parts of a generated level can be resampled with TOAD-GAN. The red bounding box shows the area to be changed, while the yellow bounding box shows which blocks can still be affected by that change. The area of effect depends on the scale which is to be resampled and is a result of the Field of View produced by the convolutional layers. Changes in a lower scale will result in larger changes in the final level.

Use the control panel to set the bounding box. The representation inside the panel shows which pixels in the noise map will be changed.

TOAD-GUI_bbox

Resample the noise map in the chosen scale. The "Noise influence" is a learned parameter that indicates how big the effect of resampling in this scale will be.

TOAD-GUI_sc3

Scale 0 is the first scale and results in the most changes. Note that the tokens outside of the bounding box change. This is because of the field of view from the convolutional layers applied to the noise map.

TOAD-GUI_sc0

You can right click a token you want to change and replace it with another token present in the level. This should be done after resampling, as resampling will regenerate the level from the noise maps which will undo these edits.

TOAD-GUI_edit

TOAD-GAN

If you are interested in training your own Generator, refer to the TOAD-GAN Github and copy the folder of your trained generator into the generators/ folder. You should now be able to open it just like the provided generators.

The necessary files are:

generators.pth
noise_amplitudes.pth
noise_maps.pth
num_layer.pth
reals.pth
token_list.pth

Any other files can be deleted if you want to keep your folders tidy.

NOTE: When a generator is opened, it will not show these files in the dialog window. That is intended behavior for askdirectory() of tkinter. Just navigate to the correct path and click "Open" regardless.

Known Bugs

  • If the level play is quit using the window ('x' button in the corner), an error message regarding py4j will occur. In spite of that, the program should continue running normally.

  • If you have two monitors with different resolutions, the GUI and the Java window might not be displayed in the correct resolution. Try moving the windows to the monitor with the other resolution if you encounter this problem. You can also change the DPI awareness for the program in the beginning of GUI.py.

Built With

  • Tkinter - Python package for building GUIs
  • py4j - Python to Java interface
  • Pillow - Python Image Library for displaying images
  • Pytorch - Deep Learning Framework
  • Maven - Used for building the Mario-AI-Framework

Authors

  • Maren Awiszus - Institut für Informationsverarbeitung, Leibniz University Hanover
  • Frederik Schubert - Institut für Informationsverarbeitung, Leibniz University Hanover

Copyright

This program is not endorsed by Nintendo and is only intended for research purposes. Mario is a Nintendo character which the authors don’t own any rights to. Nintendo is also the sole owner of all the graphical assets in the game.

Owner
Maren A.
Maren A.
Vision Transformer for 3D medical image registration (Pytorch).

ViT-V-Net: Vision Transformer for Volumetric Medical Image Registration keywords: vision transformer, convolutional neural networks, image registratio

Junyu Chen 192 Dec 20, 2022
Understanding Convolution for Semantic Segmentation

TuSimple-DUC by Panqu Wang, Pengfei Chen, Ye Yuan, Ding Liu, Zehua Huang, Xiaodi Hou, and Garrison Cottrell. Introduction This repository is for Under

TuSimple 585 Dec 31, 2022
DANet for Tabular data classification/ regression.

Deep Abstract Networks A PyTorch code implemented for the submission DANets: Deep Abstract Networks for Tabular Data Classification and Regression. Do

Ronnie Rocket 55 Sep 14, 2022
Official PyTorch implementation for Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers, a novel method to visualize any Transformer-based network. Including examples for DETR, VQA.

PyTorch Implementation of Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers 1 Using Colab Please notic

Hila Chefer 489 Jan 07, 2023
3D dataset of humans Manipulating Objects in-the-Wild (MOW)

MOW dataset [Website] This repository maintains our 3D dataset of humans Manipulating Objects in-the-Wild (MOW). The dataset contains 512 images in th

Zhe Cao 28 Nov 06, 2022
Constrained Logistic Regression - How to apply specific constraints to logistic regression's coefficients

Constrained Logistic Regression Sample implementation of constructing a logistic regression with given ranges on each of the feature's coefficients (v

1 Dec 29, 2021
Codebase for BMVC 2021 paper "Text Based Person Search with Limited Data"

Text Based Person Search with Limited Data This is the codebase for our BMVC 2021 paper. Please bear with me refactoring this codebase after CVPR dead

Xiao Han 33 Nov 24, 2022
Code for the prototype tool in our paper "CoProtector: Protect Open-Source Code against Unauthorized Training Usage with Data Poisoning".

CoProtector Code for the prototype tool in our paper "CoProtector: Protect Open-Source Code against Unauthorized Training Usage with Data Poisoning".

Zhensu Sun 1 Oct 26, 2021
U2-Net: Going Deeper with Nested U-Structure for Salient Object Detection

The code for our newly accepted paper in Pattern Recognition 2020: "U^2-Net: Going Deeper with Nested U-Structure for Salient Object Detection."

Xuebin Qin 6.5k Jan 09, 2023
ICCV2021 Oral SA-ConvONet: Sign-Agnostic Optimization of Convolutional Occupancy Networks

Sign-Agnostic Convolutional Occupancy Networks Paper | Supplementary | Video | Teaser Video | Project Page This repository contains the implementation

64 Jan 05, 2023
Machine Translation Implement By Bi-GRU And Transformer

Seq2Seq Translation Implement By Bidirectional GRU And Transformer In Pytorch Before You Run The Code You should download the data through the link be

He Wang 2 Oct 27, 2021
[CVPR 2022] Official Pytorch code for OW-DETR: Open-world Detection Transformer

OW-DETR: Open-world Detection Transformer (CVPR 2022) [Paper] Akshita Gupta*, Sanath Narayan*, K J Joseph, Salman Khan, Fahad Shahbaz Khan, Mubarak Sh

Akshita Gupta 127 Dec 27, 2022
Computationally efficient algorithm that identifies boundary points of a point cloud.

BoundaryTest Included are MATLAB and Python packages, each of which implement efficient algorithms for boundary detection and normal vector estimation

6 Dec 09, 2022
AI-UPV at IberLEF-2021 DETOXIS task: Toxicity Detection in Immigration-Related Web News Comments Using Transformers and Statistical Models

AI-UPV at IberLEF-2021 DETOXIS task: Toxicity Detection in Immigration-Related Web News Comments Using Transformers and Statistical Models Description

Angel de Paula 0 Jun 08, 2022
Code for Learning Manifold Patch-Based Representations of Man-Made Shapes, in ICLR 2021.

LearningPatches | Webpage | Paper | Video Learning Manifold Patch-Based Representations of Man-Made Shapes Dmitriy Smirnov, Mikhail Bessmeltsev, Justi

Dima Smirnov 22 Nov 14, 2022
Code for ICLR 2021 Paper, "Anytime Sampling for Autoregressive Models via Ordered Autoencoding"

Anytime Autoregressive Model Anytime Sampling for Autoregressive Models via Ordered Autoencoding , ICLR 21 Yilun Xu, Yang Song, Sahaj Gara, Linyuan Go

Yilun Xu 22 Sep 08, 2022
TilinGNN: Learning to Tile with Self-Supervised Graph Neural Network (SIGGRAPH 2020)

TilinGNN: Learning to Tile with Self-Supervised Graph Neural Network (SIGGRAPH 2020) About The goal of our research problem is illustrated below: give

59 Dec 09, 2022
A python implementation of Deep-Image-Analogy based on pytorch.

Deep-Image-Analogy This project is a python implementation of Deep Image Analogy.https://arxiv.org/abs/1705.01088. Some results Requirements python 3

Peng Lu 171 Dec 14, 2022
This is the code for Deformable Neural Radiance Fields, a.k.a. Nerfies.

Deformable Neural Radiance Fields This is the code for Deformable Neural Radiance Fields, a.k.a. Nerfies. Project Page Paper Video This codebase conta

Google 1k Jan 09, 2023
Optimizaciones incrementales al problema N-Body con el fin de evaluar y comparar las prestaciones de los traductores de Python en el ámbito de HPC.

Python HPC Optimizaciones incrementales de N-Body (all-pairs) con el fin de evaluar y comparar las prestaciones de los traductores de Python en el ámb

Andrés Milla 12 Aug 04, 2022