Listen to the samples on Soundcloud.

Paper: https://openreview.net/forum?id=NyR8OZFHw6i

Colab Demo: https://colab.research.google.com/drive/1UAKFkbPQTfkYMq1GxXfGZOJXOXU_svo6

• FIGARO: Generating Symbolic Music with Fine-Grained Artistic Control
  • Getting started
    • Setup
    • Preparing the Data
    • Download Pre-Trained Models
  • Training
  • Generation
  • Evaluation
  • Parameters
    • Training (train.py)
    • Generation (generate.py)
    • Evaluation (evaluate.py)

Prerequisites:

• Python 3.9
• Conda

1. Clone this repository to your disk
2. Install required packages (see requirements.txt). With venv:

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

To train models and to generate new samples, we use the Lakh MIDI dataset (altough any collection of MIDI files can be used).

1. Download (size: 1.6GB) and extract the archive file:

wget http://hog.ee.columbia.edu/craffel/lmd/lmd_full.tar.gz
tar -xzf lmd_full.tar.gz

2. You may wish to remove the archive file now: rm lmd_full.tar.gz

If you don't wish to train your own models, you can download our pre-trained models.

1. Download (size: 2.3GB) and extract the archive file:

wget -O checkpoints.zip https://polybox.ethz.ch/index.php/s/a0HUHzKuPPefWkW/download
unzip checkpoints.zip

2. You may wish to remove the archive file now: rm checkpoints.zip

Training arguments such as model type, batch size, model params are passed to the training scripts via environment variables.

Available model types are:

• vq-vae: VQ-VAE model used for the learned desription
• figaro: FIGARO with both the expert and learned description
• figaro-expert: FIGARO with only the expert description
• figaro-learned: FIGARO with only the learned description
• figaro-no-inst: FIGARO (expert) without instruments
• figaro-no-chord: FIGARO (expert) without chords
• figaro-no-meta: FIGARO (expert) without style (meta) information
• baseline: Unconditional decoder-only baseline following Huang et al. (2018)

Example invocation of the training script is given by the following command:

MODEL=figaro-expert python src/train.py

For models using the learned description (figaro and figaro-learned), a pre-trained VQ-VAE checkpoint needs to be provided as well:

MODEL=figaro VAE_CHECKPOINT=./checkpoints/vq-vae.ckpt python src/train.py

To generate samples, make sure you have a trained checkpoint prepared (either download one or train it yourself). For this script, make sure that the dataset is prepared according to Preparing the Data. This is needed to extract descriptions, based on which new samples can be generated.

An example invocation of the generation script is given by the following command:

python src/generate.py --model figaro-expert --checkpoint ./checkpoints/figaro-expert.ckpt

For models using the learned description (figaro and figaro-learned), a pre-trained VQ-VAE checkpoint needs to be provided as well:

python src/generate.py --model figaro --checkpoint ./checkpoints/figaro.ckpt --vae_checkpoint ./checkpoints/vq-vae.ckpt

We provide the evaluation scripts used to calculate the desription metrics on some set of generated samples. Refer to the previous section for how to generate samples yourself.

Example usage:

python src/evaluate.py --samples_dir ./samples/figaro-expert

It has been pointed out that the order of the dataset files (from which the splits are calculated) is non-deterministic and depends on the OS. To address this and to ensure reproducibility, I have added the exact files used for training/validation/testing in the respective file in the splits folder.

The following environment variables are available for controlling hyperparameters beyond their default value.

Model

Optimization

Others

The generation script uses command line arguments instead of environment variables.

The evaluation script uses command line arguments instead of environment variables.