efficient neural audio synthesis in the waveform domain

Last update: Dec 23, 2022

Overview

neural waveshaping synthesis

real-time neural audio synthesis in the waveform domain

paper • website • colab • audio

by Ben Hayes, Charalampos Saitis, György Fazekas

This repository is the official implementation of Neural Waveshaping Synthesis.

Model Architecture

Requirements

To install:

pip install -r requirements.txt
pip install -e .

We recommend installing in a virtual environment.

Data

We trained our checkpoints on the URMP dataset. Once downloaded, the dataset can be preprocessed using scripts/create_urmp_dataset.py. This will consolidate recordings of each instrument within the dataset and preprocess them according to the pipeline in the paper.

python scripts/create_urmp_dataset.py \
  --gin-file gin/data/urmp_4second_crepe.gin \ 
  --data-directory /path/to/urmp \
  --output-directory /path/to/output \
  --device cuda:0  # torch device string for CREPE model

Alternatively, you can supply your own dataset and use the general create_dataset.py script:

python scripts/create_dataset.py \
  --gin-file gin/data/urmp_4second_crepe.gin \ 
  --data-directory /path/to/dataset \
  --output-directory /path/to/output \
  --device cuda:0  # torch device string for CREPE model

Training

To train a model on the URMP dataset, use this command:

python scripts/train.py \
  --gin-file gin/train/train_newt.gin \
  --dataset-path /path/to/processed/urmp \
  --urmp \
  --instrument vn \  # select URMP instrument with abbreviated string
  --load-data-to-memory

Or to use a non-URMP dataset:

python scripts/train.py \
  --gin-file gin/train/train_newt.gin \
  --dataset-path /path/to/processed/data \
  --load-data-to-memory

efficient neural audio synthesis in the waveform domain

Related tags

Overview

neural waveshaping synthesis

real-time neural audio synthesis in the waveform domain

paper • website • colab • audio

Model Architecture

Requirements

Data

Training

Owner

Ben Hayes

Faster RCNN with PyTorch

Official implementation of VaxNeRF (Voxel-Accelearated NeRF).

Unofficial PyTorch Implementation of "DOLG: Single-Stage Image Retrieval with Deep Orthogonal Fusion of Local and Global Features"

Simple is not Easy: A Simple Strong Baseline for TextVQA and TextCaps[AAAI2021]

A benchmark dataset for mesh multi-label-classification based on cube engravings introduced in MeshCNN

Distributed Evolutionary Algorithms in Python

Accurate identification of bacteriophages from metagenomic data using Transformer

A convolutional recurrent neural network for classifying A/B phases in EEG signals recorded for sleep analysis.

Official Implementation of DE-DETR and DELA-DETR in "Towards Data-Efficient Detection Transformers"

An implementation of the BADGE batch active learning algorithm.

Band-Adaptive Spectral-Spatial Feature Learning Neural Network for Hyperspectral Image Classification

Unofficial implementation (replicates paper results!) of MINER: Multiscale Implicit Neural Representations in pytorch-lightning

Pytorch implementation of COIN, a framework for compression with implicit neural representations 🌸

Flower - A Friendly Federated Learning Framework

Hypercomplex Neural Networks with PyTorch

FinEAS: Financial Embedding Analysis of Sentiment 📈

Adversarial Graph Representation Adaptation for Cross-Domain Facial Expression Recognition (AGRA, ACM 2020, Oral)

An educational resource to help anyone learn deep reinforcement learning.

An All-MLP solution for Vision, from Google AI

This repo. is an implementation of ACFFNet, which is accepted for in Image and Vision Computing.