efficient neural audio synthesis in the waveform domain

Last update: Dec 23, 2022

Overview

neural waveshaping synthesis

real-time neural audio synthesis in the waveform domain

paper • website • colab • audio

by Ben Hayes, Charalampos Saitis, György Fazekas

This repository is the official implementation of Neural Waveshaping Synthesis.

Model Architecture

Requirements

To install:

pip install -r requirements.txt
pip install -e .

We recommend installing in a virtual environment.

Data

We trained our checkpoints on the URMP dataset. Once downloaded, the dataset can be preprocessed using scripts/create_urmp_dataset.py. This will consolidate recordings of each instrument within the dataset and preprocess them according to the pipeline in the paper.

python scripts/create_urmp_dataset.py \
  --gin-file gin/data/urmp_4second_crepe.gin \ 
  --data-directory /path/to/urmp \
  --output-directory /path/to/output \
  --device cuda:0  # torch device string for CREPE model

Alternatively, you can supply your own dataset and use the general create_dataset.py script:

python scripts/create_dataset.py \
  --gin-file gin/data/urmp_4second_crepe.gin \ 
  --data-directory /path/to/dataset \
  --output-directory /path/to/output \
  --device cuda:0  # torch device string for CREPE model

Training

To train a model on the URMP dataset, use this command:

python scripts/train.py \
  --gin-file gin/train/train_newt.gin \
  --dataset-path /path/to/processed/urmp \
  --urmp \
  --instrument vn \  # select URMP instrument with abbreviated string
  --load-data-to-memory

Or to use a non-URMP dataset:

python scripts/train.py \
  --gin-file gin/train/train_newt.gin \
  --dataset-path /path/to/processed/data \
  --load-data-to-memory

efficient neural audio synthesis in the waveform domain

Related tags

Overview

neural waveshaping synthesis

real-time neural audio synthesis in the waveform domain

paper • website • colab • audio

Model Architecture

Requirements

Data

Training

Owner

Ben Hayes

Camera ready code repo for the NeuRIPS 2021 paper: "Impression learning: Online representation learning with synaptic plasticity".

List some popular DeepFake models e.g. DeepFake, FaceSwap-MarekKowal, IPGAN, FaceShifter, FaceSwap-Nirkin, FSGAN, SimSwap, CihaNet, etc.

An NLP library with Awesome pre-trained Transformer models and easy-to-use interface, supporting wide-range of NLP tasks from research to industrial applications.

Scalable Multi-Agent Reinforcement Learning

Enhancing Knowledge Tracing via Adversarial Training

Efficient training of deep recommenders on cloud.

GndNet: Fast ground plane estimation and point cloud segmentation for autonomous vehicles using deep neural networks.

A multi-mode modulator for multi-domain few-shot classification (ICCV)

Colour detection is necessary to recognize objects, it is also used as a tool in various image editing and drawing apps.

torchbearer: A model fitting library for PyTorch

SuperSonic, a new open-source framework to allow compiler developers to integrate RL into compilers easily, regardless of their RL expertise

Dynamics-aware Adversarial Attack of 3D Sparse Convolution Network

Vision Deep-Learning using Tensorflow, Keras.

FairFuzz: AFL extension targeting rare branches

一个多语言支持、易使用的 OCR 项目。An easy-to-use OCR project with multilingual support.

PyTorch implementation of the TTC algorithm

Code for CVPR2019 Towards Natural and Accurate Future Motion Prediction of Humans and Animals

SustainBench: Benchmarks for Monitoring the Sustainable Development Goals with Machine Learning

Public repository created to store my custom-made tools for Just Dance (UbiArt Engine)

Pixel-wise segmentation on VOC2012 dataset using pytorch.