efficient neural audio synthesis in the waveform domain

Last update: Dec 23, 2022

Overview

neural waveshaping synthesis

real-time neural audio synthesis in the waveform domain

paper • website • colab • audio

by Ben Hayes, Charalampos Saitis, György Fazekas

This repository is the official implementation of Neural Waveshaping Synthesis.

Model Architecture

Requirements

To install:

pip install -r requirements.txt
pip install -e .

We recommend installing in a virtual environment.

Data

We trained our checkpoints on the URMP dataset. Once downloaded, the dataset can be preprocessed using scripts/create_urmp_dataset.py. This will consolidate recordings of each instrument within the dataset and preprocess them according to the pipeline in the paper.

python scripts/create_urmp_dataset.py \
  --gin-file gin/data/urmp_4second_crepe.gin \ 
  --data-directory /path/to/urmp \
  --output-directory /path/to/output \
  --device cuda:0  # torch device string for CREPE model

Alternatively, you can supply your own dataset and use the general create_dataset.py script:

python scripts/create_dataset.py \
  --gin-file gin/data/urmp_4second_crepe.gin \ 
  --data-directory /path/to/dataset \
  --output-directory /path/to/output \
  --device cuda:0  # torch device string for CREPE model

Training

To train a model on the URMP dataset, use this command:

python scripts/train.py \
  --gin-file gin/train/train_newt.gin \
  --dataset-path /path/to/processed/urmp \
  --urmp \
  --instrument vn \  # select URMP instrument with abbreviated string
  --load-data-to-memory

Or to use a non-URMP dataset:

python scripts/train.py \
  --gin-file gin/train/train_newt.gin \
  --dataset-path /path/to/processed/data \
  --load-data-to-memory

efficient neural audio synthesis in the waveform domain

Related tags

Overview

neural waveshaping synthesis

real-time neural audio synthesis in the waveform domain

paper • website • colab • audio

Model Architecture

Requirements

Data

Training

Owner

Ben Hayes

Tackling data scarcity in Speech Translation using zero-shot multilingual Machine Translation techniques

Data reduction pipeline for KOALA on the AAT.

Sparse Progressive Distillation: Resolving Overfitting under Pretrain-and-Finetune Paradigm

This repository is a basic Machine Learning train & validation Template (Using PyTorch)

USAD - UnSupervised Anomaly Detection on multivariate time series

PSANet: Point-wise Spatial Attention Network for Scene Parsing, ECCV2018.

Project NII pytorch scripts

A New Approach to Overgenerating and Scoring Abstractive Summaries

Weighted K Nearest Neighbors (kNN) algorithm implemented on python from scratch.

This project provides an unsupervised framework for mining and tagging quality phrases on text corpora with pretrained language models (KDD'21).

A Strong Baseline for Image Semantic Segmentation

Rethinking the U-Net architecture for multimodal biomedical image segmentation

Histology images query (unsupervised)

Fiddle is a Python-first configuration library particularly well suited to ML applications.

Research Artifact of USENIX Security 2022 Paper: Automated Side Channel Analysis of Media Software with Manifold Learning

a minimal terminal with python 😎😉

NNR conformation conditional and global probabilities estimation and analysis in peptides or proteins fragments

SymmetryNet: Learning to Predict Reflectional and Rotational Symmetries of 3D Shapes from Single-View RGB-D Images

Ascend your Jupyter Notebook usage

Much faster than SORT(Simple Online and Realtime Tracking), a little worse than SORT