Use VITS and Opencpop to develop singing voice synthesis; Maybe it will VISinger.

Related tags

Deep LearningVITS_Singing
Overview

Init

Use VITS and Opencpop to develop singing voice synthesis; Maybe it will VISinger.

本项目基于

https://github.com/jaywalnut310/vits
https://github.com/SJTMusicTeam/Muskits/
https://wenet.org.cn/opencpop/ 歌声数据

使用muskit数据预处理，获得初步数据

cd egs/opencpop/svs1/
./local/data.sh

VISinger_data
--lable
--midi_dump
--wav_dump

采样率转换

python wave_16k.py
--wav_dump
--wav_dump_16k

使用muskit将数据处理成vits的格式

1, 将lable进行拆分
python muskit/data_label_single.py

label_dump,midi_dump,wav_dump:一个文件一个标注

注意：label和lable的混用（两个单词都是对的）

VISinger_data
--label_dump
--midi_dump
--wav_dump
--wav_dump_16k

2, 将label和midi处理为frame对应的发音单元和音符（基音）
python muskit/data_format_vits.py
VISinger_data
--label_vits
--label_dump
--midi_dump
--wav_dump
--wav_dump_16k

3, 生成VITS需要的files，并分割为train和dev，test不需要（可以手动设计）
python muskit/data_format_vits.py

vits_file.txt 中的内容格式：wave path|label path|pitch path;

cp vits_file.txt VISinger/filelists/
cd VISinger/

python preprocess.py 分割为train和dev

VITS训练

cd VISinger
CUDA_VISIBLE_DEVICES=0 python train.py -c configs/singing_base.json -m singing_base 2>exit_error.log;cat exit_error.log
python vsinging_infer.py

使用16K节约内存，方便模型修改

编辑midi，然后测试

cd ../;python muskit/infer_midi.py;cd -;python vsinging_edit.py

LOSS值 MEL谱

样例音频

vits_singing_样例.wav

You might also like...
In this project, we develop a face recognize platform based on MTCNN object-detection netcwork and FaceNet self-supervised network.
In this project, we develop a face recognize platform based on MTCNN object-detection netcwork and FaceNet self-supervised network.

模式识别大作业——人脸检测与识别平台 本项目是一个简易的人脸检测识别平台，提供了人脸信息录入和人脸识别的功能。前端采用 html+css+js，后端采用 pytorch，

Xuhua Huang 5 Aug 2, 2022
Official codebase used to develop Vision Transformer, MLP-Mixer, LiT and more.

Big Vision This codebase is designed for training large-scale vision models on Cloud TPU VMs. It is based on Jax/Flax libraries, and uses tf.data and

Google Research 701 Jan 3, 2023
Use MATLAB to simulate the signal and extract features. Use PyTorch to build and train deep network to do spectrum sensing.

Deep-Learning-based-Spectrum-Sensing Use MATLAB to simulate the signal and extract features. Use PyTorch to build and train deep network to do spectru

null 10 Dec 14, 2022
Transfer style api - An API to use with Tranfer Style App, where you can use two image and transfer the style

Transfer Style API It's an API to use with Tranfer Style App, where you can use

Brian Alejandro 1 Feb 13, 2022
Voice of Pajlada with model and weights.

Pajlada TTS Stripped down version of ForwardTacotron (https://github.com/as-ideas/ForwardTacotron) with pretrained weights for Pajlada's (https://gith

null 6 Sep 3, 2021
A voice recognition assistant similar to amazon alexa, siri and google assistant.
A voice recognition assistant similar to amazon alexa, siri and google assistant.

kenyan-Siri Build an Artificial Assistant Full tutorial (video) To watch the tutorial, click on the image below Installation For windows users (run th

Alison Parker 3 Aug 19, 2022
An implementation of
An implementation of "Optimal Textures: Fast and Robust Texture Synthesis and Style Transfer through Optimal Transport"

Optex An implementation of Optimal Textures: Fast and Robust Texture Synthesis and Style Transfer through Optimal Transport for TU Delft CS4240. You c

Hans Brouwer 33 Jan 5, 2023
this is a lite easy to use virtual keyboard project for anyone to use
this is a lite easy to use virtual keyboard project for anyone to use

virtual_Keyboard this is a lite easy to use virtual keyboard project for anyone to use motivation I made this for this year's recruitment for RobEn AA

Mohamed Emad 3 Oct 23, 2021
A collection of easy-to-use, ready-to-use, interesting deep neural network models
A collection of easy-to-use, ready-to-use, interesting deep neural network models

Interesting and reproducible research works should be conserved. This repository wraps a collection of deep neural network models into a simple and un

Aria Ghora Prabono 16 Jun 16, 2022
Comments
  • couple of questions

    couple of questions

    Hello how are you ! very cool stuff you have here ,I can clearly see you love singing voice synthesis (SVS) from your forks and repos !! i wanted to ask is that a fully working Visingerr or is it a try from you to make it to sing , like can it be tested on a custom English data and have like results the same as or near the demo in the paper. Also do you have like other samples i can hear , i know that you tested it on opencpop that has almost 5.2 hours of singing data , and also in the paper they trained Visingerr for 600k iterations right ? how many iterations did you achieve on the opencpop to get the result linked below (vits_singing_样例.wav). to be honest i thought vits is data hungry like tacotron2 or fastspeech (aka needs a lot of data to get great results) , that opencpop result of your is so impressive for 5.2 hours data , i also wonder if you lowered the sample rate of opencpop from 44.1 KHz to 22KHz as i heard 44.1 KHz takes alot of time to train x10 the time needed.

    迫不及待地想知道你的消息 :)

    opened by dutchsing009 5
  • 问题

    问题

    python prepare/data_vits.py 输出 1,../VISinger_data/label_vits/XXX._label.npy|XXX_score.npy|XXX_pitch.npy|XXX_slurs.npy 2,filelists/vits_file.txt 内容格式：wave path|label path|score path|pitch path|slurs path;

    请问1 2这两步是怎么操作？

    opened by baipeng0110 3
  • 训练结果

    训练结果

    目前模型缺乏时长预测模型和基音预测模型； 训练语料中的句子修改歌词的效果；

    原歌词：雨淋湿了天空灰得更讲究

    https://user-images.githubusercontent.com/16432329/164953151-4c2513cb-f336-416b-8f04-604f13e63368.MP4

    修改歌词：你闹够了没有让我更难受

    https://user-images.githubusercontent.com/16432329/164953155-16c72670-cc89-40bc-99fe-42781c9dcdc0.MP4

    help wanted 
    opened by MaxMax2016 0
  • About release models and VISinger

    About release models and VISinger

    Hi

    This is a fantastic project that I have ever seen.

    Could you please share the released model? As on the inference step, it is said that "using the released model"

    Also, is there any plan to implement the VISinger model?

    Thank you!

    opened by shiyanpei0826 1
Owner
AmorTX
Speech
AmorTX
GitHub Repository
🤗 Transformers: State-of-the-art Natural Language Processing for Pytorch, TensorFlow, and JAX.

English | 简体中文 | 繁體中文 | 한국어 State-of-the-art Natural Language Processing for Jax, PyTorch and TensorFlow 🤗 Transformers provides thousands of pretrai

Hugging Face 77.4k Jan 05, 2023
Taking A Closer Look at Domain Shift: Category-level Adversaries for Semantics Consistent Domain Adaptation

Taking A Closer Look at Domain Shift: Category-level Adversaries for Semantics Consistent Domain Adaptation (CVPR2019) This is a pytorch implementatio

Yawei Luo 280 Jan 01, 2023
Semantic similarity computation with different state-of-the-art metrics

Semantic similarity computation with different state-of-the-art metrics Description • Installation • Usage • License Description TaxoSS is a semantic

6 Jun 22, 2022
My implementation of DeepMind's Perceiver

DeepMind Perceiver (in PyTorch) Disclaimer: This is not official and I'm not affiliated with DeepMind. My implementation of the Perceiver: General Per

Louis Arge 55 Dec 12, 2022
Large dataset storage format for Pytorch

H5Record Large dataset ( 100G, = 1T) storage format for Pytorch (wip) Support python 3 pip install h5record Why? Writing large dataset is still a

theblackcat102 43 Oct 22, 2022
[ WSDM '22 ] On Sampling Collaborative Filtering Datasets

On Sampling Collaborative Filtering Datasets This repository contains the implementation of many popular sampling strategies, along with various expli

Noveen Sachdeva 17 Dec 08, 2022
🚀 An end-to-end ML applications using PyTorch, W&B, FastAPI, Docker, Streamlit and Heroku

🚀 An end-to-end ML applications using PyTorch, W&B, FastAPI, Docker, Streamlit and Heroku

Made With ML 82 Jun 26, 2022
Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Driving

SalsaNext: Fast, Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Driving Abstract In this paper, we introduce SalsaNext f

308 Jan 04, 2023
A python comtrade load library accelerated by go

Comtrade-GRPC Code for python used is mainly from dparrini/python-comtrade. Just patch the code in BinaryDatReader.parse for parsing a little more eff

Bo 1 Dec 27, 2021
LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation

LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation by Junjue Wang, Zhuo Zheng, Ailong Ma, Xiaoyan Lu, and Yanfei Zh

Payphone 8 Nov 21, 2022
The easiest way to use deep metric learning in your application. Modular, flexible, and extensible. Written in PyTorch.

News December 27: v1.1.0 New loss functions: CentroidTripletLoss and VICRegLoss Mean reciprocal rank + per-class accuracies See the release notes Than

Kevin Musgrave 5k Jan 05, 2023
A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.

Website | Documentation | Tutorials | Installation | Release Notes CatBoost is a machine learning method based on gradient boosting over decision tree

CatBoost 6.9k Jan 04, 2023
Hierarchical User Intent Graph Network for Multimedia Recommendation

Hierarchical User Intent Graph Network for Multimedia Recommendation This is our Pytorch implementation for the paper: Hierarchical User Intent Graph

6 Jan 05, 2023
Mini Software that give reminder to drink water as per your weight.

Water Notification Desktop Python The Mini Software built in Python (tkinter) that will remind you to drink water on specific time span based on your

Om Jogani 5 Dec 16, 2022
Codes for 'Dual Parameterization of Sparse Variational Gaussian Processes'

Dual Parameterization of Sparse Variational Gaussian Processes Documentation | Notebooks | API reference Introduction This repository is the official

AaltoML 7 Dec 23, 2022
InterFaceGAN - Interpreting the Latent Space of GANs for Semantic Face Editing

InterFaceGAN - Interpreting the Latent Space of GANs for Semantic Face Editing Figure: High-quality facial attributes editing results with InterFaceGA

GenForce: May Generative Force Be with You 1.3k Jan 09, 2023
RLMeta is a light-weight flexible framework for Distributed Reinforcement Learning Research.

RLMeta rlmeta - a flexible lightweight research framework for Distributed Reinforcement Learning based on PyTorch and moolib Installation To build fro

Meta Research 281 Dec 22, 2022
TensorFlow-based implementation of "Pyramid Scene Parsing Network".

PSPNet_tensorflow Important Code is fine for inference. However, the training code is just for reference and might be only used for fine-tuning. If yo

HsuanKung Yang 323 Dec 20, 2022
KaziText is a tool for modelling common human errors.

KaziText KaziText is a tool for modelling common human errors. It estimates probabilities of individual error types (so called aspects) from grammatic

ÚFAL 3 Nov 24, 2022
PyTorch implementation of CloudWalk's recent work DenseBody

densebody_pytorch PyTorch implementation of CloudWalk's recent paper DenseBody. Note: For most recent updates, please check out the dev branch. Update

Lingbo Yang 401 Nov 19, 2022