Voice-conversion-evaluation
An evaluation toolkit for voice conversion models.
Sample test pair
Generate the metadata for evaluating models.
The directory of parsers contains several available corpus parsers.
python sampler.py [name of source corpus] [path of source dir] [name of target corpus] [path of target dir] -n [number of samples] -nt [number of target utterances] -o [path of output dir]
The pairs of metadata are sorted by src_second for long to short.
The metadata contains:
- source_corpus: The name of the source corpus.
- source_corpus_speaker_number: The number of speaker in source corpus.
- source_random_seed: Random seed used for sampling source utterance.
- target_corpus: The name of the target corpus.
- target_corpus_speaker_number: The number of speaker in target corpus.
- target_random_seed: Random seed used for sampling target utterances.
- n_samples: number of samples
- n_target_samples: number of target utterances
- pairs: List of evaluating pairs
- source_speaker: The name of the source speaker.
- target_speaker: The name of the target speaker.
- src_utt: The relative path of the source utterance, which is relative to the source dir.
- tgt_utts: List of the relative path of target utterances, which is relative to the target dir.
- content: The content of the source utterance.
- src_second: The second of the source utterance.
- converted: The entry does not appear when use sampler, you need to add the relative path for your converted output.
Metrics
The metrics include automatic mean opinion score assessment, character error rate, and speaker verification acceptance rate.
- Automatic mean opinion score assessment
- Ensemble several MBNet which is implemented by sky1456723.
python calculate_objective_metric.py -d [data_dir] -r metrics/mean_opinion_score - Character error rate:
- Use the automatic speech recognition model provided by Hugging Face.
- The word error rate on Librispeech test-other is 3.9.
python calculate_objective_metric.py -d [data_dir] -r metrics/character_error_rate - Speaker verification acceptance rate:
- You can calculate the threshold by
metrics/speaker_verification/equal_error_rate/. - And some pre-calculated thresholds are in
metrics/speaker_verification/equal_error_rate/threshold.yaml.
python calculate_objective_metric.py -d [data_dir] -r metrics/speaker_verification -t [target_dir] -th [threshold path] - You can calculate the threshold by
3 Jun 20, 2022
5 Nov 4, 2022
22 Nov 16, 2022
19 Jun 30, 2022
103 Dec 23, 2022
6 Nov 8, 2022
25.6k Jan 6, 2023
31 Oct 30, 2022
16 Dec 7, 2022
4 Nov 25, 2022
1 Mar 11, 2022
2.3k Jan 05, 2023
19 Dec 17, 2022
2 Feb 17, 2022
5.3k Jan 04, 2023
18 Dec 20, 2022
730 Jan 09, 2023
16 Dec 11, 2022
317 Dec 18, 2022
8 Dec 25, 2022
654 Jan 02, 2023
1.2k Dec 21, 2022
20 Dec 15, 2022
2 Jan 17, 2022
49 Aug 24, 2022
104 Dec 04, 2022
762 Dec 29, 2022
3 Jun 13, 2022