Loading
Du hast einen ComfyUI-Workflow? Erstelle direkt im Browser eine verständliche Workflow-Dokumentation.Workflow verständlich machen.Jetzt ausprobieren ↗
Based on the original kokoro model, see for inference
Modellquelle
Quellenbeschreibung
Based on the original kokoro model, see https://github.com/FluidInference/FluidAudio for inference
We generated the same strings with to gerneate audio between 1s to ~300s in order to test the speed across a range of varying inputs on Pytorch CPU, MPS, and MLX pipeline, and compared it against the native Swift version with Core ML models.
Each pipeline warmed up the models by running through it once with pesudo inputs, and then comparing the raw inference time with the model already loaded. You can see that for the Core ML model, we traded lower memory and very slightly faster inference for longer initial warm-up.
Quellen
1 QuelleVerifiziert 13. Sept.
Modellartefakte
64 ArtefakteANE-ja/KokoroAlbert.mlmodelc/analytics/coremldata.bin
bin · 243 B · SHA-256 14a61873d875…5ccd · Hugging Face
HerunterladenANE-ja/KokoroAlbert.mlmodelc/coremldata.bin
bin · 433 B · SHA-256 70b6d2f84292…63a6 · Hugging Face
HerunterladenQuellenauszüge
2 AuszügeNote that the Pytorch kokoro model in Pytorch has a memory leak issue: https://github.com/hexgrad/kokoro/issues/152
The following tests were ran on M4 Pro, 48GB RAM, Macbook Pro. If you have another device, please do try replicating it as well!
KPipeline benchmark for voice af_heart (warm-up took 0.175s) using hexgrad/kokoro
Test Chars Output (s) Inf(s) RTFx Peak GB
1 42 2.750 0.187 14.737x 1.44
2 129 8.625 0.530 16.264x 1.85
3 254 15.525 0.923 16.814x 2.65
4 93 6.125 0.349 17.566x 2.66
5 104 7.200 0.410 17.567x 2.70
6 130 9.300 0.504 18.443x 2.72
7 197 12.850 0.726 17.711x 2.83
8 6 1.350 0.098 13.823x 2.83
9 1228 76.200 4.342 17.551x 3.19
10 567 35.200 2.069 17.014x 4.85
11 4615 286.525 17.041 16.814x 4.78
Total - 461.650 27.177 16.987x 4.85
I wasn't able to run the MPS model for longer durations, even with PYTORCH_ENABLE_MPS_FALLBACK=1 enabled, it kept crashing for the longer strings.
KPipeline benchmark for voice af_heart (warm-up took 0.568s) using pip package
Test Chars Output (s) Inf(s) RTFx Peak GB
1 42 2.750 0.414 6.649x 1.41
2 129 8.625 0.729 11.839x 1.54
Total - 11.375 1.142 9.960x 1.54
TTS benchmark for voice af_heart (warm-up took an extra 2.155s) using model prince-canuma/Kokoro-82M
Test Chars Output (s) Inf(s) RTFx Peak GB
1 42 2.750 0.347 7.932x 1.12
2 129 8.650 0.597 14.497x 2.47
3 254 15.525 0.825 18.829x 2.65
4 93 6.125 0.306 20.039x 2.65
5 104 7.200 0.343 21.001x 2.65
6 130 9.300 0.560 16.611x 2.65
7 197 12.850 0.596 21.573x 2.65
8 6 1.350 0.364 3.706x 2.65
9 1228 76.200 2.979 25.583x 3.29
10 567 35.200 1.374 25.615x 3.37
11 4615 286.500 11.112 25.783x 3.37
Total - 461.650 19.401 23.796x 3.37
Note that it does take ~15s to compile the model on the first run, subsequent runs are shorter, we expect ~2s to load.
> swift run fluidaudio tts --benchmark
...
FluidAudio TTS benchmark for voice af_heart (warm-up took an extra 2.348s)
Test Chars Ouput (s) Inf(s) RTFx
1 42 2.825 0.440 6.424x
2 129 7.725 0.594 13.014x
3 254 13.400 0.776 17.278x
4 93 5.875 0.587 10.005x
5 104 6.675 0.613 10.889x
6 130 8.075 0.621 13.008x
7 197 10.650 0.627 16.983x
8 6 0.825 0.360 2.290x
9 1228 67.625 2.362 28.625x
10 567 33.025 1.341 24.619x
11 4269 247.600 9.087 27.248x
Total - 404.300 17.408 23.225
Peak memory usage (process-wide): 1.503 GB
special thanks to @laishere for making kokoro ANE possible his source: https://github.com/laishere/kokoro-coreml
ANE-ja/KokoroAlbert.mlmodelc/weights/weight.bin
bin · 5,45 MB · SHA-256 36089a39359b…fabc · Hugging Face
HerunterladenANE-ja/KokoroAlignment.mlmodelc/analytics/coremldata.bin
bin · 243 B · SHA-256 f6074d1039a9…4f3f · Hugging Face
HerunterladenANE-ja/KokoroAlignment.mlmodelc/coremldata.bin
bin · 484 B · SHA-256 9a0fb4a536f6…a4ea · Hugging Face
HerunterladenANE-ja/KokoroAlignment.mlmodelc/weights/weight.bin
bin · 4,03 KB · SHA-256 2e7d69128b59…5f6b · Hugging Face
HerunterladenANE-ja/KokoroNoise_v2.mlmodelc/analytics/coremldata.bin
bin · 243 B · SHA-256 53af6bf61482…3d04 · Hugging Face
HerunterladenANE-ja/KokoroNoise_v2.mlmodelc/coremldata.bin
bin · 440 B · SHA-256 9911047f924b…1a19 · Hugging Face
HerunterladenANE-ja/KokoroNoise_v2.mlmodelc/weights/weight.bin
bin · 4,37 MB · SHA-256 1102fc2d31df…6bc2 · Hugging Face
HerunterladenANE-ja/KokoroNoise_v2.mlpackage/Data/com.apple.CoreML/weights/weight.bin
bin · 4,37 MB · SHA-256 1102fc2d31df…6bc2 · Hugging Face
HerunterladenANE-ja/KokoroNoise.mlmodelc/analytics/coremldata.bin
bin · 243 B · SHA-256 de1b584b3959…b133 · Hugging Face
HerunterladenANE-ja/KokoroNoise.mlmodelc/coremldata.bin
bin · 440 B · SHA-256 6e675b07b45f…283f · Hugging Face
HerunterladenANE-ja/KokoroNoise.mlmodelc/weights/weight.bin
bin · 4,37 MB · SHA-256 5da4ccb85b78…a0d0 · Hugging Face
HerunterladenANE-ja/KokoroPostAlbert.mlmodelc/analytics/coremldata.bin
bin · 243 B · SHA-256 6640044c8755…6c43 · Hugging Face
HerunterladenANE-ja/KokoroPostAlbert.mlmodelc/coremldata.bin
bin · 556 B · SHA-256 86de3ab0c1e8…ba3b · Hugging Face
HerunterladenANE-ja/KokoroPostAlbert.mlmodelc/weights/weight.bin
bin · 13,2 MB · SHA-256 e4f300a23cc2…91c8 · Hugging Face
HerunterladenANE-ja/KokoroProsody.mlmodelc/analytics/coremldata.bin
bin · 243 B · SHA-256 7708aab83dea…c6b8 · Hugging Face
HerunterladenANE-ja/KokoroProsody.mlmodelc/coremldata.bin
bin · 421 B · SHA-256 d65d87f246a6…a33e · Hugging Face
HerunterladenANE-ja/KokoroProsody.mlmodelc/weights/weight.bin
bin · 8,06 MB · SHA-256 d3c2670eb0c5…444c · Hugging Face
HerunterladenANE-ja/KokoroTail_v2.mlmodelc/analytics/coremldata.bin
bin · 243 B · SHA-256 a06e4b91c2b8…2222 · Hugging Face
HerunterladenANE-ja/KokoroTail_v2.mlmodelc/coremldata.bin
bin · 392 B · SHA-256 527271f6df56…d905 · Hugging Face
HerunterladenANE-ja/KokoroTail_v2.mlmodelc/weights/weight.bin
bin · 79,2 KB · SHA-256 8ede31ec20df…22c7 · Hugging Face
HerunterladenANE-ja/KokoroTail.mlmodelc/analytics/coremldata.bin
bin · 243 B · SHA-256 7dd3d6b8cfbc…77db · Hugging Face
HerunterladenANE-ja/KokoroTail.mlmodelc/coremldata.bin
bin · 392 B · SHA-256 f28f17e4217d…f5d3 · Hugging Face
HerunterladenANE-ja/KokoroTail.mlmodelc/weights/weight.bin
bin · 79,2 KB · SHA-256 2d4877b5d272…ecb2 · Hugging Face
HerunterladenANE-ja/KokoroVocoder.mlmodelc/analytics/coremldata.bin
bin · 243 B · SHA-256 1cc4f1e64365…b463 · Hugging Face
HerunterladenANE-ja/KokoroVocoder.mlmodelc/coremldata.bin
bin · 626 B · SHA-256 71bac80d2f2f…8764 · Hugging Face
HerunterladenANE-ja/KokoroVocoder.mlmodelc/weights/weight.bin
bin · 46,6 MB · SHA-256 6d1f96eb5021…2ff1 · Hugging Face
HerunterladenANE-zh/assets/pinyin_phrases.bin
bin · 9,11 MB · SHA-256 ee86607bd17b…6049 · Hugging Face
HerunterladenANE-zh/g2pw/g2pw.mlmodelc/analytics/coremldata.bin
bin · 243 B · SHA-256 b5697bf7c53b…15c7 · Hugging Face
HerunterladenANE-zh/g2pw/g2pw.mlmodelc/coremldata.bin
bin · 515 B · SHA-256 3a4aa20b8e59…e06e · Hugging Face
HerunterladenANE-zh/g2pw/g2pw.mlmodelc/weights/weight.bin
bin · 152 MB · SHA-256 95b5d709600e…5aaa · Hugging Face
HerunterladenANE-zh/g2pw/g2pw.mlpackage/Data/com.apple.CoreML/weights/weight.bin
bin · 152 MB · SHA-256 95b5d709600e…5aaa · Hugging Face
HerunterladenANE-zh/KokoroAlbert.mlmodelc/analytics/coremldata.bin
bin · 243 B · SHA-256 e3b66851b7a6…c94e · Hugging Face
HerunterladenANE-zh/KokoroAlbert.mlmodelc/coremldata.bin
bin · 433 B · SHA-256 bae8190320b0…a86c · Hugging Face
HerunterladenANE-zh/KokoroAlbert.mlmodelc/weights/weight.bin
bin · 5,45 MB · SHA-256 8c3d8458f357…7e5b · Hugging Face
HerunterladenANE-zh/KokoroAlbert.mlpackage/Data/com.apple.CoreML/weights/weight.bin
bin · 5,45 MB · SHA-256 8c3d8458f357…7e5b · Hugging Face
HerunterladenANE-zh/KokoroAlignment.mlmodelc/analytics/coremldata.bin
bin · 243 B · SHA-256 7c84c8820e2e…5443 · Hugging Face
HerunterladenANE-zh/KokoroAlignment.mlmodelc/coremldata.bin
bin · 484 B · SHA-256 8bede8932574…5936 · Hugging Face
HerunterladenANE-zh/KokoroAlignment.mlmodelc/weights/weight.bin
bin · 4,03 KB · SHA-256 2e7d69128b59…5f6b · Hugging Face
HerunterladenANE-zh/KokoroAlignment.mlpackage/Data/com.apple.CoreML/weights/weight.bin
bin · 4,03 KB · SHA-256 2e7d69128b59…5f6b · Hugging Face
HerunterladenANE-zh/KokoroNoise_v2.mlmodelc/analytics/coremldata.bin
bin · 243 B · SHA-256 7eacfc0eb5b0…c012 · Hugging Face
HerunterladenANE-zh/KokoroNoise_v2.mlmodelc/coremldata.bin
bin · 440 B · SHA-256 3d8f933a62fc…99ee · Hugging Face
HerunterladenANE-zh/KokoroNoise_v2.mlmodelc/weights/weight.bin
bin · 4,37 MB · SHA-256 2bf3f47ba885…e24a · Hugging Face
HerunterladenANE-zh/KokoroNoise_v2.mlpackage/Data/com.apple.CoreML/weights/weight.bin
bin · 4,37 MB · SHA-256 2bf3f47ba885…e24a · Hugging Face
HerunterladenANE-zh/KokoroNoise.mlmodelc/analytics/coremldata.bin
bin · 243 B · SHA-256 7eacfc0eb5b0…c012 · Hugging Face
HerunterladenANE-zh/KokoroNoise.mlmodelc/coremldata.bin
bin · 440 B · SHA-256 3d8f933a62fc…99ee · Hugging Face
HerunterladenANE-zh/KokoroNoise.mlmodelc/weights/weight.bin
bin · 4,37 MB · SHA-256 2bf3f47ba885…e24a · Hugging Face
HerunterladenANE-zh/KokoroNoise.mlpackage/Data/com.apple.CoreML/weights/weight.bin
bin · 4,37 MB · SHA-256 2bf3f47ba885…e24a · Hugging Face
HerunterladenANE-zh/KokoroPostAlbert.mlmodelc/analytics/coremldata.bin
bin · 243 B · SHA-256 9ac50dad158a…829f · Hugging Face
HerunterladenANE-zh/KokoroPostAlbert.mlmodelc/coremldata.bin
bin · 556 B · SHA-256 c9246412170c…6f95 · Hugging Face
HerunterladenANE-zh/KokoroPostAlbert.mlmodelc/weights/weight.bin
bin · 13,2 MB · SHA-256 bca73c9f402c…daf7 · Hugging Face
HerunterladenANE-zh/KokoroPostAlbert.mlpackage/Data/com.apple.CoreML/weights/weight.bin
bin · 13,2 MB · SHA-256 bca73c9f402c…daf7 · Hugging Face
HerunterladenANE-zh/KokoroProsody.mlmodelc/analytics/coremldata.bin
bin · 243 B · SHA-256 3c90a008bfe0…54a1 · Hugging Face
HerunterladenANE-zh/KokoroProsody.mlmodelc/coremldata.bin
bin · 422 B · SHA-256 7f1932bc07f5…fc61 · Hugging Face
HerunterladenANE-zh/KokoroProsody.mlmodelc/weights/weight.bin
bin · 8,06 MB · SHA-256 0d7229d31a47…5cf8 · Hugging Face
HerunterladenANE-zh/KokoroProsody.mlpackage/Data/com.apple.CoreML/weights/weight.bin
bin · 8,06 MB · SHA-256 0d7229d31a47…5cf8 · Hugging Face
HerunterladenANE-zh/KokoroTail_v2.mlmodelc/analytics/coremldata.bin
bin · 243 B · SHA-256 cbffea509dfc…9151 · Hugging Face
Herunterladen--- license: apache-2.0 language: - en base_model: - hexgrad/Kokoro-82M pipeline_tag: text-to-speech --- Based on the original kokoro model, see https://github.com/FluidInference/FluidAudio for inference ## Benchmark We generated the same strings with to gerneate audio between 1s to ~300s in order to test the speed across a range of varying inputs on Pytorch CPU, MPS, and MLX pipeline, and compared it against the native Swift version with Core ML models. Each pipeline warmed up the models by running through it once with pesudo inputs, and then comparing the raw inference time with the model already loaded. You can see that for the Core ML model, we traded lower memory and very slightly faster inference for longer initial warm-up. Note that the Pytorch kokoro model in Pytorch has a memory leak issue: https://github.com/hexgrad/kokoro/issues/152 The following tests were ran on M4 Pro, 48GB RAM, Macbook Pro. If you have another device, please do try replicating it as well! ### Kokoro-82M PyTorch (CPU) ```bash KPipeline benchmark for voice af_heart (warm-up took 0.175s) using hexgrad/kokoro Test Chars Output (s) Inf(s) RTFx Peak GB 1 42 2.750 0.187 14.737x 1.44 2 129 8.625 0.530 16.264x 1.85 3 254 15.525 0.923 16.814x 2.65 4 93 6.125 0.349 17.566x 2.66 5 104 7.200 0.410 17.567x 2.70 6 130 9.300 0.504 18.443x 2.72 7 197 12.850 0.726 17.711x 2.83 8 6 1.350 0.098 13.823x 2.83 9 1228 76.200 4.342 17.551x 3.19 10 567 35.200 2.069 17.014x 4.85 11 4615 286.525 17.041 16.814x 4.78 Total - 461.650 27.177 16.987x 4.85 ``` ### Kokoro-82M PyTorch (MPS) I wasn't able to run the MPS model for longer durations, even with `PYTORCH_ENABLE_MPS_FALLBACK=1` enabled, it kept crashing for the longer strings. ```bash KPipeline benchmark for voice af_heart (warm-up took 0.568s) using pip package Test Chars Output (s) Inf(s) RTFx Peak GB 1 42 2.750 0.414 6.649x 1.41 2 129 8.625 0.729 11.839x 1.54 Total - 11.375 1.142 9.960x 1.54 ``` ### Kokoro-82M MLX Pipeline ```bash TTS benchmark for voice af_heart (warm-up took an extra 2.155s) using model prince-canuma/Kokoro-82M Test Chars Output (s) Inf(s) RTFx Peak GB 1 42 2.750 0.347 7.932x 1.12 2 129 8.650 0.597 14.497x 2.47 3 254 15.525 0.825 18.829x 2.65 4 93 6.125 0.306 20.039x 2.65 5 104 7.200 0.343 21.001x 2.65 6 130 9.300 0.560 16.611x 2.65 7 197 12.850 0.596 21.573x 2.65 8 6 1.350 0.364 3.706x 2.65 9 1228 76.200 2.979 25...
Source context: 2090 downloads · 10 likes · Pipeline text-to-speech · Repo FluidInference/kokoro-82m-coreml