Loading
Du hast einen ComfyUI-Workflow? Erstelle direkt im Browser eine verständliche Workflow-Dokumentation.Workflow verständlich machen.Jetzt ausprobieren ↗
> Note: For full-duplex (real-time) inference, use the 8-bit variant instead. 4-bit quantization degrades PersonaPlex response quality significantly — INT8 is both 30% faster (112ms vs 158ms/step) and produces...
Modellquelle
Quellenbeschreibung
Note: For full-duplex (real-time) inference, use the 8-bit variant instead. 4-bit quantization degrades PersonaPlex response quality significantly — INT8 is both 30% faster (112ms vs 158ms/step) and produces coherent responses where INT4 generates garbled output.
PersonaPlex 7B full-duplex speech-to-speech model converted to MLX safetensors with 4-bit quantization for Apple Silicon.
Quellen
1 QuelleVerifiziert 14. Aug.
Modellartefakte
40 ArtefakteQuellenauszüge
2 AuszügeConverted from nvidia/personaplex-7b-v1 (based on Kyutai Moshi architecture).
Swift inference: soniqo/speech-swift
Library Docs: soniqo.audio
| Component | Architecture | Size |
|---|---|---|
| Temporal Transformer | 32-layer, 4096d, 32 heads (7B params) | ~3.5 GB (4-bit) |
| Depformer | 6-layer, 1024d, 16 heads, per-codebook weights | ~50 MB (fp16) |
| Mimi Codec | SEANet encoder/decoder + 8L transformer + 16 RVQ codebooks | ~370 MB (fp16) |
| Embeddings | Text + 16 audio embeddings + output heads | ~940 MB (fp16) |
| Total | ~4.9 GB |
[User Audio 24kHz] → [Mimi Encoder] → 16 codebook tokens @ 12.5Hz
↓
[Temporal Transformer: 32L, dim=4096, 7B params]
17 streams: text + 8 user audio + 8 agent audio
↓
[Depformer: 6L, dim=1024, per-codebook weights]
16 sequential steps → agent audio codebook tokens
↓
[Agent Audio 24kHz] ← [Mimi Decoder] ← codebook tokens @ 12.5Hz
18 voice presets available:
| Category | Voices |
|---|---|
| Natural Female | NATF0, NATF1, NATF2, NATF3 |
| Natural Male | NATM0, NATM1, NATM2, NATM3 |
| Variety Female | VARF0, VARF1, VARF2, VARF3, VARF4 |
| Variety Male | VARM0, VARM1, VARM2, VARM3, VARM4 |
temporal.safetensors — Temporal transformer (4-bit quantized, group_size=64)depformer.safetensors — Depformer layers + input projections (fp16)embeddings.safetensors — Text/audio embeddings + output heads (fp16)mimi.safetensors — Mimi neural audio codec (fp16)voices/*.safetensors — Voice preset embeddingstokenizer_spm_32k_3.model — SentencePiece tokenizerconfig.json — Model configurationin_proj): kept fp16 (packed Q+K+V format)import PersonaPlex
let model = try await PersonaPlexModel.fromPretrained()
let response = model.respond(
userAudio: audioSamples, // [Float] 24kHz mono
voice: .NATM0,
maxSteps: 500
)
swift run personaplex-cli --input question.wav --output response.wav --voice NATM0
See soniqo/speech-swift for build instructions.
CC-BY-NC-4.0 (same as upstream PersonaPlex)
@article{nguyen2025personaplex,
title={PersonaPlex: Enhancing Human-Centric AI Through Full-Duplex Multi-Turn Conversations With Persona-Conditioned Voice Responses},
author={Nguyen, Tu Anh and others},
journal={arXiv preprint arXiv:2504.07966},
year={2025}
}
mimi.safetensors
safetensors · 367 MB · SHA-256 09b782f06298…3f50 · Hugging Face
voices/NATF0.safetensors
safetensors · 408 KB · SHA-256 d95e5096049c…b2eb · Hugging Face
Herunterladenvoices/NATF1.safetensors
safetensors · 384 KB · SHA-256 559f2e18438c…1dc5 · Hugging Face
Herunterladenvoices/NATF2.safetensors
safetensors · 408 KB · SHA-256 5a1813cd0fcb…2e3f · Hugging Face
Herunterladenvoices/NATF3.safetensors
safetensors · 408 KB · SHA-256 7f4f7a486032…6e4d · Hugging Face
Herunterladenvoices/NATM0.safetensors
safetensors · 400 KB · SHA-256 1079ee62d0f5…9860 · Hugging Face
Herunterladenvoices/NATM1.safetensors
safetensors · 408 KB · SHA-256 0f55dc51054a…b55e · Hugging Face
Herunterladenvoices/NATM2.safetensors
safetensors · 400 KB · SHA-256 8e30e7a3d701…1485 · Hugging Face
Herunterladenvoices/NATM3.safetensors
safetensors · 368 KB · SHA-256 e12f935112e2…be7c · Hugging Face
Herunterladenvoices/VARF0.safetensors
safetensors · 544 KB · SHA-256 30d9e5b0a308…55b1 · Hugging Face
Herunterladenvoices/VARF1.safetensors
safetensors · 400 KB · SHA-256 a68c0712d14b…15d3 · Hugging Face
Herunterladenvoices/VARF2.safetensors
safetensors · 408 KB · SHA-256 593cab99cd41…ed33 · Hugging Face
Herunterladenvoices/VARF3.safetensors
safetensors · 496 KB · SHA-256 5199bfdf64e9…81a6 · Hugging Face
Herunterladenvoices/VARF4.safetensors
safetensors · 456 KB · SHA-256 92197c4bcf0b…d873 · Hugging Face
Herunterladenvoices/VARM0.safetensors
safetensors · 344 KB · SHA-256 b1d50afb0011…b646 · Hugging Face
Herunterladenvoices/VARM1.safetensors
safetensors · 368 KB · SHA-256 24429a7e14b9…683b · Hugging Face
Herunterladenvoices/VARM2.safetensors
safetensors · 552 KB · SHA-256 67b0683ea5d3…d7c8 · Hugging Face
Herunterladenvoices/VARM3.safetensors
safetensors · 368 KB · SHA-256 cb5789fe6b2a…fe43 · Hugging Face
Herunterladenvoices/VARM4.safetensors
safetensors · 440 KB · SHA-256 3ab0fbda59ae…8c1d · Hugging Face
Herunterladen--- language: - en license: cc-by-nc-4.0 tags: - speech-to-speech - full-duplex - mlx - apple-silicon - personaplex - moshi base_model: nvidia/personaplex-7b-v1 pipeline_tag: audio-to-audio --- > **Note**: For full-duplex (real-time) inference, use the [8-bit variant](https://huggingface.co/aufklarer/PersonaPlex-7B-MLX-8bit) instead. 4-bit quantization degrades PersonaPlex response quality significantly — INT8 is both 30% faster (112ms vs 158ms/step) and produces coherent responses where INT4 generates garbled output. # PersonaPlex-7B MLX 4-bit [PersonaPlex](https://arxiv.org/abs/2504.07966) 7B full-duplex speech-to-speech model converted to MLX safetensors with 4-bit quantization for Apple Silicon. Converted from [nvidia/personaplex-7b-v1](https://huggingface.co/nvidia/personaplex-7b-v1) (based on [Kyutai Moshi](https://kyutai.org/moshi/) architecture). **Swift inference**: [soniqo/speech-swift](https://github.com/soniqo/speech-swift) **Library Docs**: [soniqo.audio](https://soniqo.audio) **Blog**: [PersonaPlex on Apple Silicon — Full-Duplex Speech-to-Speech in Native Swift with MLX](https://blog.ivan.digital/nvidia-personaplex-7b-on-apple-silicon-full-duplex-speech-to-speech-in-native-swift-with-mlx-0aa5276f2e23) ## Model Details | Component | Architecture | Size | |-----------|-------------|------| | Temporal Transformer | 32-layer, 4096d, 32 heads (7B params) | ~3.5 GB (4-bit) | | Depformer | 6-layer, 1024d, 16 heads, per-codebook weights | ~50 MB (fp16) | | Mimi Codec | SEANet encoder/decoder + 8L transformer + 16 RVQ codebooks | ~370 MB (fp16) | | Embeddings | Text + 16 audio embeddings + output heads | ~940 MB (fp16) | | **Total** | | **~4.9 GB** | ## Architecture ``` [User Audio 24kHz] → [Mimi Encoder] → 16 codebook tokens @ 12.5Hz ↓ [Temporal Transformer: 32L, dim=4096, 7B params] 17 streams: text + 8 user audio + 8 agent audio ↓ [Depformer: 6L, dim=1024, per-codebook weights] 16 sequential steps → agent audio codebook tokens ↓ [Agent Audio 24kHz] ← [Mimi Decoder] ← codebook tokens @ 12.5Hz ``` ## Voices 18 voice presets available: | Category | Voices | |----------|--------| | Natural Female | NATF0, NATF1, NATF2, NATF3 | | Natural Male | NATM0, NATM1, NATM2, NATM3 | | Variety Female | VARF0, VARF1, VARF2, VARF3, VARF4 | | Variety Male | VARM0, VARM1, VARM2, VARM3, VARM4 | ## Files - `temporal.safetensors` — Temporal...
Source context: 9550 downloads · 42 likes · Pipeline audio-to-audio · Library mlx · Repo aufklarer/PersonaPlex-7B-MLX-4bit