Axii Diarization Coreml

--- license: other license_name: mixed license_link: LICENSE.md tags: - speaker-diarization - coreml - apple - macos - ios - sortformer - wespeaker - speaker-embedding language: - en pipeline_tag: audio-classification --- # Speaker Diarization CoreML Models CoreML conversions of speaker diarization and speaker embedding models for on-device inference on Apple platforms. ## Models | Model | Original | Size | Description | |-------|----------|------|-------------| | `sortformer_4spk_v21.mlpackage` | [nvidia/diar_streaming_sortformer_4spk-v2.1](https://huggingface.co/nvidia/diar_streaming_sortformer_4spk-v2.1) | 441 MB | Sortformer diarization model — end-to-end neural speaker diarization supporting up to 4 speakers, streaming capable | | `wespeaker_resnet34.mlpackage` | [WeSpeaker ResNet34](https://github.com/wenet-e2e/wespeaker) | 25 MB | ResNet34 speaker embedding model — extracts 256-dim speaker embeddings for speaker verification and identification | ## Format Both models are in Apple `.mlpackage` format (FP32). On first load, CoreML compiles them to `.mlmodelc` and caches the compiled version for subsequent fast loading. - **Sortformer**: Input `mel_features (B, 128, T)` → Output `speaker_probs (B, T/8, 4)` sigmoid probabilities per speaker per frame - **ResNet34**: Input `fbank_features (1, 80, T)` → Output `embedding (1, 256)` speaker embedding vector ## Usage These models are designed for use with the [AxiiDiarization](https://github.com/AugustDev/AxiiDiarization) Swift library: ```swift import AxiiDiarization let pipeline = try DiarizationPipeline( sortformerModelPath: "path/to/sortformer_4spk_v21.mlpackage", embModelPath: "path/to/wespeaker_resnet34.mlpackage" ) let result = try pipeline.run(samples: audioSamples) for segment in result.segments { print("\(segment.speaker.label): \(segment.start)s - \(segment.end)s") } ``` ## Licenses The models in this repository have separate licenses from their original authors: - **Sortformer v2.1**: Licensed by NVIDIA Corporation under the [NVIDIA Open Model License](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/). Commercial use permitted. See original model card: [nvidia/diar_streaming_sortformer_4spk-v2.1](https://huggingface.co/nvidia/diar_streaming_sortformer_4spk-v2.1) - **WeSpeaker ResNet34**: Licensed under [Apache License 2.0](https:/...