Qwen2.5 Omni 3B Audio CoreAI
qwen-research
Fonte do modelo
- Tipo
- Other
- Tamanho do arquivo
- 3,91 GB
- Modelo base
- Qwen/Qwen2.5-Omni-3B
- Versão
- vRevision 348148d
- Atualizado
- 19 de out. de 2026
- Downloads
- 2,3 mil
- Fonte
- Hugging Face
Descrição da fonte
Core AI is Apple's on-device ML runtime in iOS 27 / macOS 27 and the successor to Core ML: PyTorch models are exported with Apple's coreai-torch (LLMs: coreai.llm.export) into .aimodel bundles that run on the GPU or the Neural Engine, e.g. Qwen3-8B 4-bit decodes at 94 tok/s on an M4 Max GPU, MLX 90 under the same protocol (apple-silicon-llm-bench, macOS 27 beta 26A5353q, 2026-06-11).
This model has no row on DeviceMark, the on-device LLM leaderboard.
Qwen2.5-Omni-3B Audio Understanding — Core AI
Qwen2.5-Omni-3B's Thinker converted to Apple
Core AI (.aimodel / .aimodelc, iOS 27 / macOS 27) for on-device audio understanding —
the model describes the sounds it hears (events, texture, emotion, music), it is not a
transcriber. "I hear a loud hissing sound." · "...a continuous sine wave sound." · "...a series of
beeps."
Part of the CoreAI-Model-Zoo. Device-verified on iPhone 17 Pro (A19 Pro) and M4 Max.
Use it
New to Core AI? Start with CoreAIKit 0.7.3. Follow its requirements and first-run steps for qwen3-0.6b, then open the same release's ChatDemo. The README records the tested OS/SDK and download size; model and device coverage is stated per example.
⚡ One line — this model is the default behind the kit's task op
(import CoreAIOps; no session, no model plumbing, downloads on first use):
let scene = try await CoreAI.describeAudio(audioURL)
Every op, one shape — Cookbook.
▶️ Run it (source) — the AudioChat runner (GUI + CLI, one app for every audio-understanding model in the catalog):
git clone --branch 0.7.3 --depth 1 https://github.com/john-rocky/coreai-kit
export DEVELOPER_DIR=/Applications/Xcode-27.0.0-RC.app/Contents/Developer
open -a /Applications/Xcode-27.0.0-RC.app coreai-kit/Examples/AudioChat/AudioChat.xcodeproj
# → Run, then pick "Qwen2.5-Omni 3B Audio" in the model picker
# agents / headless (macOS):
cd coreai-kit/Examples/AudioChat
swift run -c release audiochat-cli --model qwen2.5-omni-3b-audio --audio sample.wav --prompt "What do you hear?"
Use Xcode build 27A266a from the release's .xcode-pin; adjust the app path if your installation is named differently.
💻 Build with it — complete; the glue is kit API, copy-paste runs:
import CoreAIKit
import FoundationModels
let audio = try await KitAudioModel(catalog: "qwen2.5-omni-3b-audio")
try await audio.attach(samples: AudioFile.pcm16kMono(audioURL)) // clip → encoder buffer
let session = LanguageModelSession(model: audio)
let reply = try await session.respond(to: question)
// reply.content: what the model heard, described fully on-device
The take-home is Examples/AudioChat/Sources/QuickStart.swift
— this exact code as one typed function, no UI; the CLI is an argument shell over it, and
the GUI drives the same KitAudioModel(catalog:) behind a LanguageModelSession.
Live mic? MicRecorder (kit API) captures 16 kHz mono [Float] — attach that instead.
One clip per session; attach a new clip to ask about different audio.
Integration checklist
- SPM:
https://github.com/john-rocky/coreai-kit(exact 0.7.3) → product CoreAIKit - Info.plist:
NSMicrophoneUsageDescription— only if you record - Entitlements: none needed (macOS)
- First run downloads the model — ~5,485 MB (Mac) — then it loads from the
local cache (Application Support; progress via the
downloadProgresscallback) - Measure in Release — Debug is ~3× slower on per-token host work
What's here
Two models, run as a pair on the coreai-pipelined GPU engine:
| path | what | size |
|---|---|---|
gpu-pipelined/qwen2_5_omni_3b_thinker_int8lin_n750_s1/ | text decoder (Qwen2.5-3B int8lin, S=1) — macOS | 3.9 GB |
gpu-pipelined/qwen2_5_omni_3b_audio_encoder_fp16_k15.aimodel/ | Whisper-style audio encoder (fp16, K=15 ≈ 30 s) — both platforms | 1.2 GB |
ios/qwen2_5_omni_3b_thinker_n750_ios/ | text decoder AOT (.aimodelc, iPhone 17 Pro / h18p) | 4.5 GB |
The decoder's audio embeds ride one static-input buffer (audio_embeds [750,2048]); the prompt's
<|AUDIO|> placeholders carry extension ids vocab + slot the graph gathers. TMRoPE collapses to
1-D for audio+text, so positions are engine-native (no rope-shift inputs). iPhone uses the AOT
decoder so the 3.9 GB graph dodges the on-device JIT jetsam; the AOT weights mmap as clean pages, so
it loads comfortably (≈5.9 GB headroom after load on a 12 GB device, with the
increased-memory-limit entitlement).
Use it
The coreai-audio app
(record from the mic / choose a file / demo clip → "what do you hear?"), or
CoreAIKit:
let model = try await KitAudioModel(model: .qwen2_5Omni3B) // downloads decoder + encoder
try await model.attach(samples: pcm16kMono) // mel → encoder → static buffer
let answer = try await LanguageModelSession(model: model).respond(to: "What do you hear?")
The 16 kHz log-mel front end is Whisper-large-v3 (Accelerate/vDSP), bit-exact with the HF feature extractor (gated cos 1.0). Any clip is decoded to 16 kHz mono, ≤ ~30 s.
Conversion / numerics
Conversion code + gates:
conversion (export_qwen2_5_omni_thinker.py /
export_qwen2_5_omni_audio.py). Decoder int8lin gates top-1-exact vs the fp32 HF oracle; the
encoder static rework is cos 1.0 vs eager (GPU 0.99999); the Swift vDSP mel is cos 1.0 vs the HF
extractor. iPhone greedy matches the Mac content (white-noise → "I hear a loud hissing sound.").
License
Apache-2.0 (inherits Qwen2.5-Omni-3B). A community conversion — not affiliated with Alibaba or Apple.
More models in this format: Core AI Model Zoo — 75 models, each with the recipe that produced it.
Want a different model on-device? Open a request — free, open weights only; the export and its measured numbers get published publicly.
Fontes
1 fonte
Fontes
1 fonteVerificado 19 de out.
Artefatos de modelo
1 artefato
Artefatos de modelo
1 artefatoios/qwen2_5_omni_3b_thinker_n750_ios/qwen2_5_omni_3b_thinker_int8lin_n750_s1.h18p.aimodelc/main-h18p-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin
bin · 3,91 GB · SHA-256 db37567ba7eb…fdf6 · Hugging Face
Download fileTrechos de fonte
3 trechos
Trechos de fonte
3 trechos--- license: other license_name: qwen-research license_link: https://huggingface.co/Qwen/Qwen2.5-Omni-3B/blob/main/LICENSE base_model: Qwen/Qwen2.5-Omni-3B tags: - core-ai - coreai - audio - audio-understanding - on-device - apple - qwen2.5-omni library_name: coreai pipeline_tag: audio-text-to-text base_model_relation: quantized --- Core AI is Apple's on-device ML runtime in iOS 27 / macOS 27 and the successor to Core ML: PyTorch models are exported with Apple's `coreai-torch` (LLMs: `coreai.llm.export`) into `.aimodel` bundles that run on the GPU or the Neural Engine, e.g. Qwen3-8B 4-bit decodes at 94 tok/s on an M4 Max GPU, MLX 90 under the same protocol ([apple-silicon-llm-bench](https://github.com/john-rocky/apple-silicon-llm-bench), macOS 27 beta 26A5353q, 2026-06-11). <!-- gen-cards:devicemark begin (managed by scripts/gen-cards + tools/devicemark_row.py — edit cards.json, not this block) --> This model has no row on [DeviceMark](https://devicemark.github.io/), the on-device LLM leaderboard. <!-- gen-cards:devicemark end --> # Qwen2.5-Omni-3B Audio Understanding — Core AI [Qwen2.5-Omni-3B](https://huggingface.co/Qwen/Qwen2.5-Omni-3B)'s **Thinker** converted to Apple **Core AI** (`.aimodel` / `.aimodelc`, iOS 27 / macOS 27) for **on-device audio *understanding*** — the model describes the **sounds** it hears (events, texture, emotion, music), it is **not** a transcriber. *"I hear a loud hissing sound."* · *"…a continuous sine wave sound."* · *"…a series of beeps."* Part of the [CoreAI-Model-Zoo](https://github.com/john-rocky/coreai-model-zoo). **Device-verified on iPhone 17 Pro (A19 Pro) and M4 Max.** <!-- gen-cards:use-it begin id=qwen2.5-omni-3b-audio (managed by scripts/gen-cards — edit cards.json / QuickStart.swift, not this block) --> ## Use it **New to Core AI? [Start with CoreAIKit 0.7.3](https://github.com/john-rocky/coreai-kit#readme).** Follow its requirements and first-run steps for `qwen3-0.6b`, then open the same release's [ChatDemo](https://github.com/john-rocky/coreai-kit/tree/0.7.3/Examples/ChatDemo). The README records the tested OS/SDK and download size; model and device coverage is stated per example. ⚡ **One line** — this model is the default behind the kit's task op (`import CoreAIOps`; no session, no model plumbing, downloads on first use): ```swift let scene = try await CoreAI.describeAudio(audioURL) ``` Every op, one...
--- license: apache-2.0 base_model: Qwen/Qwen2.5-Omni-3B tags: - core-ai - coreai - audio - audio-understanding - on-device - apple - qwen2.5-omni library_name: coreai pipeline_tag: audio-text-to-text --- # Qwen2.5-Omni-3B Audio Understanding — Core AI [Qwen2.5-Omni-3B](https://huggingface.co/Qwen/Qwen2.5-Omni-3B)'s **Thinker** converted to Apple **Core AI** (`.aimodel` / `.aimodelc`, iOS 27 / macOS 27) for **on-device audio *understanding*** — the model describes the **sounds** it hears (events, texture, emotion, music), it is **not** a transcriber. *"I hear a loud hissing sound."* · *"…a continuous sine wave sound."* · *"…a series of beeps."* Part of the [CoreAI-Model-Zoo](https://github.com/john-rocky/coreai-model-zoo). **Device-verified on iPhone 17 Pro (A19 Pro) and M4 Max.** <!-- gen-cards:use-it begin id=qwen2.5-omni-3b-audio (managed by scripts/gen-cards — edit cards.json / QuickStart.swift, not this block) --> ## Use it ⚡ **One line** — this model is the default behind the kit's task op (`import CoreAIOps`; no session, no model plumbing, downloads on first use): ```swift let scene = try await CoreAI.describeAudio(audioURL) ``` Twenty ops, one shape — [Cookbook](https://github.com/john-rocky/coreai-kit/blob/main/docs/COOKBOOK.md). ▶️ **Run it (source)** — the [AudioChat runner](https://github.com/john-rocky/coreai-kit/tree/main/Examples/AudioChat) (GUI + CLI, one app for every audio-understanding model in the catalog): ```bash git clone https://github.com/john-rocky/coreai-kit open coreai-kit/Examples/AudioChat/AudioChat.xcodeproj # → Run, then pick "Qwen2.5-Omni 3B Audio" in the model picker # agents / headless (macOS): cd coreai-kit/Examples/AudioChat swift run audiochat-cli --model qwen2.5-omni-3b-audio --audio sample.wav --prompt "What do you hear?" ``` 💻 **Build with it** — complete; the glue is kit API, copy-paste runs: ```swift import CoreAIKit import FoundationModels let audio = try await KitAudioModel(catalog: "qwen2.5-omni-3b-audio") try await audio.attach(samples: AudioFile.pcm16kMono(audioURL)) // clip → encoder buffer let session = LanguageModelSession(model: audio) let reply = try await session.respond(to: question) // reply.content: what the model heard, described fully on-device ``` The take-home is [`Examples/AudioChat/Sources/QuickStart.swift`](https://github.com/john-rocky/coreai-kit/blob/main/Examples/AudioChat/S...
Source context: 2320 downloads · 2 likes · Pipeline audio-text-to-text · Library coreai · Repo mlboydaisuke/Qwen2.5-Omni-3B-Audio-CoreAI
