Introducing AnySpeechRecognizer and AnyDecisionModel: Open Models for Speech and Decisions on Apple Platforms

A developer has published two open Swift packages aimed at Apple platforms: AnySpeechRecognizer, starting at version 0.1.0, and AnyDecisionModel, at 0.6.0. They extend the pattern set by AnyLanguageModel, which offers one Swift API with a choice of backend model. Both packages can pull open models from the Hugging Face Hub and run them on device, according to the announcement. They require Swift 6.2 or later and macOS 26 or iOS 26 or later.
AnySpeechRecognizer wraps Apple's SpeechAnalyzer API, introduced in macOS 26 and iOS 26. In Apple's framework you add modules such as SpeechTranscriber and SpeechDetector to an analyzer, feed it a file or a stream of audio buffers, and read results from async sequences; AssetInventory handles model installation so the app controls download timing. The new package mirrors SpeechAnalyzer, SpeechTranscriber, DictationTranscriber, SpeechDetector and AssetInventory, with Apple's engine as the default. Swapping the engine can be as simple as changing the import, or changing only how the transcriber is constructed.
Three Parakeet variants are offered. Parakeet v3 is NVIDIA's Parakeet TDT 0.6B v3, which recognizes 25 European languages and detects which one it is hearing, so no locale is passed. Parakeet Ultra is Moondream's post-trained version of v3, reported as more accurate in most of the publisher's measurements. Parakeet Unified is NVIDIA's English-only model. All run on the package's own Core ML runtime, supporting file and live input, volatile results, word timing and word confidence. Alibaba's Qwen3-ASR comes in 0.6B and 1.7B sizes, covers 30 languages and 22 Chinese dialects including Mandarin, Cantonese, Japanese and Korean, and detects language automatically or accepts one explicitly. It returns one final result per segment of up to 40 seconds, with no volatile results, word timing or confidence, runs on the GPU, and is slower than Parakeet, making it better suited to files than live audio.
Downloads are explicit: roughly 600 MB per Parakeet model plus about 590 MB for a live encoder, 0.9 to 2.4 GB for Qwen3-ASR, and about 250 MB for Sortformer. Files land in the standard Hugging Face cache, so an unsandboxed Mac app shares them with Python and other Hub tools. Each converted Core ML repository records its source checkpoint and per-file checksums, and the package pins a fixed revision. The Parakeet repos add an 8-second encoder for live input that cuts live analysis time by about a third at the cost of somewhat higher word error rate. A new engine requires implementing SpeechRecognitionModel and SpeechRecognitionExecutor.
Speaker diarization is an addition Apple's framework lacks. SpeakerDiarizer runs NVIDIA's Streaming Sortformer on Core ML alongside an Apple or Parakeet transcriber on the same timeline, tracking up to four speakers even when they overlap. A SpeakerTimeline attributes each timed word to a speaker for per-speaker transcripts. Sortformer v2 is the default; v2.1 was trained with more meeting data and measured more accurate on meetings.
Performance was measured with a hear command-line tool on an M4 Max Mac Studio running macOS 27.0.1, on LibriSpeech files of 3 to 8 minutes. Parakeet transcribes an hour of audio in about 20 to 40 seconds from a file, or about 1.5 to 2.5 minutes live, using roughly 100 MB of memory after Core ML compiles the model. Qwen3-ASR is much slower and needs a few gigabytes. On AMI meeting recordings with overlapping speech, every tested engine exceeded 23% word error rate. The publisher notes the numbers come from one machine and a few datasets, that testing so far covered only macOS 27 and not iPhone, and that further engines are planned, including Core AI on macOS 27 and iOS 27.
AnyDecisionModel targets a different problem: asking yes-or-no, multiple-choice and rating-scale questions with typed results and probabilities. It maps a question onto a model's probability distribution rather than parsing generated text, using a local model via MLX, Core AI or llama.cpp, or a hosted decision model from Jev or Ollama. In Swift, a DecisionSession holds one state, text or JSON, and exposes probability(of:) returning a Double, choice(_:from:) returning a Choice with a probability per option and a confidence value, and score(_:levels:) returning a Score with an expected level and per-level probabilities. Enums conforming to Choosable give exhaustive switches. Questions are independent, so one answer never feeds a later question; batches go through decide(_:). For next-token backends, each answer gets a one-token label and the probabilities are read from a single forward pass.
A sample macOS app, Emoji Magnet, asks one yes-or-no question per emoji across 343 emoji as the user types, with each asked against the typed phrase. The hosted Jev backend answers all 343 in one request, settling in about half a second. Without an API key, Qwen3 4B runs locally via MLX with prefix caching; 16 questions per forward pass take about 0.45 seconds on an M4 Max, and a full sweep takes about 10 seconds with results appearing per group.
Cloudflare's Clef models show a trained-decision-head approach: a state plus a schema of typed questions yields probabilities across all options in one forward pass. Clef-Flash, the 9B variant, runs through MLXHeadDecisionModel; the 4-bit build is a 6.2 GB download and its model card recommends at least 16 GB of memory. It accepts only text and JSON states in this backend, though Clef itself can read images and video. The package adds calibration (a temperature on all answers plus a bias on yes-or-no) and rotationDebiasing, which averages probabilities over rotations of the option order to counter position bias, at the cost of one prompt evaluation per rotation. Entropy-based confidence measures how concentrated the probabilities are, not correctness. There are six backends; next-token ones cap choices at 26 options and scores at 10 levels, while decision heads have no such limit. CoreAI currently cannot be combined with AnySpeechRecognizer because coreai-kit 0.7.3 requires swift-huggingface below 1.0.
Why it matters: Apple developers who want speech or classification without hand-rolling model plumbing get drop-in Swift APIs, and the diarization module covers a gap Apple's framework leaves open. For product teams, probability-typed decisions are easier to threshold and calibrate than parsed LLM text, which matters for triage and routing features. The noted limits — macOS-only testing so far, no GPU-class speed for Qwen3-ASR, memory needs for Clef, and a package dependency conflict — are worth checking against a real app before committing.
Based on reporting from the original publisher. Visit the source for full context and later updates.