--- title: Cheap / utility LLM routing --- # Cheap / utility LLM routing ## What it is The small, utility LLM calls Omniscio makes on your behalf — AI-suggested reply chips, Omni per-session briefings, TTS summaries (the short spoken readouts), and voice-intent parsing (what turns "snooze this til tomorrow" into an action) — run on a cheap, fast model rather than your main Claude session. **That cheap model is hardcoded to Groq's Llama 4 Scout 17B — there is no user setting for it.** A "Settings → AI Provider" panel used to let you pick it from a unified model dropdown; it was removed (2026-06-20) because the choice added surface area without real value — the default served virtually everyone, and every feature that genuinely needs a different model already pins its own in code (Omni → Anthropic, session titles → OpenRouter + Qwen, email pre-screen → OpenRouter's Gemini 2.5 Flash-Lite). The underlying `defaultLlmProvider` / `defaultLlmModel` / `lightweightLlmApiKey` settings still exist (kept for upgrade-migration safety, and `lightweightLlmApiKey` doubles as the multi-model council's Together-AI key slot) but are no longer user-editable. The same **unified model picker** that powered that panel still drives four OTHER settings surfaces that DO let you choose a model — see "Configuring it" below. If Groq fails in a way that clearly cost nothing (bad request, 5xx, auth error, or an empty answer), Omniscio silently retries the same prompt against Anthropic Haiku so the feature still works. The exception is a **timeout or a cancelled request**: the provider may have already run (and billed) it before the reply was lost, so Omniscio does NOT fire a second paid call for it — it surfaces the error instead, so you're never charged twice for one request. A run of repeated timeouts still trips the circuit breaker, which then routes everything to Anthropic. **This fallback is silent by design** — the old amber settings-panel chip and the "open settings" toast were removed with the panel. **What is NOT routed through the hardcoded default (deliberately):** 1. **Session title generation** is hardcoded to OpenRouter + Qwen 3 32B (the same model [Plain Speak](plain-speak.md) uses) regardless of the default. Llama 3.1 8B Instant on Groq broke prompt adherence on short prompts ("What project is this?" produced an editorialized "(empty response — insufficient context)" instead of a real title). Pinning to a stronger model with the same prompt fixed it. There's no settings UI to change this — it's the same default for everyone. Multimodal titles (image attachments) still route to Anthropic directly per the carveout below. 2. **The inbound email pre-screen** (the classifier that inspects incoming email for prompt injection before any agent sees it) and **the AI conditions evaluator** inside Automation Rules. Both of those are security-critical paths that pin their model in code rather than honoring the default — a user-picked third-party provider could be tricked into a wrong verdict, and the cost of either is far higher than the savings. The AI conditions evaluator calls Anthropic Haiku directly. The inbound email pre-screen runs on OpenRouter's `google/gemini-2.5-flash-lite` (a fixed pin, auto-falling back to Claude Haiku 4.5 whenever OpenRouter fails or its breaker is open). Either way these are not user-configurable. ## Where to find it ### Configuring it **There is nothing to configure for the utility calls — the cheap model is hardcoded.** To change it in a future model swap, edit `DEFAULT_MODELS.groq` (the per-provider default in [/src/shared/model-latest.ts](/src/shared/model-latest.ts)) or the `CHEAP_UTILITY_PROVIDER` constant in [/src/main/services/llm-provider-service.ts](/src/main/services/llm-provider-service.ts) — see "How it works". Provider API keys for the few keyless providers (e.g. Together AI for the council) live under **Settings → Accounts**, where the same save-time key check ("Checking…" / red-reject / **"Save anyway"** on a network blip or on a provider rejection) applies — mechanics in `secret-key-input-contract.md` I9. The **unified model picker** is still wired into four settings surfaces that DO let you pick a model, all reading from the same catalog: **Voice Commands → Omni briefing model**, **Catch-Up Card settings → summary model**, **Email Summarizer settings → default model**, and **Quick Replies → AI suggestion model**. Each surface picks its own recommended fallback (e.g. Sonnet for catch-up summaries) but uses the same dropdown — every supported model (Anthropic, Groq, Together AI, DeepSeek, OpenAI, OpenRouter) grouped by provider, friendly names, a `(reasoning)` flag on reasoning models, and the same inline amber reasoning-model warning. These pickers are unaffected by the AI Provider panel's removal. ## How it behaves ### How it works The unified picker is the React component [/src/renderer/src/components/ui/ModelPicker.tsx](/src/renderer/src/components/ui/ModelPicker.tsx). It reads from a single curated catalog at [/src/shared/model-catalog.ts](/src/shared/model-catalog.ts) — every catalog entry carries a `provider`, an optional `isReasoning` flag, and an optional `recommended: true` marker (one per provider). The picker renders the standardized ``/`