Omniscio

Cheap / utility LLM routing

The cheap default model Omniscio uses for its small utility AI calls such as reply chips, briefings, TTS summaries and voice intent, the user-facing pickers that override it (a global Utility AI model picker plus per-feature ones), the silent cheap-provider to Anthropic fallback and circuit breaker, and the steps to add a new model to the catalog.

On this page

What it is

The small, utility LLM calls Omniscio makes on your behalf — AI-suggested reply chips, Omni per-session briefings, TTS summaries (the short spoken readouts), voice-intent parsing and Mission Control's built-in helpers (what turns "snooze this til tomorrow" into an action) — run on a cheap, fast model rather than your main Claude session.

The default is DeepSeek V4.1 Flash on the company lane — and you can change it. It was Groq's Llama 4 Scout until 2026-10-16, when the company Groq key died; the gateway serves DeepSeek V4.1 Flash from DeepSeek's own API (since 2026-10-23), failing over to DeepInfra. A "Settings → AI Provider" panel that held a single unified dropdown was removed (2026-06-20), but the model is still yours to pick in two places: a global "Utility AI model" picker (cheapUtilityModel) that covers every cheap/background helper, and per-feature pickers for the surfaces that need their own choice. Precedence is per-feature pick > global picker > automatic (your own DeepSeek key when you have one) > the built-in Luna default — see "Configuring it". A few features deliberately pin their own model in code and ignore all of that (session titles → OpenRouter + Qwen, email pre-screen → OpenRouter's Gemini 2.5 Flash-Lite). The underlying defaultLlmProvider / defaultLlmModel / lightweightLlmApiKey settings still exist (kept for upgrade-migration safety, and lightweightLlmApiKey doubles as the multi-model council's Together-AI key slot) but are no longer user-editable.

The same unified model picker that powered that panel now drives the global "Utility AI model" override plus several per-feature settings surfaces — see "Configuring it" below.

If the cheap provider fails in a way that clearly cost nothing (bad request, 5xx, auth error, or an empty answer), Omniscio silently retries the same prompt against Anthropic Haiku so the feature still works. The exception is a timeout or a cancelled request: the provider may have already run (and billed) it before the reply was lost, so Omniscio does NOT fire a second paid call for it — it surfaces the error instead, so you're never charged twice for one request. A run of repeated timeouts still trips the circuit breaker, which then routes everything to Anthropic. This fallback is silent by design — the old amber settings-panel chip and the "open settings" toast were removed with the panel.

What is NOT routed through the utility default (deliberately):

  1. Session title generation defaults to OpenRouter + Qwen 3 32B (the same model Plain Speak uses) rather than the utility default. Llama 3.1 8B Instant on Groq broke prompt adherence on short prompts ("What project is this?" produced an editorialized "(empty response — insufficient context)" instead of a real title). Pinning to a stronger model with the same prompt fixed it. Session titles have their own Initial title model and Title refresh model pickers in the same per-feature models panel (blank = this default); they appear when the in-development Session Title Styles feature is switched on, so a stock install keeps the Qwen pin. Multimodal titles (image attachments) still route to Anthropic directly per the carveout below.
  2. The inbound email pre-screen (the classifier that inspects incoming email for prompt injection before any agent sees it) and the AI conditions evaluator inside Automation Rules. Both of those are security-critical paths that pin their model in code rather than honoring the default — a user-picked third-party provider could be tricked into a wrong verdict, and the cost of either is far higher than the savings. The AI conditions evaluator calls Anthropic Haiku directly. The inbound email pre-screen runs on OpenRouter's google/gemini-2.5-flash-lite (a fixed pin, auto-falling back to Claude Haiku 4.5 whenever OpenRouter fails or its breaker is open). Either way these are not user-configurable.

Where to find it

Configuring it

The utility model is configurable — globally and per feature. The model the cheap/background helpers run on resolves in this order, first match wins:

  1. A per-feature pick. Each feature that has its own setting overrides everything else. The per-feature pickers live in Settings → Session → Behavior → Advanced session settings, in the per-feature models panel: Briefings, Weekly Summary, Article Reader, Vault Summary, Meeting Summary, Mind Map AI, Email / Supermail summaries, Text correction, Reply suggestions, AI suggestions & titles (the separate initial/refresh title pickers appear only with the in-development Session Title Styles feature). Voice Commands, Email Summarizer and Quick Replies each also carry their own model dropdown.
  2. The global "Utility AI model" picker (cheapUtilityModel) — same Advanced fold. It controls every cheap/background helper that has no pick of its own: session titles, suggestions, summaries, digests and more. "Use automatic" (the default) clears it.
  3. Automatic: your own DeepSeek key when you have one set up (the call then bills your key, not the company lane), otherwise the built-in default.

So a reader who wants a cheaper, a stronger, or simply a different model for these background calls can pick one — the change takes effect immediately, with no restart.

Leaving everything on automatic keeps the shipped default: DeepSeek V4.1 Flash on the company lane. To change that default in a future model swap, edit DEFAULT_MODELS.deepseek (the per-provider default in model-latest.ts) or the CHEAP_UTILITY_PROVIDER constant in llm-provider-service.ts — see "How it works". Provider API keys for the few keyless providers (e.g. Together AI for the council) live under Settings → Accounts, where the same save-time key check ("Checking…" / red-reject / "Save anyway" on a network blip or on a provider rejection) applies — mechanics in secret-key-input-contract.md I9.

Every surface uses the same dropdown, reading the same catalog — every supported model (Anthropic, Groq, Together AI, DeepSeek, OpenAI, OpenRouter) grouped by provider, friendly names, a (reasoning) flag on reasoning models, and the same inline amber reasoning-model warning. A pick whose provider has no usable key degrades to the automatic default rather than breaking the call.

How it behaves

How it works

Each helper-model setting is a row (LauncherSettingRow.tsx) around the app's one model picker, the session launcher's helper chip — the same chip that picks a session's model. It reads from a single curated catalog at model-catalog.ts — every catalog entry carries a provider, an optional isReasoning flag, and an optional recommended: true marker (one per provider). The chip lists the catalog for the row's scope grouped by provider, keeps a saved unknown id visible as "Custom: <id>", and its "Use default" writes the row's recommended model back.

Each picker writes its OWN settings field via onChange — a per-feature one (e.g. jarvisBriefingModel, dailyDigestModel) or the global cheapUtilityModel. There is no longer an AiProviderSettings.tsx writing the global defaultLlmProvider / defaultLlmModel fields: those are written by no UI, chat() ignores them for the cheap utility default (it uses CHEAP_UTILITY_PROVIDER), and they persist only for upgrade-migration safety and the council's Together-AI key slot. The resolved choice — per-feature pick, else the global override, else the automatic DeepSeek swap, else the base default — is applied at the chat() chokepoint by resolveCheapUtilityModel (provider-resolution.ts).

Every routed feature calls a single function, llmProviderService.chat({ source, system, messages, accountId, ... }), defined in llm-provider-service.ts. For the cheap utility default it uses the hardcoded CHEAP_UTILITY_PROVIDER ('groq') unless the caller passes opts.provider; it still honors the optional settings.lightweightLlmApiKey override (for keyless providers like Together AI), calls the chosen provider via its OpenAI-compatible endpoint (Groq: api.groq.com/openai/v1, Together: api.together.xyz/v1, DeepSeek: api.deepseek.com/v1, OpenAI: api.openai.com/v1, OpenRouter: openrouter.ai/api/v1), and records per-call token cost via trackApiCost. Callers can pin a specific provider for one call by passing provider: 'anthropic' in the options — useful when a call can't tolerate the fallback pattern.

The model name itself is resolved by resolveModel() in the same file, which reads from a centralized DEFAULT_MODELS constant defined in model-latest.ts (re-exported through the /src/shared/types.ts barrel) — the single source of truth for per-provider defaults. Resolution priority: explicit opts.model → per-source override from GROQ_MODELS_BY_SOURCE (Groq only — e.g. voice-intent uses llama-3.1-8b-instant) → provider default from DEFAULT_MODELS. (The old user-picked settings.defaultLlmModel step was removed with the AI Provider panel.) To change a default model in a future model swap, edit DEFAULT_MODELS in model-latest.ts; to change a per-source pin, edit GROQ_MODELS_BY_SOURCE in claude-models.ts.

Title-gen is the one source that opts out of the resolver entirely. Inside ai-suggestion-service.ts callWithApiKey(), the cheap-eligible branch detects costSource === 'title-gen' and passes provider: 'openrouter' + model: 'qwen/qwen3-32b' (MODEL_QWEN3_32B_OPENROUTER) through to llmProviderService.chat(). Those explicit values short-circuit both the user's settings.defaultLlmProvider and the per-source GROQ_MODELS_BY_SOURCE lookup. Qwen 3 is a reasoning model, and the reasoning: { effort: 'low' } that chat() sends when a caller states no budget does NOT bound it on OpenRouter (on/off only there, and its busiest host, DeepInfra, ignores the parameter). So the Qwen naming calls — session titles and SMS contact names — switch thinking off themselves: Qwen's /no_think soft switch in the message plus effort: 'none' (QWEN_NO_THINK_SUFFIX in llm-egress-client.ts). Measured 2026-09-29: with thinking, a title took a median 10.5–15 s and ran past its 20 s deadline on about 1 in 6 real first messages, and the SMS-name call's 50-token budget ran out mid-thought on every real call (each then re-asked of Haiku); without it, a title takes ~1 s at half the cost and blind-judged no worse. What a cloud title call is sent: the first 2,000 and last 1,000 characters of the first message (cloudTitleExcerpt in ai-title-generation.ts; the tail keeps a request typed after a pasted log). Measured on 60 real first messages: it halved a title's cost ($0.000191 → $0.000093) for a small blind-judged quality drop (3.90 → 3.62). The on-device lane still gets the message's own first characters, the input its model was trained on. The Anthropic Haiku fallback still applies — if OpenRouter returns a non-2xx, the circuit opens and subsequent title-gen calls go to Haiku exactly like the other cheap-eligible sources. That Haiku fallback runs on the user's OWN key, so a keyless PAID user gets a company-gateway Haiku backstop instead (callTextTitleViaGateway, fired when the title-gen circuit is OPEN or the cheap call hit a definitive non-timeout failure — a cheap TIMEOUT still waits, so no double-charge — under the title-gen daily-$ cap); and if every AI path is down the final give-up derives a clean, scaffolding-stripped LOCAL title (deriveLocalTitle) rather than echoing the raw first-60 chars — see title-gen-recovery-contract.md I5/I6. A sustained cheap-provider outage (which the fallback otherwise masks) also raises a self-clearing "AI features are using a backup provider" notice — see the gateway-routing contract's "unmask" §. Multimodal title-gen (when the first message has images) bypasses this routing entirely and goes straight to Anthropic Haiku 4.5 through the company gateway lane — never OpenRouter — see "Multimodal title carveout" below.

A circuit breaker guards the cheap provider: 5 consecutive failures open the circuit, after which Omniscio fails fast to Anthropic Haiku for a 60-second cooldown (transient errors) or a 10-minute cooldown (permanent errors like HTTP 401/403, "invalid api key", or "model not found"). A single success in the half-open probe slot closes the circuit again. The circuit-break event is still broadcast on the LLM_PROVIDER_FALLBACK IPC channel as breaker instrumentation, but it now has NO renderer listener — the settings-panel chip and the permanent-error toast (useAppLlmProviderAlerts.ts) were removed with the AI Provider panel, so the cheap→Anthropic fallback is silent. The channel is baselined as emit-only in main-renderer-push-emitter-coverage.test.ts.

Double-charge protection on a timeout-after-accept. A metered call that times out or is cancelled is ambiguous — the provider may have already run and billed it before the reply was lost. Two guards keep Omniscio from paying twice for one logical call: (1) on a timeout/abort, chat() does NOT fall back to a second paid Anthropic call — it re-throws (only clearly-unbilled failures like 4xx / 5xx / empty-content fall back), routed through isTimeoutOrAbortError(); and (2) every metered create() carries an Idempotency-Key header minted fresh for that one call and reused only across that call's own SDK retries, so the SDKs' own internal retries dedupe into a single charge where the provider honors it — real on OpenAI, a forward-compatible no-op on Anthropic (no idempotency support today). The value is never a content hash of the request body. The Omniscio credits gateway claims a key per user and then refuses any repeat of it for 7 days as already processed (HTTP 409, no body), so two byte-identical requests that share a key — the same Right-now board ranked again after its verdict expires, for instance — have the second one turned down outright. The timeout still counts toward the circuit breaker. Full posture, the SDK-plumbing detail, and the cross-reference to the Google method-restriction rule live in the external-integration-retry contract, a-metered-ai-call-never-double-charges.

Reasoning-model handling (Groq). Reasoning models (gpt-oss-*, qwen-qwq-*, deepseek-r1, deepseek-reasoner, OpenAI o1/o3) split the max_tokens budget between an internal chain-of-thought and the visible content — once the budget is exhausted, the model returns empty content. Omniscio handles this in three layers so titles/suggestions still land:

  1. Cheap-provider request (Groq only): llmProviderService.callCheapProvider() always sends reasoning_effort: 'low' to keep the reasoning step short. Non-reasoning models on Groq ignore the field. Together AI, DeepSeek, OpenAI, and OpenRouter do not currently use this field via the same path.
  2. Title-gen budget: aiSuggestionService.generateSessionTitle() allocates maxTokens: 1000 (was 50) so even a chatty reasoning step still leaves comfortable headroom for the visible title.
  3. Empty-content tripwire: if the cheap provider returns an empty text despite the above, callCheapProvider() throws a synthetic 502 so the existing Anthropic Haiku fallback kicks in instead of propagating an empty string up the stack. The error names why: finish_reason, input and output tokens, and the reasoning tokens the provider reported (reasoning=unreported when it sent none). The Anthropic leg's "returned no text content" error names stop_reason and its token usage the same way, so finish_reason=length with reasoning equal to the output says the model spent the whole budget thinking.

Switching thinking off (thinking: 'off'). DeepSeek V4.1 Flash and Claude Haiku 5.5 both think by default, and their thinking counts against maxTokens. A caller that wants a short answer passes thinking: 'off', and chat() sends each provider its own switch: thinking: { type: 'disabled' } to Anthropic (direct and the gateway's Anthropic lane) and to DeepSeek's own API (api.deepseek.com, your own key). Providers with no such switch are sent nothing. Some Claude models cannot switch thinking off. Claude Opus 5.5, Sonnet 5.5, Fable 5 and Fable 5.1 keep thinking always on and refuse thinking: { type: 'disabled' } with HTTP 400. So the app names the models KNOWN to accept it instead: Haiku 5.5, Opus 5, Sonnet 5, and the older ids it still routes to (Opus 4.8, 4.7 and 4.6, Sonnet 4.6 and 4.5). Every other Claude id, including one released after this list was written, is treated as always-on: the direct Anthropic call asks for output_config: { effort: 'low' } instead, and the gateway's Anthropic lane sends no thinking field. The id is matched after stripping an anthropic/ prefix, a date suffix (-20250929) and a bracketed suffix ([1m]), so a dated snapshot or a long-context id reads as its model. The list lives in one place, canSwitchThinkingOff in model-latest.ts, and anthropicThinkingOffParams in anthropic-client.ts turns it into request fields for both Anthropic request builders. A new model is safe by default; add it to the list only once it is known to accept thinking: { type: 'disabled' } (2026-11-01: What Next failed on Opus 5.5 when the list named the always-on models instead). Known gap on the company DeepSeek lane: since 2026-10-23 the gateway serves DeepSeek from DeepSeek's own API through LiteLLM's hosted_vllm route. That route turns a top-level thinking field into reasoning_effort and never forwards it, and DeepSeek does not read the OpenRouter-style reasoning: { enabled: false } that chat() still sends there. So on that lane the switch is not honoured until the gateway forwards it. When the lane still comes back empty, the call falls back to the Anthropic backstop, which does honour it.

The picker surfaces an inline amber note when the chosen model is flagged isReasoning in the catalog (or matches the legacy regex pattern for custom ids), so the user understands why responses on those models can feel slower than non-reasoning ones.

Catalog lookups and the reasoning regex

MODEL_CATALOG is a curated list, not auto-fetched, so friendly names and descriptions stay deliberate, retired models stay out of the picker, and the pricing rows stay in lockstep. Each entry's isReasoning flag drives both the per-provider reasoning_effort mapping and the amber warning chip; recommended: true marks one entry per provider, which the council model card badges as that provider's recommended pick; speed, cost and strengths are the picker card's at-a-glance metadata. getCatalogEntry trims its input and returns undefined for an unknown id, which a caller renders as "Custom: <id>". getProviderForModel turns a persisted model id back into its provider, and resolveModelOverride uses it so an empty, whitespace or unknown override falls back to the caller's own default rather than leaving a paid call with no provider. councilMemberSpecFromModelId derives a council member's provider and label from the same entries and falls back to anthropic plus the raw id for an unknown model. getEntriesForScope('anthropic') filters to Claude models and groupByProvider keeps catalog order for the picker's <optgroup>s. getRecommendedFor takes a scope but no provider and always returns the Anthropic default — Claude Haiku — for both 'all' and 'anthropic', which each surface may override with its own ?? fallback.

supportsReasoningEffort reads the catalog's flag first and falls back to REASONING_MODEL_PATTERN for ids that are pinned but not in the catalog, because sending reasoning_effort to a non-reasoning model returns HTTP 400 on Groq. The pattern matches gpt-oss, qwen-qwq, qwen3-\d, deepseek-r1, deepseek-reasoner and word-bounded o1/o3; qwen3-\d is deliberately narrower than qwen3, so the instruct models (qwen3-coder, qwen3-vl-*-instruct) and the different qwen3.8-* family do not match. qwen/qwen3-32b, the cheap lane's classifier, is in the catalog as Qwen3 32B but carries no isReasoning flag, so the regex is what marks it as reasoning.

Which providers need a key. The cheap utility calls run on the company lane for a signed-in paid user, so they work before you paste anything. A Groq key of your own still works where a feature is pinned to Groq (voice commands): that call goes straight to Groq on your key and is never rerouted to the company lane. A saved legacy lightweightLlmApiKey likewise still goes to Groq, the vendor it was saved for. Everything else is keyless until you supply one: Together AI, OpenAI and OpenRouter have no built-in key, so a call to them returns a 401 from callCheapProvider(), which then falls through to the Anthropic fallback described above. To use them, add the key under Settings → Accounts. Your own lightweightLlmApiKey (Settings → Accounts) always takes precedence over any built-in key.

The current routed call sites are ai-suggestion-service.ts (suggestion chips — suggestions; titles also call llmProviderService.chat but with hardcoded provider: 'openrouter' + model: 'qwen/qwen3-32b' overrides as described above, so the user's setting does not apply to them), session-briefing-llm.ts (Omni), tts-service.ts (TTS readout summaries), and voice-llm-providers.ts (voice intent). The security carveouts live in email-prescreen.ts and automation-condition-evaluator.ts. The conditions evaluator instantiates the Anthropic SDK directly. The email pre-screen calls llmProviderService.chat() with a hardcoded provider: 'openrouter' + model: 'google/gemini-2.5-flash-lite' (so the user's dropdown does not apply); chat()'s own fallback then routes a failed or circuit-open call to Claude Haiku 4.5.

Omni's and Mission Control's built-in AI runs on this same lane. Everything the Mission Control built-in helpers do — the What-Next reasoner, the seasonal learn read, narrative query's AI fallback, the advisor's quick question and strategic coach, the What Changed summary, shift handoff, automation suggestions, the Team Chat bot, email field extraction, the sprint retrospective, the import column resolver and the activity feed's batch summary — funnels through one helper, pmLlmCall, which calls chat() with no provider pin. So the same key order that funds titles and the daily digest funds them: your own DeepSeek key, then the Omniscio allowance and credits, then the Anthropic backstop. Each helper keeps its own daily spend ceiling, checked locally before any call is made, so a runaway still stops on its own. pmLlmCall (and the Welcome Headlines generator) always asks for thinking: 'off'. Their budgets are small (32 to 2,400 tokens), and before 2026-10-29 a default-thinking model spent all of it thinking, so every engine came back empty and What-Next answered ai_failed. What-Next also asks for a complete answer: its prompt limits the answer's size, its 2,400-token budget is sized to those limits, and an answer an engine still cuts off moves to the next engine (2026-10-30). When an answer cannot be read, the main-process log names the reason from a closed list, with no answer text. None of it needs a Claude API key of its own (2026-11-03). A helper that wants a particular Claude tier — What-Next's answer asks for the smart tier, or worker with Cheaper AI on — passes that tier to pmLlmCall, which sends provider: 'anthropic' with routeAnthropicThroughGateway: true, so the tier is served by the same key order: your own Claude key when you have one, otherwise Omniscio credits or the company allowance on their anthropic/<model> lane. (Before this, a tier call was a direct Claude call that threw on a machine with no Claude key, and What-Next answered ai_failed.) The What-Next Investigator behind "look deeper", and the on-track "check email, meetings and chat" answer that shares it, no longer use Anthropic tool calling: pm-what-next-investigate-loop.ts runs a JSON lookup loop on chat() (Haiku through the same opt-in, responseFormat: 'json_object') in which each reply is either one {"lookup":{…}} request, run by the existing executor under the same budget, or the final findings JSON. It keeps the round cap, the 20-second budget, the per-turn timeout and its pm-what-next-investigate daily cap, checked before every turn. Every text a person wrote reaches it fenced and neutralized by asFoundData, so a hidden instruction in an item title cannot add a source or a finding. The advisor quick question sends the person's question and their board names and counts as context, scrubbed first; that egress is acknowledged (PII audit F10) because board names are what the question is about.

Image title routing — Haiku DIRECT from Anthropic via the gateway (NOT OpenRouter). When a session's first operator message carries an image attachment (PNG / JPEG / GIF / WebP), the image title call routes through llmProviderService.chat({ provider: 'anthropic', model: MODEL_HAIKU_LATEST, routeAnthropicThroughGateway: true, source: 'title-gen' }) with the image attached. The routeAnthropicThroughGateway opt-in sends it to the company gateway's /internal/v1/anthropic lane, where LiteLLM's anthropic/* route calls Anthropic's Messages API DIRECTLY on the company key (Haiku 4.5). Text titles stay on Qwen via OpenRouter; image titles bypass OpenRouter entirely. It is metered under the same title-gen gateway feature as the text titles. This replaced a bespoke gateway fetch (callMultimodalTitleApi) + a direct-Anthropic-vision fallback (callAnthropicVisionTitle) on 2026-07: the old fetch rode a SEPARATE title-gen-multimodal gateway feature that sat at a constant 402 "at capacity" cap (since the 2026-07-06 gateway routing change), so screenshot titles broke app-wide; routing to the Anthropic lane under title-gen dodges it (see internal-ai-gateway-routing-contract). Haiku 4.5 was picked over Qwen3-VL (a prior route) because Qwen tended to describe the most salient thing on screen rather than weight the operator's typed words as the primary intent; it costs ~$1/M in, $5/M out ($0.002 per titled image). The first up to 3 images on the first message pass through as Anthropic-shape {type:'image',source:{type:'base64',media_type,data}} blocks (chat()'s toOpenAiMessage converts them to OpenAI image_url blocks for the gateway's OpenAI-compatible endpoint, which LiteLLM translates back to Anthropic); each is size-capped at 5 MB, unsupported MIME types (PDFs / Office formats) are silently skipped, and text-document attachments (.md / .txt / .json / etc.) are inlined as truncated text alongside. When the gateway is unavailable (signed out / toggle off), chat() falls back to a DIRECT Anthropic call WITH the image on the user's own key — so an image-only first message still gets a real title (instead of the empty-prompt "Untitled Session" sentinel), and it is Haiku direct from Anthropic either way. The company lane's model allow-list (internal-models.ts) must list the Anthropic Haiku id or the lane 403s. On any miss the image call returns null and title-gen falls through to the text-only route; the orchestrator's retry schedule (TITLE_RETRY_DELAYS = [5_000, 45_000]) covers retries. A title-gen daily-$ backstop in aiSuggestionService.callImageTitleViaAnthropic() caps runaway image-regen spend (chat()'s per-source breaker handles transient backoff). Cost is tracked by chat() under source: 'title-gen' + provider anthropic. This lives in aiSuggestionService.callImageTitleViaAnthropic(), gated by pickUsableImages() in title-orchestrator.ts.

No minimum character threshold. In ai_title mode (the default), Omniscio generates an AI title for any non-empty first message regardless of length — a 3-word prompt like "fix the bug" still gets a meaningful AI-generated title rather than echoing the message text. Only the empty + zero-images case skips the AI call. The first_prompt mode opt-out (Settings → Sessions → Session naming) is the way to suppress AI title generation entirely; that mode uses the prompt verbatim as the title and never spends tokens.

Last verified 2026-10-06