Cron failure alerts (part 2)

What it is

The continuation of part 1. Part 1 covers what the card says and when one is raised; this page covers the settings that control it, its relationship to cron self-healing, and the code behind it.

Where to find it

The switches live in Settings → Notifications → Alert types. The code itself is named in the developer reference below.

How it behaves

Settings

Both settings live at Settings → Notifications, defined as optional booleans on AppSettings in src/shared/types.ts (search cronFailureAlerts). The UI is in src/renderer/src/features/settings/sections/notifications/NotificationSettings.tsx (search cron-failure-alerts-enabled). Search-index entries: cron-failure-alerts-enabled and cron-failure-alerts-always-fire in settings-search-index.ts.

Setting	Default	What it does
cronFailureAlertsEnabled	true	Master toggle. When off, both the Windows toast AND the inbox card are suppressed. Existing pending rows stay in the inbox until the user acknowledges them; future failures of any job stop alerting.
cronFailureAlertsAlwaysFire	false	Bypass Focus Mode batching for the OS toast. When on, the producer passes bypassFocusMode: true so the toast pierces Focus Mode's count/time threshold and fires immediately. Does NOT bypass silenceUntil, the muted gate, or the master off.

The "Always fire" nested toggle is gated on the master being true — toggle the master off and the nested row is hidden.

Interaction with self-healing

When self-healing is enabled per-job and a heal lands in pending or spawned state, the failure-alert producer suppresses the alert insert for the same job (Skip condition 4 above). The heal pipeline owns the failure for the duration of its lifetime. The reasoning: a heal card in the inbox already tells the user "this job failed and Omniscio is trying to fix it," and a failure card next to it would be noise telling them the same thing twice.

The opposite direction also holds: a cron.failure_alert card in the inbox does not suppress heal creation. The order in notifyJobResult is insertOrBumpFailureAlert → createHealAttemptIfEligible, so the alert insert sees pre-existing heal attempts (the ones it must defer to) but the subsequent heal-eligibility gate sees no alert at all (alerts are not on the heal pipeline's radar).

If self-healing is off for a job, every failure produces an alert card; the heal hook never fires. If self-healing is on but the heal eligibility gate rejects (e.g. 3-strike cap, system-marked failure, one-off mode), no heal lands, so no heal card — and the alert card fires normally.

The detail pane's inline Turn on auto-fix control is the user-facing bridge from "I see the alert and want to set up auto-recovery" to actually turning the feature on — it flips healingEnabled: true on the job in place (no trip to the editor) and confirms with a success toast. It's only shown when the job currently has healingEnabled === false. Separately, the Fix with AI footer button opens a Start-session dialog to fix the job by hand right now — a normal Claude session pre-filled with the error (it pauses the job while you work). It's the hands-on counterpart to automatic self-heal; the sibling escalation card's own Fix it is what still triggers a one-off manual heal (see cron-self-healing.md § "Manual Fix it").

See cron-self-healing.md for the full heal pipeline flow.

For agents

Files (for agents with repo access)

Producer (main process)

• src/main/services/cron/cron-engine-notify.ts — notifyJobResult + notifyPermanentFailure: the success/retry/permanent-failure notification arm that bumps the inbox card (insertOrBumpFailureAlert), gates the chime (showSystemNotification), and auto-dismisses on success (autoDismissOpenAlertOnSuccess). The engine keeps thin same-named delegates in cron-engine-service.ts that thread its injected emitPush; the three alert helpers themselves are free functions in cron-failure-alert.ts.
• src/main/services/notification-service.ts — fireSystemNotification(category: 'cron-failure', opts) (line 619).
• src/main/db/queries-cli-pending.ts — findOpenAlertByTarget, bumpAlertPayload, findIdempotent, insertPending, markApproved, countPendingByKind.
• src/main/db/migrations/20260715051230-resolve-orphaned-cron-failure-alerts-on-job-delete.ts — the AFTER DELETE ON cron_jobs trigger + one-time backfill that close a deleted job's open failure/heal cards (invariant I9). The CRON_JOB_DELETE handler in src/main/ipc/cron-job-handlers.ts emits CLI_PENDING_CHANGED afterward so the inbox drops the closed card live.

Schema and types

• src/shared/cli-pending-types.ts — CronFailureAlertPayload interface, CLI_PENDING_MAX_OPEN_PER_KIND_CRON_FAILURE = 5, the 'cron.failure_alert' actionKind in CliActionKind, and the per-kind status semantics docblock.
• src/shared/ipc-schemas.ts — cronFailureAlertPayloadSchema (line 4788) — Zod payload validated at insert time AND at approve-time revalidation in the dispatcher.
• src/shared/ipc-channels/index.ts — IPC.CRON_FAILURE_ALERT_CHANGED = 'cron:failure-alert-changed' push channel. Payload shape: { jobId: string, action: 'inserted' | 'updated' | 'auto-dismissed' }.
• src/shared/types.ts — cronFailureAlertsEnabled and cronFailureAlertsAlwaysFire on AppSettings (line 1389), defaults at line 2123.

Dispatcher (approve / reject)

• src/main/services/cli/cli-pending-dispatcher.ts — 'cron.failure_alert' arms in both the payload-revalidation switch (line 333) and the dispatch switch (line 681). The dispatch arm is intentionally a no-op: acknowledge is just markApproved, no downstream side effect to invoke. The dispatcher then calls markDispatched so the orphan reconciler doesn't pick it up as crashed-mid-dispatch on next boot. Reject flips the row to rejected with the supplied reason — the X-key / generic dismiss path (snoozing, from the header clock or H, never rejects; it routes through the universal snooze palette instead, which leaves the row pending but snoozed).

Renderer

• src/renderer/src/features/cli-pending/CronFailureAlertPane.tsx — inline detail pane (ApprovalPaneShell) with the action buttons; each hotkey is advertised per-button via title + aria-keyshortcuts (a hover key-cap), not a static footer hint. The inbox row preview is a generic cli-pending row carrying the failureAlertCard discriminator (see below); the whole row is the click target and routes through the inbox dispatcher (activateUnifiedItem → setActiveApproval({ kind: 'cli-pending', id })). There is no footer Dismiss button — the shared top-right Archive removes the card; Snooze dispatches the universal snooze-entity event and the pane auto-closes once the row is snoozed (subscribes useInboxSnoozeStore + isInboxItemSnoozed). Registers its N/P/O/H keys (plus S whenever the Auto-fix row is shown) via approval-pane-hotkeys.ts, consulted by the approval intercept in useKeyboardShortcuts.ts. Fix with AI opens the shared StartSessionDialog (components/inbox/StartSessionDialog.tsx) pre-filled with the job's error (launch source cron-failure-alert-start-session); the dialog's optional onLaunched fires on a successful launch, and the pane then pauses the job (toggleJob) + resolves the alert (resolveInboxApproval approve) + toasts (addToast, grandfathered/baselined). The sibling CronHealEscalationView.tsx escalation view keeps its own footer (Fix it / View details / Dismiss — no Pause, its job is already paused; no S) and still routes Fix it through useCronStore.healNow → IPC.CRON_JOB_HEAL_NOW.
• src/renderer/src/features/settings/sections/cli-pending-approval/CliPendingApprovalModal.tsx — dispatches the active cli-pending row to CronFailureAlertPane when row.actionKind === 'cron.failure_alert'; falls through to the generic JSON-payload pane otherwise.
• src/renderer/src/stores/cli-pending-approval-items.ts — wires the failureAlertCard discriminator onto each UnifiedInboxItem derived from a cron.failure_alert row, so the inbox row renderer mounts the failure-alert card preview variant. The discriminator is preview-only — activation always goes through the standard cli-pending approval path.

Feature events (declared in registry)

src/shared/feature-registry/index.ts declares three event ids:

• cron_failure_alert_inserted (allowList: jobIdHash, isUpdate, failureCount)
• cron_failure_alert_acknowledged (allowList: jobIdHash)
• cron_failure_alert_snoozed (allowList: jobIdHash)

The jobIdHash field is a stable SHA-256 prefix over the job UUID — non-PII, lets analytics correlate inserted → acknowledged → snoozed without persisting raw ids.

Tests

• tests/unit/stores/session-navigation-cron-failure-alert.test.ts and tests/unit/shared/cron-failure-alert-payload.test.ts — the card's navigation and payload shape.
• tests/unit/services/cron-engine-alert-insert.test.ts — insertOrBumpFailureAlert skip-condition matrix.
• tests/unit/services/cron-engine-alert-dismiss.test.ts — autoDismissOpenAlertOnSuccess push-on-update gating.
• tests/unit/services/cli-pending-dispatcher-cron-failure.test.ts — dispatcher's no-op approve arm + reject-with-reason snooze path.
• tests/unit/db/queries/queries-cli-pending-alert.test.ts — findOpenAlertByTarget / bumpAlertPayload / findIdempotent / markApproved SQL behavior.
• tests/unit/db/migration-20260715051230-resolve-orphaned-cron-failure-alerts.test.ts — the resolve-on-delete trigger closes the deleted job's cards (and spares other jobs / non-cron kinds / null targets), the one-time backfill, missing-table guards, and idempotency. The CLI_PENDING_CHANGED emit is asserted in tests/unit/cron-job-handlers.test.ts.
• tests/unit/features/inbox/inbox-cron-failure-visibility.test.tsx — inbox row visibility under various pending-row states.
• tests/unit/features/cli-pending/CronFailureAlertPane.test.tsx — the footer actions (Fix with AI opens the pre-filled StartSessionDialog; launching it pauses via toggleJob(id, false) + resolves (approve) + toasts; cancelling the dialog does neither; Fix disabled when healPending; Pause → toggleJob(id, false) + toast; View job), parse-failure fallback, recent-runs strip, heal-pending banner, row-id swap reset, the Auto-fix toggle (reflects healingEnabled, flips BOTH ways, honest ON/OFF state), plus: the footer hotkeys advertised on the buttons (no static hint line) with Fix on N, h/n/p/o/s hotkey registration (S registered whenever the Auto-fix row shows — including when healing is already on — and omitted while a fix runs), unmount clears the registry, auto-close-on-snooze, the honest Auto-fix state (no false "handled" check, row hidden while a heal is in flight), the truthful recent-run summary (summarizeRecentFailures), the error empty-state, and the Job field surfacing the full humanized job name (slug → Title Case with the raw slug on hover; a plain name verbatim).
• tests/unit/hooks/approval-pane-hotkeys.test.ts — the pane-hotkey registry (set/get/clear, case-insensitive, last-write-wins).
• tests/unit/hooks/useKeyboardShortcuts-approval.test.ts — the approval intercept: Enter/X plus the pane-registered keys fire, H wins over snoozeSession, guards (input/repeat/modifier), cleared-registry no-op.
• tests/unit/stores/cli-pending-store-cron-failure.test.ts — cliPendingSelectItems decoration of failureAlertCard.
• tests/unit/stores/session-navigation-cron-failure-alert.test.ts — failure-alert rows route through setActiveApproval({ kind: 'cli-pending', id }) like every other cli-pending row (single activation path shared by click + arrow-key + swipe + archive auto-advance).
• tests/unit/shared/cli-pending-types-cron-failure.test.ts — payload type narrowing.
• tests/unit/shared/feature-registry-cron-failure.test.ts — registry shape for the three event ids.

Where to look when it goes wrong

• electron-log. The producer logs three notable lines under [cron-engine]:
  • Job "<name>" failed permanently: <error> — every final failure logs this once. Missing this means the failure didn't reach notifyJobResult at all (engine crash, mid-flight tick, etc.).
  • cron.failure_alert per-kind cap (5) reached; skipping insert for job <id> — five+ pending alert rows means the queue is full. Dismiss or snooze older rows.
  • cron.failure_alert daily key already used for job <id> (status=<status>); skipping re-insert — the daily idempotency key is occupied. Status will be rejected for a previously-snoozed row or approved for a previously-acknowledged one. The card won't re-fire until tomorrow's date.
  • insertOrBumpFailureAlert failed <error> — the producer's outer catch fired. Should be very rare — only DB-corruption / SQLite-died conditions reach this branch. The engine tick continues regardless.
• cli_pending_actions table. Filter WHERE action_kind = 'cron.failure_alert' to see every alert row. target_id is the failing job's id; payload_json is the CronFailureAlertPayload JSON; status tells you pending (visible in inbox), approved (acknowledged or auto-dismissed), or rejected (snoozed).
• AMC_INSTANCE_ID. If alerts mysteriously aren't firing, check that you're running the default install — npm run dev without env overrides — and not a sandbox or e2e instance, which suppress the entire pipeline.

Related

• Cron failure alerts (part 1) — what the card says and when one is raised.
• cron-self-healing.md — the pipeline that can repair the job this alert is about.
• cli-pending-actions.md — the inbox queue these cards share.