QW miss reporting (Report missed question widget)

What it is

A "Report missed question widget" item in the 3-dot menu on agent messages. Omniscio's QuestionWidget parser turns certain agent end-of-turn shapes into a clickable answer widget (numbered options + free-text fallback) instead of letting them sit as plain markdown. The parser is rule-based and inevitably misses real questions that look answerable but don't quite match the patterns — when that happens, the user sees a regular markdown answer with no widget, the parser team has no way to know it happened, and the same shape misses forever.

This menu item closes that loop in one click. Open the 3-dot menu on any agent message you think should have rendered as a QuestionWidget but didn't, click Report missed question widget, optionally jot a note about why you expected a widget, hit Submit, and Omniscio packages the agent's raw markdown, the surrounding conversation, the active parser version, and your note, then ships the bundle as an email to the parser team. They drop it into the regression-test fixtures and the next parser revision can't miss that shape again.

This is one of two reports that route through Omniscio's unified bug-report channel (bug-report:submit, kind: 'overlay' | 'qw-miss'). The sibling flow is plain-speak-feedback.md. Both share the same BugReportDialog component, the same IPC handler, and the same dual-delivery transport — only the per-kind bundle contents and subject-line slug differ.

Where to find it

The item sits in the 3-dot menu on an agent message — the same menu that hosts the sibling "Report bad overlay" item. Clicking it opens a centered modal dialog of its own. The switch that hides the item from every agent message lives at Settings → Lab → QW miss reporting.

How it behaves

When the menu item is visible

The "Report missed question widget" menu item appears in an agent-message 3-dot menu only when ALL of the following are true:

1. The message is an agent message (the 3-dot menu only exists on agent rows — operator/user messages don't host it).
2. The global setting QW miss reporting (Settings → Lab → QW miss reporting, default on) is enabled. Flipping the setting off hides the item from every agent message live — useful if the option is noise in your menu.
3. The message is not on cooldown for the qw-miss kind. After a successful submission, the same (message id, qw-miss) pair is hidden for 5 minutes so a stray double-click can't fire a duplicate report. Cooldown is per-(message, kind) — submitting qw-miss does NOT put the sibling "Report bad overlay" item on cooldown for the same message, and vice versa.

Note what's not in this list: there's no check for whether the message contains question-shaped content. The whole point of the menu item is to flag false-negative parser misses — by definition the parser thinks there's no question, but you disagree. So the item is available on every agent message regardless of content shape; you're the judge of what looks answerable.

The cooldown is in-memory only. Refreshing the Omniscio window wipes it. The intent is "stop me from immediately re-clicking", not "remember across sessions" — five minutes is plenty to cover the email/disk write round-trip without being so long it locks the user out of legitimate re-reports.

What the dialog shows

Clicking "Report missed question widget" opens a centered modal dialog titled Report missed question widget:

• A short description paragraph: "Flag this agent message as a QuestionWidget parser miss. The full message and surrounding thread are attached automatically — leave a note if you want to tell us why you expected a widget."
• A Notes (optional) textarea, autofocused, capped at 2,000 characters, with the placeholder "Why did you expect a QuestionWidget here? (optional — leave blank to just flag it)".
• Cancel and Submit buttons in the footer. Ctrl+Enter / Cmd+Enter inside the dialog also submits (per the Omniscio useCtrlEnterSubmit cross-modal contract).

The dialog deliberately does not preview the message text. The user just saw the message in the chat — re-showing it inside the dialog adds visual noise without adding information, and the server-side bundle is the authoritative copy. The dialog's job is to capture the optional note and confirm intent.

The dialog uses the standard DialogShell primitive — Esc closes it, clicking outside closes it, and the body is keyboard-focus-trapped. Submit is fire-and-forget: clicking Submit closes the dialog immediately so the user can keep working, then ships the report in the background. A per-(message, kind) cooldown is marked optimistically at Submit-time so re-opening the menu can't double-fire; on background failure the cooldown is cleared so the user can re-flag without waiting out the 5-minute window.

What happens on Submit

The button fires an IPC call (bug-report:submit) carrying kind: 'qw-miss', the message id, and the user's notes. The main process:

1. Looks up the target message in conversation_messages, joined to its sessions row and (optionally) its projects row, filtered by is_deleted = 0 on the session and the project. If the message doesn't exist, or the session/project has been soft-deleted, the handler returns a friendly error toast ("Message not found or session archived.") and the dialog stays open so the user can copy their note.

2. Fetches up to 200 preceding messages (oldest-first) from the same session via the shared fetchPrecedingThread() helper — same code path as the overlay-feedback flow. The bundle carries the slice plus a truncated flag and the totalThreadLength so the parser team knows whether they're seeing the whole conversation or a trimmed window.

3. Builds a JSON report bundle containing: a fresh UUID, the report timestamp, the Omniscio app version, the user's notes, the session id and title, the project name, the session's pendingAction at the time of the click, the agent message id + raw content + timestamp, the preceding-messages array, the truncation/total-length pair, a lengthBucket classifier ('short' / 'medium' / 'long' computed from the agent message length), the active qwParserVersion setting ('v1' / 'v2' / 'v3'), and a checkFor: 'user-reported-qw-miss' tag the parser-team's harness uses to filter.

4. Fans the bundle out across two destinations in parallel — Resend (an email, subject [BUG: amc-qw-miss] <session title>, with the full bundle and Omniscio logs attached) and Firestore (the developer's bug-report database). Both are governed by the bugReportTransport setting (default both) and gated by the same Send bug reports & feedback switch (RT-F024) — both stop together when it's off. The amc-qw-miss slug routes Resend reports to the QuestionWidget fixture pipeline instead of the generic Omniscio bug queue. The report counts as delivered if either destination succeeds.

   Delivery (2026-05-20, ErrPort added): The report fans out across up to three destinations in parallel — Resend, Firestore, and ErrPort (the structured error dashboard) — all three gated by the one Send bug reports & feedback switch (RT-F024). The report counts as delivered if any destination succeeds. Only if every leg fails is the report kept in the <userData>/bug-reports/ outbox, so it is never silently lost, and retried later by the hourly outbox drain. (ErrPort was retired 2026-09-28; delivery is now Resend + Firestore only.)

   Very long conversations (2026-09-26): the report is fitted to what the relay accepts instead of being refused. The oldest thread messages are left out first — the report says how many it kept out of how many ("Thread: 120/200 (truncated)") — and only if that is still not enough is the one long message itself cut short, with a note saying so. A normal-sized report is sent unchanged.

5. The dialog closes the instant Submit is pressed (the cooldown is marked first, so the menu item is hidden); the IPC then resolves in the background. On send success a green Report sent — thanks! toast fires — the user has already returned to the chat by then, so the toast confirms a completed background action rather than gating a wait.

6. Only if every leg (Resend, Firestore) fails: the bundle is written to disk at <userData>/bug-reports/qw-miss-<sessionId>-<msgIdShort>-<bundleId>.json (a path-traversal-safe filename — every interpolated id is stripped of any character outside [a-zA-Z0-9._-], runs of two-or-more dots are collapsed, and parts are capped at 64 chars). The user still sees a success toast, but the IPC response carries delivery: 'disk-fallback' for telemetry. The file can be mailed manually later if the user wants to escalate. If even the disk fallback fails (read-only volume, permission error), the IPC response carries delivery: 'failed', the per-(message, kind) cooldown is cleared so the user can re-flag from the menu immediately, and a red error toast fires with the underlying error message; the bundle is lost.

The dialog is React.lazy()-loaded — its weight (DialogShell, Button, useToast, framer-motion) stays out of the entry chunk and only downloads when the user actually opens one of the bug-report menu items.

How reports get back to fixtures

The bundle is intentionally shaped to drop straight into the QuestionWidget regression-test corpus:

1. Detach the .json attachment from the report email (or pick it up from <userData>/bug-reports/ if it disk-fellback).
2. Confirm the agent message text really should have been a widget — the user's note is the hint, not the verdict. Some reports will turn out to be misjudgments and that's expected.
3. Drop the agent message body into tests/fixtures/question-widget/should-render/ as a new .md fixture following the schema documented in that directory's README.
4. Re-run the parser test suite. The new fixture fails (parser still misses), the parser team adjusts the rules, the fixture passes — and the same shape can never silently regress again.

This loop is the entire point of the feature: silent parser misses are nearly impossible to find by code review or by trawling sessions manually, but they're trivial for users to spot in real time. One click → emailed JSON → fixture drop closes the gap.

The qwParserVersion field on the bundle matters here because Omniscio ships three parser implementations side-by-side (v1 legacy cascade, v2 soak-window rewrite, v3 minimal high-precision). v3 is the parser users run today — the Settings → Lab picker that let you switch versions was removed 2026-05-31 — but the bundle still records which version was active, so a miss reported against v3 is not the same as a miss against v1, and the parser team can route the fixture to the specific version that needs the fix.

How to disable the menu item

Settings → Lab → flip QW miss reporting off. The setting is qwMissReportingEnabled in AppSettings (default true), persisted in config.json like any other setting. With the toggle off, the item disappears from every agent message's 3-dot menu live — no reload needed, the visibility is computed reactively from the settings store.

Turning the toggle off does NOT affect parser behavior — QuestionWidgets still render for messages the parser does recognize. It only hides the menu item that reports parser misses.

Telemetry

Every successful submission fires the qw_miss_report_submitted feature event with metadata { delivery } — delivery is the only allow-listed metadata key, and its value is one of 'email' | 'firestore' | 'disk-fallback' | 'failed'. 'email' / 'firestore' name exactly which leg won; 'disk-fallback' means every remote leg failed and the report landed in the outbox instead (retried later); 'failed' means even that kept-record write failed. The event surfaces under Stats → Feature usage as QW Miss Reports, with the Top breakdown showing whether reports are reaching a remote endpoint, piling up on disk after every remote fails, or hard-failing entirely (a signal the team uses to detect provider outages quickly). No raw message text, no user notes, no session ids leak into telemetry — only the delivery outcome.

Privacy

What the parser team receives in the email bundle: the agent message raw markdown, the preceding conversation messages (operator + agent + system rows alike, up to 200, oldest-first), the user's free-form notes, the active parser version, session and project names, and Omniscio's collected logs. The free-form userNotes field is the only place where unrestricted user-typed text lands in the bundle — every other field is either reproduced verbatim from the existing session transcript or a small enum (parser version, length bucket, pending action).

What it does NOT include: API keys, OAuth tokens, secrets from config.json, content from other sessions, content from soft-deleted sessions (the SQL filter excludes any session or project row with is_deleted = 1 — a flagged report against a since-archived session returns "Message not found or session archived" rather than leaking content the user thought was discarded).

The 200-message thread cap is generous because QuestionWidget shapes often depend on the framing turn before them ("Do you want me to do X or Y?" only reads as a question after the preceding context) — short threads ship in full, very long threads ship a 200-message tail and the bundle marks itself truncated: true so the parser team knows they're looking at a window.

For agents

Implementation references

For agents working in the repo:

• 3-dot menu item + dialog mount: agent-message 3-dot menu inside src/renderer/src/components/ui/MessageBubble/MessageBubble.tsx — the menu item lives next to the sibling "Report bad overlay" item; both are gated on the per-(message, kind) cooldown from src/renderer/src/stores/bug-report-cooldown-store.ts. The shared BugReportDialog is mounted lazily once bugReportKind flips to 'overlay' or 'qw-miss'.
• Shared dialog: src/renderer/src/components/ui/BugReportDialog.tsx. Notes-only, parameterized by kind: 'overlay' | 'qw-miss'; uses DialogShell with initialFocusRef={textareaRef} (the trap's 50 ms timer would otherwise pull focus to the header close X), wires useCtrlEnterSubmit scoped to the dialog body (invariant I5). Submit is fire-and-forget: calls markSubmitted(messageId, kind) optimistically, closes the dialog, runs the IPC in the background, then toasts success or — on a success: false envelope or a rejected promise — calls clearSubmitted(messageId, kind) and toasts the underlying error. A submittedRef single-fire guard absorbs key-repeat double-fires before the close completes.
• Settings toggle: in src/renderer/src/features/settings/LabSettings.tsx with data-setting-id="qw-miss-reporting-enabled"; matching index entry in src/renderer/src/features/settings/settings-search-index.ts. The AppSettings field + DEFAULT_SETTINGS entry live in src/shared/types.ts, and the matching Zod field in updateSettingsSchema is in src/shared/ipc-schemas.ts.
• IPC channel: bug-report:submit registered as IPC.BUG_REPORT_SUBMIT in src/shared/ipc-channels/index.ts. Payload schema bugReportSubmitSchema in src/shared/ipc-schemas.ts (kind: 'overlay' | 'qw-miss', messageId, userNotes).
• Bundle builder: buildQwMissReportBundle() in src/main/services/qw-miss-report-builder.ts — looks up the message + session + project (soft-delete filtered), pulls the preceding thread via the shared fetchPrecedingThread() helper in src/main/services/report-thread.ts, reads the active parser version from getSettings().qwParserVersion (falls back to 'v1' per the CLAUDE.md getSettings() gotcha), returns null when the message or session/project is missing or soft-deleted.
• Delivery transport: sendBugReportEmail() in src/main/services/email/resend-service.ts (Resend leg — subject prefix [BUG: amc-qw-miss], JSON bundle attached, Omniscio log attachments appended via collectLogAttachments() under the shared 40 MB Resend cap) + submitBugReportBundleToFirestore() (Firestore leg). Both run through the one shared chain, deliverReport in src/main/services/bug/report-delivery.ts. Kept-record fallback <userData>/bug-reports/qw-miss-<sessionId>-<msgIdShort>-<bundleId>.json written by sendReport only when nothing delivered — retried automatically by the hourly outbox drain.
• IPC handler: src/main/ipc/bug-report-handlers.ts — single handler dispatches to the per-kind builder, then hands the scrubbed bundle to sendReport({ kind: 'bundle', ... }) (report-delivery.ts), which runs every leg and writes the kept record only when nothing delivered (sanitized filename parts, resolved-path containment check), returns { success: true, data: { delivery: 'email' | 'firestore' | 'disk-fallback', correlationId? } } on the delivered/kept paths and { success: false, error, data: { delivery: 'failed' } } only when even the kept-record write fails — so the renderer surfaces a real error toast on a genuinely lost bundle rather than the cheerful success copy. Telemetry fires on every outcome regardless of envelope shape.
• Telemetry: feature id qw_miss_report_submitted in src/shared/feature-registry/index.ts with metadataAllowList: ['delivery'].

Related

The sibling report flow is plain-speak-feedback.md — same dialog, same IPC channel, different bundle shape and subject-line slug, with its menu item sitting right beside this one in the agent-message 3-dot menu. plain-speak.md covers the Plain Speak rewrite feature itself; it is not directly related to QW parsing, but the two bug-report items share that menu and the bug-report-v2 design conventions. Both route through bug-report-intake.md, the parent bug-routing system, where the [BUG: amc-qw-miss] subject prefix is the sibling slug that the QuestionWidget fixture pipeline routes off. The parser team's own tools are the other half of the loop: qw-parser-snapshots.md is the production-snapshot regression-diff workflow whose frozen rows a fresh user report complements, and qw-triage.md is the read-only npm run triage:qw scorer that mines the live database for likely silent-miss and over-fire bugs — reported bundles are the user-side complement to that automated sweep. Finally, stats.md is where the QW Miss Reports feature event shows up, with the Top metadata breakdown showing the delivery split.