Bake-Off (fan-out)
Run the same prompt against several models at once, side by side, and keep the answer that wins — how to open a Bake-Off, save a setup as a preset, let contestants see each other's work, and what stops it running away with your spend.
On this page
What it is
Name: shown in the UI as Bake-Off. The internal/code name stays fan-out (setting
fanoutEnabled, IPCSESSION_FANOUT, control-server route/fanout, thefanoutfeature id) — renaming the label never renames the identifiers.
Status: shipped — visible to everyone. Bake-Off ships in the Prompt Tools sidebar group (its Bake-Off row) and via the command palette; the
fanoutfeature is nowstatus: 'shipped', so the oldfanoutEnabledLab toggle no longer gates it. This page is the authoritative reference for how it works.
Bake-Off lets you type one prompt once and launch it as many separate sessions at the same time — each session running in a project you choose, on a harness/model setup you choose. It's the "run this same task across everything and compare" launcher: the same prompt against Opus vs. Sonnet vs. Codex, or against three different repos, or any mix.
You don't build a list of individual sessions row-by-row. Instead you fill in two sections and Bake-Off launches the cross-product:
- Projects / repos — a multi-select dropdown of the projects a session can start in (inbox-style views such as Gmail or Settings are not listed — a session cannot run there). Pick as many as you want. The project you are in is ticked for you when Bake-Off opens: the session's own project when you open it from a session, otherwise the project you are viewing.
- Harness & model setups — a list of setups. Each setup is the exact same configuration bar you get on a new session (Harness · Provider · Model · Thinking level · MCP servers · Local/Cloud), reused as a shared module so there is only one thing to maintain, not two. Every setup card shows its whole bar open, right in the card — you see each setup's harness, provider and model at a glance without pressing anything, so two setups that differ only in model look different. Add as many setups as you want: Add setup stays pinned at the top of the list while you scroll, and the Duplicate button on every setup inserts an exact copy right below it (then change one field to compare). The Local/Cloud chip shows on every setup from the start, before you pick a project, whenever Cloud sessions is turned on; a setup set to Cloud runs all of its sessions in the cloud instead of on your computer. A saved SSH computer is not offered here, because a Bake-Off launch cannot run on one.
It then launches one session per (project × setup) pair. A live line shows the math — "2 projects × 3 setups = 6 sessions" — and the Launch button carries the count. Ctrl+Enter (Cmd+Enter on Mac) launches too, even while you are still typing in the prompt box. If a setup leaves a field on default, the session inherits the project/global default exactly like a normal new session.
Each session is named Bake Off #<n> [<model>]: <title> — or Swarm #<n> [<model>]: <title>
with the swarm switch on. Every session in the group is numbered by its place (#1, #2…), the
same number its briefing gives it, so rows never look identical even when they run the same model; a
launch that starts only one session has no number. The number and the model's display name
(Claude Opus 5.5, GPT-5.5…) come first so you can tell the sessions apart in the sidebar even
when the name is truncated. The model is the one the session will actually run — a setup left on
"default" is named after the default it resolves to (your saved default for that engine, the route you
pinned it to, or the engine's own), never after the engine; only an engine with no model the app can
name shows the engine's name. <title> is a real AI-generated title of your prompt — produced by the
same titler every other session uses, not the raw prompt text (so a prompt that opens with a URL no
longer becomes a URL "name"). The title is generated once per launch and shared across all the
variants (they run the same prompt); a name you type into Bake-Off replaces the <title> half.
Launching is instant (fire-and-forget). Clicking Launch closes Bake-Off immediately and runs the launch in the background — you're never held on a "Launching…" spinner. A summary toast confirms the result once it settles. If the launch fails — the whole batch errored, or every target was skipped so nothing started — an error toast appears with a Try again link that reopens Bake-Off (a fresh dialog) so you can adjust and relaunch. (Above 10 sessions it still asks you to confirm before it dismisses.)
Where to find it
How you open it
Three ways, all opening the same dialog described above:
- New Bake-Off… in a session's ⋯ menu (the Session group) — opens with that session's project already ticked. Main window only: a popped-out session window cannot show the dialog, so the item is hidden there.
- The Bake-Off row in the Prompt Tools sidebar group (an action row — clicking it opens the dialog rather than switching projects).
- The command-palette action "Bake-Off…" (Quick Launch / command palette).
How it behaves
Presets (save a setup and reload it)
Bake-Off can save your configuration as a named preset so you can rebuild a comparison in one click instead of re-picking projects and setups every time. The Presets bar sits at the top of the dialog:
- Save — names the current setup (its projects, its harness/model setups, and — if you leave the "Include prompt" box checked — the prompt) and stores it. Saving under a name that already exists asks you to confirm an overwrite. Pressing Enter or Ctrl+Enter in the preset-name box saves the preset — it never presses Launch, even if you press it again while the save is still finishing.
- Load a preset… — pick a saved preset from the dropdown to drop it back into the dialog. It restores your projects and setups; it fills the prompt only if that preset saved one, so a "contestants-only" preset (saved with the prompt box unchecked) never wipes a prompt you are in the middle of writing.
- Delete — removes the loaded preset after a confirmation, with an Undo in the toast.
Presets are stored in your app settings (the same place your other preferences live), so they persist across restarts. A preset that points at a project you have since deleted still loads — the missing project is simply dropped. A preset that names a model or harness that has since been retired also loads fine; that one setup is just skipped-with-a-reason at launch, exactly like any other bad target.
Contestants who know about each other (optional)
By default the contestants are strangers: each one gets your prompt and nothing else, and none of them knows the others exist. That is usually what you want — the whole point of a bake-off is comparing independent attempts.
The "Let the contestants know about each other" toggle, under the harness setups, changes that. It is off by default, and turning it on prefixes every contestant's first prompt with a short briefing — sent to the model, never shown in your chat, which still displays exactly what you typed.
Each contestant is told:
- that it is one of N sessions running the same task at the same time, and which one it is;
- every other contestant — its model (the same label the sidebar shows), its project, and its real session id;
- the exact call to message one:
POST /sessions/<their id>/peer-message, which token file to authenticate with, and which header to send.
It never tells them what to do with that. The copy states who is there and how the channel works and stops — no "collaborate", no "stay independent". That is deliberate: a nudge in either direction would quietly decide the outcome of the comparison you launched. What the contestants do with the channel is up to them.
The ids are real, not a lookup hint. Bake-Off mints every contestant's session id before it starts launching, so contestant 1 can be told contestant 3's id even though contestant 3 does not exist yet. Without that, no contestant could ever know about the ones created after it.
Details worth knowing:
- A skipped target never appears in anyone's roster. Validation runs first, so if a project was deleted the survivors are told there are two contestants, not three.
- One contestant gets nothing. If only a single session survives, there is no "you are 1 of 1" preamble — the prompt is untouched.
- A cloud contestant gets the roster but no messaging recipe. It runs on a remote VM that cannot reach the control server on your machine, so it is told that plainly instead of being handed an address that would fail. It still appears in everyone else's roster and can still be messaged by them — delivery happens inside Omniscio, not over the network to the VM.
- A contestant that failed to start refuses messages, and the briefing says so, so a peer does not read a refusal as its own mistake.
- Presets remember the toggle. A preset saved before this existed loads with it off.
Swarm mode — the sessions work it out together as teammates (optional)
The "Work together as a swarm" switch, right under "Let the contestants know about each other", turns the bake-off from a comparison into a team. It is off by default, and turning it on also turns on "know about each other" and locks it there, because a swarm needs every contestant to know who the others are. Closing the Bake-Off window switches it back off, so your next ordinary bake-off is never a swarm by accident.
When you launch with it on:
- They are teammates, not contestants. Their sessions are named
Swarm #<n> [<model>], and their briefing calls each one a teammate, because they are working together rather than competing. - The teammates share one private message board. It is created before any of them starts, and only they are on it — anyone else asking for it is told it does not exist. You can watch it from the Boards panel, where it is listed as a group.
- Each teammate hears nothing from the board until it has posted. The app holds every board update back from a teammate until its own first report is in, so the first round is truly independent — nobody anchors on a fast teammate's early headlines. After that, its first update names everything it was held back from.
- Each one gets a working method, on top of the roster:
- Do the whole task alone first, as deeply as if it were the only one on it — never splitting the work or skipping a part because a teammate might cover it.
- Post a complete first report to the board before reading anyone else's.
- A post wakes every other teammate that has finished its turn, so there is no need to message them about it. Only a teammate running on a cloud computer cannot see the board, so it gets a short message after each post instead — but only once every teammate on your computer has posted its first report.
- Read the board, compare, challenge, merge the better ideas, and post again — only when there is something new, never just to agree.
- Keep going, with no limit on rounds, until the group agrees your goal is fully met. Nobody agrees until every teammate on your computer has posted its first report. If the others are done and someone still hasn't posted, one teammate sends each missing one a single, unhurried message; if that is refused (it failed to start or stopped), the group goes on without it.
- The app appoints the reporter — Teammate 1, the first contestant that launches — and every teammate is told who it is. The group never chooses one. When it agrees the goal is met, the teammate whose post settles it wakes the reporter with one message; after that nobody posts or messages again.
- The reporter writes the full answer to you in its own reply. Every other teammate ends with one line pointing to it and then archives itself.
- Only the reporter's answer reaches your inbox. The reporter is never held back. While the swarm is still working, every other teammate's in-between "status" turns stay out of your inbox, badge and chime — for every engine, Claude and Codex alike — and they still get each other's messages and board notices. When the swarm settles, the reporter lands in your inbox with the normal notification and the other teammates are filed away quietly instead of arriving beside it.
- You can always see who the lead is. The reporter carries a Lead badge in the swarm's Members list, and its title is the one the answer comes from.
- Nothing is left hanging. If a whole swarm goes quiet for 10 minutes with nobody working, the reporter is put in your inbox and the rest are filed away; if the reporter itself never answered, the newest finished teammate surfaces instead so the work is never lost. A real question or a permission request from any teammate always reaches you straight away.
- A message launched as "Swarm board ready" confirms the board was set up.
- The swarm is ONE row in the sidebar, like an Overseer. While any teammate is still working (or resting between turns), the project's Overseers section shows a single row named after the task, with a count, and every teammate tucked under it — none of them also appears as its own row in Live, Pinned or Needs You. The row turns amber only when a teammate really needs you (the reporter's final answer, a question, a failure). A teammate that failed or signed off drops out of the group and shows in its normal place. When nobody is in play any more, the row goes away.
- Routine updates are grey, not amber. A teammate (or a crew member) resting on a routine update draws a calm grey dot labelled "Routine update", so it never looks like something waiting on you.
- Opening a teammate shows the swarm's chrome. A Swarm · <task> row sits under the session title with a Members menu (every teammate, its status, and how many have signed off — click one to open it) and a Board button (<kbd>Alt</kbd>+<kbd>Ctrl</kbd>+<kbd>Shift</kbd>+<kbd>B</kbd>, rebindable) that shows the swarm's message board right there, read-only. The header and board still work after the swarm has finished, so you can read how the team got to the answer.
- Swarm rows are not in the Overseers hub itself — the sidebar row and the teammate header are the way in. Dragging a swarm row or putting it in a folder never saves it into your arrangement.
- Every swarm you ever ran lives in the Swarms panel. The Swarms row in the Agent Tools group (directly under Overseers, boxes icon) lists all of them, newest first — finished and archived ones included — with each swarm's agents, the model that ran each and a click into any of their sessions. See Swarms.
Unlike plain "know about each other", these instructions are deliberately directive — you asked for a team, so they are told to act like one. That only applies with the swarm switch on.
Things worth knowing:
- Each new post can wake every other contestant, and each wake is a paid turn. "Post only when there is something new" is what keeps it from looping; there is no hard cap, by your choice.
- Large swarms are paced, not dropped. A session can send about 120 messages an hour, so a 20-contestant swarm can hit that. Over the limit, a wake-up message is held and delivered late rather than lost.
- A contestant that failed to start never holds the group up — only contestants that have posted count toward agreement, and only one of them can be the reporter.
- A cloud contestant can't reach the board from its remote machine. It is told so, works on its own, and can still be messaged by the others. Its turns are never held back from your inbox.
- Typing straight into a non-reporter while the others are still working: its reply shows in that open chat, but not in your inbox.
- If the board can't be created, nothing launches — you get the usual error with Try again, and nothing has been spent.
- Presets remember the swarm switch too; an older preset loads with it off.
Not to be confused with Swarms in the Overseers hub (swarm.md) — there, a lead agent splits one goal into different jobs for a team of workers. A Bake-Off swarm is several agents attacking the SAME task and converging on one answer.
Guardrails (why it can't hurt you)
- Capped at 20 sessions per launch. Over the cap, the Launch button is disabled with a hint — it never silently drops targets. Above 10, it asks you to confirm first ("This starts N sessions").
- It can't freeze your machine. Every fan-out spawn is a background spawn and is NOT marked user-initiated, so it flows through Omniscio's existing session-spawn pacer (the same throttle that protects the box from a burst of respawns). A fan-out is a controlled, paced burst — not an un-throttled storm. Session count is never the cause of a freeze; the pacer serializes the launches.
- Bad targets are skipped, not fatal. If a target's project is missing, isn't a real spawnable folder, or names a model a harness no longer offers, that one target is skipped with a reason and the rest still launch. You get a "launched X, skipped Y" summary toast.
- A paced target is queued, not skipped. When the box is already busy the spawn pacer may accept a target and hold it for a moment instead of refusing it. That target is queued — it is coming, and the summary says so — rather than reported as a failure or silently dropped.
- MCP applies to the first turn. Because a fan-out session's whole point is the one first turn, any per-setup MCP-server override is stamped on the session row before the first spawn (not after), so it takes effect immediately.
Related
The provider pages — Claude providers, Gemini, DeepSeek and the rest — explain which models a Bake-Off can put in the ring and what each costs.
Last verified 2026-10-23