Judged checks

What it is

A crew promises outcomes, and each promise says how it will be proven. Most are proven by the app reading one of its own numbers. Judged checks are for the promises that cannot be: the goal is real but words, not digits.

For one of those, the crew's lead writes a rubric when the promise is declared — three to five criteria in plain words, what a five looks like for each, and the bar the promise has to reach. Once the mission has declared the number it is judged on, the app runs a judge: its own session, on an AI that is not the crew's own lead, which reads the real work and scores it against that rubric. The score goes into the mission's history beside every other reading, so a judged mission is read exactly like a numeric one — trend, red run and all.

Before this existed a crew had two dishonest options for such a goal: invent a number, or write "unprovable" and be taken at its word. Neither is checked by anybody.

Where to find it

• The crew's Dashboard, in the Overseers hub. A promise carrying a rubric shows as Judged beside its status, and a Score history section shows what the mission is judged on, every reading of it, and who produced each one.
• One button — Overrule — sits on any score a judge produced.
• Nothing in your inbox. A score, and a promise going red, show on the crew's Dashboard and in the ladder's own cards; judging never sends you a message of its own.

How it behaves

• The judge is independent. It runs on an AI engine that differs from the crew lead's — a different provider, or a different model on the same one. If the only reachable engine is the lead's own, no judge runs and the history says so, because a lead marking its own paper is not a check.
• It is read-only. The judge is the same kind of locked session the app already runs for a crew's memory scribe and its period summaries: reading, searching and fetching tools only, no writing, no access to your keys, and hidden from your session list.
• It runs on the mission's own cycle, and only from the app. A judged score is taken once per cycle of the number the mission declared — a number read daily is judged daily. No session can start a judge, so a lead that did not like its score cannot buy another one, and the cost is bounded by the mission's own declared cycle and the judge's own hourly floor rather than by how badly the mission is doing.
• It reads the real work. Its brief carries the promise's own words and its own history, and it is seated in the crew's project folder so it can open the actual files. It never scores a summary the lead wrote for it. If the work is not there to read, it is asked to say so rather than fill the gap.
• A score never mixes with a number. A score sits in the same history as the app's own numbers, but on its own scale: the mission's trend, red run and staleness read the NUMBER's readings, and a promise's scores are read as their own line, turning red when they fall below their bar. A passing score can never hide a failing number, and a lead cannot clear a judged red by posting a number on top of it.
• The judge's rhythm is not the lead's to set. The judge runs no more often than once an hour, whatever cycle the mission declares, so how often the app spends on it can never be turned up by the crew being judged.
• You can overrule it, and the score you overruled survives. One click on the Dashboard records your own score for that reading, at the bar. The judge's score is not deleted: it stays in the history, so anyone reading it can see the judge said one thing and you said another, and when.
• A score that could not be taken says so. If the judge could not run, or answered something that could not be read against the rubric, the history shows Not scored with the plain reason where the score would have been, and no reading is written. A missing score never reads as a pass.
• Changing a rubric applies forward only. A stored score keeps the bar it was taken against, so a lead cannot erase a red run by moving the goalposts afterwards.
• What it costs: one judge session per crew per cycle (never more often than hourly), and only for a crew that has declared a success number and has an open judged promise. A crew with several judged promises has them scored in turn rather than one starving the rest, and a crew that declares no number is scored by nobody and costs nothing.

Related

• The number a mission is judged on, its cycle and all its readings — Agent Crews.
• What the app does when a number stays red — the ladder — Agent Crews — part 2.
• The crew-facing routes and settings — Agent Crews — for agents (part 3).
• How a lead is scored by the app — The lead scorecard.
• Contract: crew-judged-checks-contract.md