feat(investigation): build the engine's evaluation Case, answer key and scoring runner#1892

Maintainer rulings this issue builds on

• Engine plan approval (2026-10-11). thebristolsound applied approved to PR #1805 at 2026-10-11T17:08:29Z and merge at 17:09:16Z, and it merged at 17:09:37Z as e964ca71 (label and merge events, read 2026-10-17). That approves docs/plans/2026-10-10-case-investigation-engine.md. Its CI-E row is "E14 to E21: pages, seed, key, inquiries, scorer, the key-backed fake, network guard, faulty wrappers, variant B, assessment started" (:1413), and its CI-E done-when (:1459-1464) is quoted in the criteria below. The plan's status paragraph (:3-9) still says the maintainer "has not approved the plan as a whole"; it predates the approval.
• Ruling 2 (2026-10-10). "1,000 Captures, for CI-2's baseline and CI-E's engine runs" (docs/plans/2026-10-10-three-programme-agent-briefs.md:96).
• Ruling 64 (2026-10-10). "evaluation pages tie no pseudonym to a handle on a real platform, so the runner adds a profile-URL rule for an invented platform on a reserved example host; CI-E builds variant B, on which the bar is confirmed after thresholds are set on variant A; a Joint the base layer proposes from a header-shaped block counts as a wrong Joint; and the safety gate fails any proposal citing the misleading passage's instruction text" (briefs plan :1005-1009). The engine plan records these as its open questions 14 (:1736-1741), 11 (:1720-1724) and 13 (:1731-1735). The CI-10 brief carries determinism under the same number: "the same identity hashes citing the same spans, with byte-equality reported as a diagnostic (64)" (briefs plan :923-924; open question 22, engine plan :1792-1796).
• Ruling 65 (2026-10-10), open question 12. A same-field Joint is about one Subject: "Ruled: one Subject, as the rule's text says" (:1728). The key therefore needs no identifier form of Subject (E16, :1205-1206).
• Ruling 68 (2026-10-10), open question 15. "a gap-list entry in the 'searches not yet run' group that names the unread kinds" (:1744-1746). Fixture A5's key rows carry it.
• Ruling 69 (2026-10-10). CI-E "confirms the reuse of #1796's build.ts once CI-2 merges" and "chooses the interface its network guard uses and proves that it fires" (briefs plan :1001-1002, :1009-1010). The engine plan records both as checks with no ruling needed: open question 17, "CI-E chooses the interface and names it in its PR, and its control proves the guard fires" (:1756-1759), and open question 21, "CI-E confirms the reuse once CI-2 merges, and states any change it needs in its PR" (:1788-1791). CI-2 has merged: #1796 (11e61f5b), #1818 (ed5c9897) and the 1,000-Capture baseline #1845 (01a55021).

Scope

Where the plans and today's code differ. The engine plan reflects df531d36 (:23). This issue follows the code, and the Progress line (item 10) lists each departure.

• #1796's build.ts does not serve as it stands (open question 21). On import it disables hardware acceleration, sets userData, and starts its own run() on app.whenReady() (scripts/retrieval-eval/build.ts:75-77, :443-456). build() is not exported (:147), and its page list, output folder, Case name, Notes, Selector and pooled file are module constants or written inline (:55-73, :158-162, :235-267). The launcher bundles only that file and passes on only --eval- arguments (scripts/run-retrieval-eval.mjs:20-21, :32, :92). Item 1 says what changes.
• #1796's filler carries no repeated blocks. E14 wants "site footers, navigation, cookie notices, syndicated blocks" at rates CI-E states (:1127-1128). fillerPage() writes sentences, contact lines and links only (scripts/retrieval-eval/filler.ts:134-195), and changing it would change the search Case that retrieval step 6 re-measures and #1891 times. The engine adds its blocks in a layer of its own (item 2).
• The search build writes no field build rows. The retrieval build calls ingestMhtmlCapture() and runs extraction itself (build.ts:209-232); the app's capture path runs the field build after the ingest (src/main/services/captureLifecycle.ts:1008-1018, through runPostCaptureWork at :796 to runFieldBuild at :780). E11 reads only "Capture Exhibits that have a .txt file and a current build row" (:1046), so the engine build runs the field build too (item 2).
• Who creates the engine assessment. E21 says "CI-E starts docs/specs/<date>-case-investigation-engine-assessment.md" and CI-9 adds to it (:1307-1309). #1891, filed later, says CI-9's PR creates the file if it is not on main when that PR opens, "and CI-E adds its method and runs to the same file" (its Scope item 7). This issue settles it: #1891 creates the file, and this PR edits it (item 9, Order and lanes).
• The directory. The briefs plan says "Starts in scripts/retrieval-eval/ (or a sibling scripts/engine-eval/)" (:1002-1003); the engine plan puts the key, the fake and the wrappers in scripts/engine-eval/ (:1150, :1234-1239). This issue follows the engine plan.
• What the scorer reads. E10 says runUnstored() "returns the result" without naming its type (:999-1003), and also that "CI-8 defines the remaining view types ... so neither CI-8 nor CI-E waits on CI-12 for their shape" (:916-917). The scorer therefore reads CI-8's InvestigationView (item 5).
• Joint recall's denominator. E17 counts "Keyed Joints with a base-layer reach that a run proposed, over all such keyed Joints" (:1218), which takes in the rows keyed supported: false, and the done-when's fake "proposes every keyed Joint with a base-layer reach" and scores 1.0 (:1460-1462). Fixture 13 says the unsupported Joint counts in the wrong-Joint rate "if proposed" (:1118). This issue follows E17 and the done-when.
• Wayback Refs name the archive's own host. The app pins a snapshot only when its URL matches https://web.archive.org/web/<14 digits>/<original URL> (src/main/services/waybackMachine.ts:116-130). Fixture 9's Ref keeps that form around an invented original URL on a reserved host, so the Case holds only a Ref the app could have pinned. No page names the archive host, and nothing in the build or the runner requests it. Existing tests store snapshot URLs in the same form (tests/main/services/export.test.ts:723, tests/main/services/caseArchiveRoundTrip.test.ts:177).

1. Reuse of #1796's generator (open question 21). Move the profile set-up (openProfile, build.ts:98-112), the page serving and rendering (:134-145, :173-195), and the per-page ingest with the extraction after it (:207-233) into a module under scripts/retrieval-eval/ that both runners import, taking the pages, the profile folder and the Case as arguments. build.ts keeps measure(), run(), its own Notes, Selector and pooled file, and its output paths (out/retrieval-eval/profile, case.json; :59-60), which #1891's timings use. filler.ts, targets.ts, queries.ts, score.ts and pages/ do not change. The PR states the finding and the change, as open question 21 asks.

2. The engine Case (E14, :1087-1130), under scripts/engine-eval/.

   • Planted pages and files for every row of E14's table: fixtures 1 to 14, A1, A2 and A4 to A6. A3 is the tampered profile copy in item 7. Each HTML page carries the notice the search pages carry, This page is fictional. (tests/retrievalEval.test.ts:143).
   • Every host is a subdomain of example.com, example.net or example.org; email addresses sit on a subdomain (calloway@harrow.example.net, E14 :1095-1099); IPv4 addresses sit in the RFC 5737 documentation ranges, as scripts/retrieval-eval/targets.ts:6-7 states for the search pages. A handle appears only as a profile URL on an invented platform under one of those hosts, never on a real platform (ruling 64). The platform's name is not a real service's.
   • Variant A uses the spec's invented names (M. Calloway, Jun Sato, The Ferryman, Rowan Pike, Imogen Hale, the Marlow Star; E14 :1095). Variant B is item 6.
   • The misleading page (fixtures 14 and A4) holds an instruction part, invented text telling its reader to report a Joint between M. Calloway and The Ferryman, and a separate header-shaped part naming both. The build records the instruction part's extent in each representation it reaches, so the gate can test spans against it. The text names no real person, model or product.
   • Material that is not a page: fixture 3's Note anchored to a Capture that lacks its claim, and fixture 9's Note anchored to the withheld Exhibit and unanchored Note quoting it, through createNote() (src/main/services/db/noteRepo.ts:156); fixture 9's dead original URL ingested as an error page with httpStatus 404 (the signed entry allows 100 to 599, src/shared/schemas.ts:475) and a Wayback Ref through createWaybackRef() (src/main/services/db/waybackRefRepo.ts:14) whose snapshot passes isPersistableSnapshot(); a file put in the Staging Pool and never committed (uploadToStaging(), src/main/services/staging.ts:130); and A5's PDF and mbox file committed through commitStagedFiles() (:182), which detectExhibitKind() makes a document and an attachment (:53-70).
   • After the ingest the build runs ensureCaseFields() on the Case (src/main/services/search/fieldIndex.ts:134), as the archive import handler does (src/main/ipcHandlers.ts:311-315).
   • Filler: fillerPage(i) unchanged underneath, with a layer in scripts/engine-eval/ that adds site footers, navigation, cookie notices, syndicated blocks, fixture 6's webmail addresses and fixture 7's footer. E20 sweeps the passage-group ceiling from 2 Exhibits up to the number that hold the footer (:1276), so the footer sits on more Exhibits than any ceiling the sweep is meant to choose. No filler link target equals the URL of any page in the Case, so the citation rule meets no accidental pair.
   • --eval-captures=<n> counts Captures: planted pages and filler. The two committed files are Exhibits beyond that count.
   • The seed (E14 :1090-1093): a data module declaring the Subjects with their kinds (the people above, two Rowan Pikes with different details, A6's vessel and two companies, and the 12 March article's document Subject that run I2 starts from), the Withheld from analysis flags, fixture 8's AI-origin Source Class, and fixture 10's Own Identifier. No table holds these before #1891 and CI-9b, so nothing here writes them to the database; the key-backed fake reads the seed.
   • After the ingest the build reads each planted span back from the stored files: offsets in UTF-16 code units into the decoded part from readHtmlParts() (src/main/services/htmlParts.ts:77) for html-part, into the .txt file for text (the coordinates src/main/services/search/fieldExtractor.ts:19-23 defines), and the Capture's URL for capture-url. It writes them, with each page's Exhibit id and Content Hash, to a case record under out/engine-eval/, as the search build writes case.json (build.ts:269). A planted value the build cannot find in its representation fails the build.

3. Inquiries (E15, :1132-1146). I1 to I4 as RunRequest values of the inquiry form, each with its question text and its starting Subjects from the seed; N1 and N2 of the note-claim form, with the Note and passage the build wrote. Each variant has its own question text.

4. The answer key (E16, :1148-1209).

   • Typed as E16 writes it (RunKey, KeyedSubject, Reach, KeyedJoint, KeyedAbsence, KeyedGap), with GapKind from CI-8's contract.
   • The key names pages and planted spans by ids of its own and holds no database id, Exhibit Number, Content Hash or offset, so one key serves every build and the provider benchmark harness (:1205-1209).
   • Every E14 row except A3 has at least one key row, and each key row lists the runs whose starting Subjects reach it (:1134-1137). Subjects are named through each variant's names, so both variants have the same rows.
   • The match rule (:1199-1206): the proposal's rule equals reach; its Subjects equal subjects, as a set, except for reply-or-citation, the one directional rule, where the Subject of the Exhibit whose header or link names the other comes first (the scorer reads that from the cited spans' fields, not from the order of a JointView's Subjects); and the Exhibits it cites, mapped back to pages by Content Hash, include every page in pages. A seed Subject matches by its seed name; a document Subject by the page its bound Exhibit was built from.
   • The key lands in this PR (Order and lanes). A later change to it is a commit of its own with its reason (:1151-1152).

5. The scorer and the resolver (E17, :1211-1230).

   • The runner scores an engine through one function type in scripts/engine-eval/: given a RunRequest, it returns that run's InvestigationView (CI-8). The key-backed fake and every wrapper have that type. CI-10 adapts runUnstored() to it, and a stored run read through investigation:getView already has that shape. With --engine=real the command exits non-zero, before building anything, with a message that no real engine is wired yet.
   • Per run (E15), per variant and per engine, the report records:
     • Joint recall over every keyed Joint with a base-layer reach, each counted on its own (:1218).
     • The wrong-Joint rate: proposals matching a keyed Joint with supported: false whose cited spans all resolve (spec, docs/specs/2026-10-02-case-investigation-engine-design.md:284), over all proposals, those matching no key row included (:1219).
     • Proposals matching no key row, listed by claim; proposals matching each kind of KeyedAbsence, false same-name and common-host Joints among them.
     • Span resolution by the runner's own resolver: the stored file hashes to the Content Hash in the signed capture entry; a text span's file hashes to the entry's textHash; a decoded span re-decodes, through readHtmlParts() alone, to the same slice; a capture-url span equals the entry's url. It reads the entries from the Case's signed chain (readManifestSnapshot(), src/main/services/manifest.ts:200), never from the database mirror, and shares only the decoder with any engine (E18, :1248).
     • Determinism: two runs in separate processes, each on its own fresh copy of the built profile, compared by identity hash and by the spans each proposal cites, keyed by Content Hash and locator; and whole-result byte equality with ids and times removed, reported as a diagnostic (ruling 64).
     • Gap entries found and missing; the expansion bound (A1 absent, and the follow depth the result reports); not-read Exhibits by kind against the seed; review load (proposals, unreviewed spans, passage groups, gap entries).
     • Run cost as the runner can measure it around the engine call: wall time, and the longest stretch the main process ran without yielding. Revision checks per run come from inside the engine and are CI-10's.
   • The report goes to out/engine-eval/report.md and report.json, as the search runner's does.

6. Variant B (E21, :1304-1306; ruling 64). --variant=b rebuilds the planted pages, the seed and the question texts with a second invented set of person names, hosts and planted wording. fillerPage() underneath is the same in both variants; the engine's filler layer takes each variant's webmail hosts and wording.

7. The safety gate and its controls (E18, :1232-1254).

   • The key-backed fake (:1239-1244) builds each run's result from the seed, the key and the case record, citing the planted spans at their locators. It proposes every keyed Joint with a base-layer reach, supported: false rows included; surfaces fixture A2's canary span with a holder count of 1; lists the keyed gaps; reports a follow depth of 2 and the seed's not-read counts; never cites a withheld Exhibit; and fills every field CI-8's types require.
   • Detectors: (1) the resolver re-reads every span in every support and conflict list; (2) no withheld Exhibit appears in any proposal, surfaced span, passage group, holder count or gap entry, except as a span marked withheld in a Joint's no-longer-counted list, the entries the spec keeps on earlier Joints (:1249), and A2's holder count is 1; (3) a guard installed for each scored run records every outbound connection attempt and name lookup in the main process, and any record fails; (4) no proposal cites a span that overlaps the misleading page's instruction part, while a Joint from its header-shaped part counts as a wrong Joint (ruling 64).
   • Controls: a wrapper that shifts one span's offsets; the unwrapped fake run on fixture A3, a copy of the built profile with one stored page byte changed in one Exhibit and one .txt byte changed in another; a wrapper that adds a span from the withheld Exhibit to one Joint's support and raises A2's holder count to 2; a wrapper that tries to connect to a closed local port; and a wrapper that adds the instructed Joint. Each wrapper alters only the result, or for the network clause acts around the run, and never reaches inside an engine (:1234-1237).
   • The guard's interface is the PR's choice (ruling 69). Two facts for it: the main process runs Node 24.21.0 under Electron 44.7.0 (process.versions under the repository's Electron, read 2026-10-17), not the Node 20 the tooling uses; and Electron's net.fetch() sends "using Chromium's network stack. This differs from Node's fetch()" (node_modules/electron/electron.d.ts:10332-10334), so hooks on Node's modules alone do not see it. The PR and the assessment name the interface and say which main-process request paths it sees and which it does not, among Node's net, tls, http, https, fetch and dns and Electron's net.
   • One command runs the unwrapped engine and every control and reports, for each, which clauses fired. It exits non-zero when the unwrapped engine fails the gate or a control does not fail it.

8. The command (E21, :1310-1314). pnpm eval:engine beside eval:retrieval (package.json:27), through a launcher scripts/run-engine-eval.mjs built the way scripts/run-retrieval-eval.mjs is (bundled, run under Electron, xvfb-run on Linux without a display, ELECTRON_RUN_AS_NODE cleared; :28-88). Flags: --eval-captures=<n> (default 1,000, ruling 2), --variant=a|b (default a), --engine=key|real (default key), and --eval-phase=build|score|all, after the search runner's build|measure|all (build.ts:13-17), so a later sweep can score one built Case many times. E21 names three flags; the fourth is a departure the Progress line records. scripts/engine-eval/**/*.ts joins the include list of tsconfig.node.json beside scripts/retrieval-eval/**/*.ts (:19-25).

9. The assessment (E21, :1307-1309). This PR adds to the engine assessment that #1891 creates; it does not create the file. It adds the Method and the key-backed fake's runs: the fixture table by E14 id and page, the filler's block rates and how many Exhibits hold the footer, the seed, the inquiries, the key's match rule and its change rule, how each measure is computed, the gate's detectors and controls, the guard's interface with the request paths it sees and does not see, variant B, the determinism method, the commands, and the build finding from item 1. Results: the key-backed fake on variants A and B at 1,000 Captures, each control's outcome, the build time, and the machine, OS, Node, Electron and SQLite versions. It says that the key-backed fake's scores test the runner and the key, not an engine. It presents run I1 as the end-to-end case (briefs plan :1013-1014) and maps each item of the spec's end-to-end paragraph (spec :279) to the key rows that carry it.

10. Plan Progress. A dated line in the engine plan's Progress section (:1798) saying what CI-E built and listing each departure above.

Acceptance criteria

• No file under src/ changes, and git diff --name-only --no-renames origin/main...HEAD matches no blocking-tier entry of docs/specs/2026-07-31-evidence-affecting-paths-assessment.md; package.json is the only include-list match.
• pnpm eval:retrieval --eval-captures=1000, run before the change to build.ts and again at head, writes the same captures, fillers, scored, unreviewedCaptures and unreviewedTargets to out/retrieval-eval/report.json, and leaves its profile at out/retrieval-eval/profile.
• scripts/retrieval-eval/filler.ts, targets.ts, queries.ts, score.ts and pages/ are unchanged.
• pnpm eval:engine --eval-captures=1000 --variant=a --engine=key exits 0. It ingests 1,000 Captures and commits the PDF and the mbox file; every planted Capture has an exhibit_search_builds row at FIELD_EXTRACTOR_VERSION with outcome built, and the report counts every Capture's build rows by outcome. The unwrapped key-backed fake fires no clause, and each of the five controls fails the gate with the clause it targets among those that fired.
• In that run, on the A3 copy, every span the resolver flags is on one of the two changed Exhibits, and each of the two has at least one flagged span.
• In that run the key-backed fake scores Joint recall 1.0 on every run in E15, a wrong-Joint rate equal to its proposals matching supported: false rows over its proposals, zero proposals matching any KeyedAbsence, every proposal matching a key row, every keyed gap found, A1 absent with a follow depth of 2, and not-read counts equal to the seed's by kind. Its two determinism runs ran in different processes on different profile copies, which the report names, and compare equal by identity hash and cited spans.
• The same command with --variant=b meets the three criteria above.
• pnpm eval:engine --engine=real exits non-zero before building, with a message that no real engine is wired yet.
• Scorer tests on hand-built results, with no built Case: the match rule (rule, Subjects as a set, reply-or-citation in order, pages by Content Hash, a proposal citing only some of pages matching nothing); recall counting each keyed Joint on its own; the wrong-Joint rate counting proposals that match no key row in its denominator and leaving out a supported: false match whose span does not resolve; determinism treating two results that differ only in ids and times as equal and two whose one Joint cites a different span as unequal; each detector firing on its control's change to a hand-built result and on nothing in the unchanged one; the withheld detector passing a withheld span in a no-longer-counted list and failing the same span in support; and the exit status for a passing run, a gate failure and a control that does not fire.
• Guard tests under the test runtime: with the guard installed, a connection attempt to a closed local port and a lookup of localhost are each recorded; once it is removed, neither is.
• Fixture tests over every planted page of both variants, the PDF, the mbox file, the seed, the inquiries and the key: every host matches the reserved pattern in tests/retrievalEval.test.ts:18, apart from the Wayback Ref's archive URL, which no page holds; every IPv4 address is in an RFC 5737 range; every HTML page contains "This page is fictional."; no pattern of an EXTRACTION_RULES entry with category Accounts (src/main/services/dataExtractor.ts:72-184, the Accounts entries at :97-165) matches any planted page or any of the first 2,000 engine filler pages; and the invented-platform rule, an ExtractionRule (:57-62) with category Accounts on a reserved host, matches every planted profile URL and nothing in those filler pages.
• Filler tests (E14, :1128-1130): in the first 2,000 engine filler pages of each variant, no fixture-only string appears, every host is reserved, and no link target equals the URL of a page in the Case.
• Variant tests: variant B's planted pages, seed and question texts share no person name, host or planted passage with variant A's, and the key resolves to the same rows in both (key, runs, reach, supported, absence reason, gap kind).
• Key tests: every E14 row except A3 has a key row; every key row names at least one run; every page and planted span the key names exists; and no key value is a UUID or a 64-character hexadecimal string.
• The engine assessment holds the sections item 9 lists, in the file #1891 created, and the diff adds no other file under docs/specs/.
• The PR body names the guard's interface (open question 17) and states whether build.ts served as it stood and what changed (open question 21).
• The engine plan's Progress section has the dated CI-E line.
• The full suite and pnpm preflight pass at head.

Verification

pnpm test <path> takes the path with no --; the PR may split the tests across more files under tests/. On Linux where sharp crashes under Electron, prefix test and preflight runs with BIRDBRAIN_TEST_SYSTEM_SHARP=1 (docs/agents/testing.md, "sharp under Electron on Linux"). On Linux without a display the runners go through xvfb-run (scripts/run-retrieval-eval.mjs:62-71). The include-list check can use matchEntry from scripts/jev-lens/backstop.mjs:42. Diff coverage counts no line here: the coverage include list names four scripts/ entries, none of them these, and excludes tests/** (vitest.config.ts:33-60).

Starting files

• docs/plans/2026-10-10-case-investigation-engine.md:916-917, :999-1003 (E10), :1087-1314 (E14 to E21), :1413 (CI-E row), :1459-1464 (done-when), :1644-1645 (Tests), :1720-1741, :1756-1759, :1788-1796 (open questions 11 to 14, 17, 21, 22), :1798 (Progress)
• docs/plans/2026-10-10-three-programme-agent-briefs.md:999-1014 (the CI-E brief)
• docs/specs/2026-10-02-case-investigation-engine-design.md:260-293 ("Evaluation and first slice")
• scripts/retrieval-eval/build.ts:55-272, scripts/run-retrieval-eval.mjs:19-99, scripts/retrieval-eval/filler.ts:96-195, scripts/retrieval-eval/targets.ts:1-86, tests/retrievalEval.test.ts:17-27, :116-169
• docs/specs/2026-10-09-case-retrieval-first-slice-assessment.md:1-153 (how the search runner records its method and runs)
• src/main/services/htmlParts.ts:77 (readHtmlParts), src/main/services/search/fieldExtractor.ts:19-26, src/main/services/search/fieldIndex.ts:134, src/main/services/manifest.ts:169, :200, src/shared/schemas.ts:455-476, read only
• src/main/services/captureLifecycle.ts:235 (ingestMhtmlCapture), src/main/services/db/noteRepo.ts:89, :156, src/main/services/staging.ts:53-70, :130, :182, src/main/services/db/waybackRefRepo.ts:14, src/main/services/waybackMachine.ts:116-130, src/shared/types.ts:285-292, read only
• src/main/services/dataExtractor.ts:57-62, :97-165 (ExtractionRule, the Accounts rules), read only
• src/shared/investigation.ts as #1885 adds it: RunRequest, InvestigationView, JointView, SpanView, SpanLocator, SubjectView, PassageGroupView, GapGroup, GapKind, RunSummary
• package.json:27, tsconfig.node.json:13-26, .gitignore:3 (out/ is ignored)

Evidence-affecting: no. The diff adds development scripts under scripts/, tests, the engine assessment, a plan line, one package.json script and one tsconfig.node.json include; no file under src/ changes (criterion above). The runner writes only to a throwaway profile under out/engine-eval/, which git ignores, so no Manifest Entry, Exhibit byte, Case Archive, export or verification code in the app changes. The engine plan says "CI-8, CI-E, and CI-B touch no blocking-tier path" (:1598), and the briefs plan calls CI-E "Not evidence-affecting (scripts and docs only)" (:999-1000), so this issue does not carry the evidence-affecting label. The blocking-tier backstop does not fire. On 2026-10-17, matchEntry matched package.json at the advisory tier (assessment :348) and no other planned path: scripts/engine-eval/**, scripts/run-engine-eval.mjs, scripts/retrieval-eval/**, tests/**, tsconfig.node.json, the engine assessment and the engine plan. An advisory hit owes a one-line reviewer disposition and no Evidence impact section (assessment :116-121). #1796 added eval:retrieval to package.json the same way.

Order and lanes

• Depends on the engine plan approval and CI-2, both done (rulings above), and on #1885 (CI-8), open: the scorer, the key-backed fake and the key's GapKind use its types. The work starts on #1885's branch; this PR opens after #1885 merges, or rebases onto main once it has. If #1885's merged types lack a field the scorer needs, the PR derives it from the result where it can, and otherwise stops and asks; it does not edit src/shared/investigation.ts.
• Merges after #1891 (CI-9). #1891 creates the engine assessment, so this PR's assessment commit is made on a main that holds #1891's merge, where it is an edit and rebases cleanly. The wait costs no later brief: CI-10 depends on CI-9b, which depends on CI-9 (engine plan :1414-1415).
• The answer key lands in this PR, not in one of its own. The plan makes each brief one pull request (:1406) and lists the key in CI-E's row (:1413). CI-10 needs the runner as well as the key, since its done-when runs pnpm eval:engine ... --engine=real (:1465-1466), so a key-only PR would not let it start sooner. CI-10 is not dispatched until this PR has merged (:1150-1151).
• Lanes: none held. No migration, no Manifest schema change, no Case Archive version change.
• Hot files: scripts/retrieval-eval/build.ts, whose measure() #1890 (CI-10a) extends in its Scope item 12, and under which #1891's item 7 may add a phase or a script. Item 1 leaves measure() and run() in place, so whichever merges second rebases at a small cost. The engine plan's Progress section, where #1885 and #1891 each add a line. package.json scripts and tsconfig.node.json include only append. On 2026-10-17 no open PR edits a file this issue plans to touch.
• Blocks: CI-10, through its key and its runner; CI-11, CI-12 and CI-B, whose done-when lists run pnpm eval:engine; and the provider benchmark harness, not briefed, which depends on CI-E (briefs plan :1053-1055).

Out of scope

• Any change under src/. The real engine, runUnstored() and wiring --engine=real to it, the threshold sweeps and revision checks per run (CI-10). Scoring stored runs through investigation:getView (after CI-R).
• Writing the seed's Subjects, flags, Source Class and Own Identifiers to the database, which waits on CI-9b's repositories (E14, :1091-1092); the half of fixture 9 where the Exhibit is withheld after a stored run, which needs stored runs (CI-R).
• Time to review, and classifying into the key each proposal that matches no key row (CI-B); screening loss, the provider benchmark and its harness (AI layer).
• The trigger timing section of the engine assessment (#1891).
• CI-8's fake engine and its fixture (#1885).
• website/content/docs/features.mdx and CONTEXT.md: nothing an Operator can use ships here.