Groq menu highlights API#15

P2 Date-based pauses are ignored

If Groq sends an HTTP-date in Retry-After, Number(...) returns NaN and retryAfterMs falls back to 65 seconds. A longer pause is then ignored for all halls, which can waste the limited attempts on early retries. Support both numeric and HTTP-date values, and test a future date.

P2 Provider text enters logs

runHighlightJob logs raw error.message, including errors from both JSON.parse calls in generateHighlights. Those errors can contain fragments of the provider body or model output, despite the promise that logs never contain provider bodies. The 200-character limit does not remove that text. Replace parse errors with fixed messages and test the logged output for malformed responses.

P2 Audits mix old and new

The script resumes an existing file without checking its saved promptVersion against the current selector. After a prompt or model change, completed halls keep old picks while unfinished halls use the new code. This makes the audit unreliable for comparing changes. Reject mismatched runs using a saved model and prompt hash, or require a separate output file.

P2: A menu edit can still return the cached old menu and highlights for up to 60 seconds. Say old picks are invalidated on the next uncached response.

P2: Retry-After also permits an HTTP-date, but Number(...) treats that value as invalid and retries after only 65 seconds, potentially before the requested cooldown. Parse both date and delay-seconds forms before applying the minimum interval.

P2: The evidence this review rests on lives only in the gitignored .wrangler/group-eval/ directory, so it is absent from the repo. Commit the screening snapshot, rubric, and prompt, or state the exact archive location, so the 12-result analysis and prompt hash can be verified.

P2: sourceTitle throws when a selected item name strips to nothing, and the single-source branch in validateSelections does not catch it. One such item rejects the whole meal's highlights. Fall back to a non-empty display title or skip that source instead of throwing.

P2: The audit's source data lives only in the gitignored .wrangler/ directory and is not committed. Reviewers cannot re-derive the table counts or pick verdicts from the repo. Commit a sanitized copy of the inputs or name where the data is archived.

P2: The reproduce commands point to scripts that are not in the repo, so none of them can run as written. Either add the scripts in this PR or correct the commands to the scripts that exist.

P2: The build command here points to docs/reports/build-highlight-pick-audit.py, which is not in this PR. Commit the builder or remove the instruction, since the report's counts and validation claims depend on it.

P2: This assert.fail inside the fetcher is swallowed by runHighlightJob's catch, so the test cannot fail when Groq is called. Count the calls and assert zero instead.

P2: assert.throws here has no error matcher, so any exception passes, including a TypeError from a regression. Pass a matcher per case, such as /Invalid model output|Meal keys|Invalid meal groups|Invalid group|belong|more than one group/, or pair each input with its expected message.

P2: This report tunes on the Oct 4 holdout it declares off-limits. The §2–§3 Oct 4 'proposed' groups and policy fixes came from inspecting Oct 4 outputs, so Oct 4 can no longer serve as a holdout for these changes. Move these Oct 4 labels to a dev-only fixture or mark the holdout as spent.

P2: The reproduction command points to docs/reports/build-highlight-comparison.py, which is not in the repository, so the documented graphic cannot be rebuilt. Commit the builder or correct the command to the script that actually exists.

P2: The text says malott/m1 should yield 4 highlights, but the proposed JSON gives it 5. Either the count or the malott/m1 list is wrong. Reconcile them before this report is used as a reference.

P2: The repeat /test/tick check cannot exercise result reuse. The next alarm is about 65 s out and scheduleHighlights does nothing while an alarm exists, so groqCalls === 1 passes even if reuse broke. Remove this block and the "saved results reused" claim in the final log, since reuse is covered by test/highlights.test.ts.

P2: This overstates the model comparison. The archived holdout shows Qwen with higher coverage than 20B (35/51 vs 27/51), and GPT-OSS 120B was never run on the Oct 5 holdout because of quota. Limit the claim to the Oct 4 development run and note the untested holdout. Qwen's validation gap is one response in one run, so 'more often' overstates it.

P2: If setAlarm fails here, refresh() rejects even though the snapshot is already saved, so the scheduled run is reported as failed. Highlight scheduling is best-effort, so log the failure instead of propagating it.

P2: The reproduce commands reference npm run highlights:eval, which is not defined in package.json, and no evaluator or scorer script exists under scripts/. Either add the script and its fixtures or remove this section, since the claim that the checkout reproduces the results is currently false.

P3: This comment is now inaccurate. With highlights enabled for today, the handler also reads highlights:v1, so responses use two KV reads, not one. Update the comment to name both reads.

P3: The methodology claim is self-contradictory. §3 was written after reading the v5 Oct 4 outputs (§1, and the collins 'endorse the actual v5 run' note), so it cannot be a blind gold set. Either drop the 'before seeing any output' claim or build the holdout labels from the menus alone before reading model results.

P2: Resume skips any hall that has an entry, even when generation failed or the fetch failed. A 429, 5xx, timeout, or fetch error is never retried on rerun. Re-run halls that have no saved selections instead.

P3: The backticks in "General Tsos" and "Albertos" pair up inside the same table cell and render as an inline code span, which garbles that row. Escape them as ` so the source name renders literally, or use the apostrophe if that matches the source name.

P2: Resuming an audit file keeps halls captured under an older PROMPT_VERSION, and nothing compares the saved version. After a prompt bump the output mixes old and new picks. Fail fast when the saved version differs.

P3: The ready highlights example omits provider, model, generatedAt, and menuVersion, which every ready response includes. Add them so clients see the full public shape.

P3: The test count here (114) differs from the 136 tests stated in the PR description. Confirm the count from the final run and update whichever figure is stale.

P3: A multi-source group whose sources all have the same name is accepted, which the prompt forbids ("Represent a meal program once"). Skip or collapse variant groups whose normalized source names are identical, using the same name key as the single-source dedupe.

P3: The test count here (117) conflicts with the 136 tests stated in the PR description. Update this figure to match the actual test run output.

P3: The Total row does not match the per-hall rows. Completion tokens should be 2,346 and total tokens should be 25,190. Fix both values so the table reconciles, since this report is the usage record for the quota.

P3: The review cites blind-development.json as its source, but that file is not in the repo. Commit the input or the rubric it was checked against, or state where it lives, so the labels and IDs can be reproduced.

P3: Use the exported HIGHLIGHTS_KEY instead of the literal. Import it with import { HIGHLIGHTS_KEY } from '../src/highlights';. If the key changes, this poll would otherwise read a stale key and fail only with the generic 10 s timeout.

P3: The Italian Pitzer Sunday members omit m1:s0:i3, which §1 identifies as a pasta in the same bar. Either add it or document why it is excluded, since members are meant to be all merged variants or components.

P3: This runs Groq for ok menus that have no items. Production skips them through hasItems. Add the same check so the audit does not spend an attempt on an empty hall-day. This needs hasItems imported from ../src/highlight-selection.

P3: This catch-all returns unavailable without logging, so a saved record that fails validation or a bug in this block is invisible in logs. Log the error (without the record contents) before returning the empty result.

P3: Variant groups print item names without their stations, which breaks the header's promise to print each pick next to its station. Add the station to each variant item.

P3: This check accepts impossible calendar dates such as 2026-99-99, which are then sent to the providers and stored as an audit. Use the shared isValidDate validator.

P3: The claim that v8 'did no worse' without station-guide hints is not backed by any archived comparison. Either cite the run that measured it or state that it was not measured.

P3: The audit allows two attempts per hall, but production allows MAX_VERSION_ATTEMPTS (3) in src/highlight-jobs.ts. Import that constant and use it so the audit matches the shipped retry policy. Adjust the 14-attempt cap to match.

P2: A resumed audit never checks its saved promptVersion, so it can print old selections as results from the current pipeline after a version bump. Reject a mismatched saved version and require deleting the file to restart.

P3: The Frary account conflicts with the shipped code. A bad combined title splits in the same call and is never retried, so three failed calls cannot be explained by it. Either state that run 1 used an earlier pipeline, or correct the cause and say whether Frary ended ready or exhausted.