test(providers): replace covered settings-page greps with executed tests (#311)

* test(providers): replace covered secret-path greps with executed tests

The nonce, planner, pending-key, icon and safe-text wiring of
configProviders.qml is now exercised by executed tests: twelve wiring
tests in tests/test_secret_prompt_lifecycle.py that extract the
production functions and run them against fake sources, an argv test
that runs the generated secret-prompt commands against fake
kdialog/codexbar binaries and pins the recorded argv, and three
real-page tests in tests/tst_provider_filters.qml that drive hostile
descriptor text through the live delegates.

Each removed fragment was verified by breaking the behavior it
describes and confirming an executed test goes red. No production
code changes.

Evidence (mutation -> red test):

test_refresh_nonce.sh (providers surface):
- `property int commandRunSerial: 0` -> init 100: the initTestCase
  RUN=1 probe in ProviderCommandWiring goes red.
- `import "CommandLedger.js" as CommandLedger` -> dropped import:
  tst_provider_filters fails to load the page (failOnWarning).
- `function disconnectCommandsByKind(kind)` -> sourcesOfKind stubbed
  to []: test_reloadRetiresInFlightListCommand goes red.
- `disconnectCommandsByKind("list")` -> call removed from reload():
  same test goes red (stale source never disconnected).
- `commandRunSerial += 1` -> increment removed:
  test_runCommandsMintDistinctNonceSources goes red (shared source).
- `CommandLedger.withRunNonce(command, commandRunSerial)` -> bare
  command: same test goes red (nonce prefix missing).
- `CommandLedger.opened(...)` -> ledger write removed: same test goes
  red (descriptor lookup misses).
- `configSource.connectSource(sourceName)` -> connect removed: same
  test goes red (nothing connected).

test_security_regressions.sh (providers surface):
- `plainText: reloadProvidersButton.Accessible.name` -> literal
  "Reload!": test_reloadTooltipRoutesAccessibleName goes red.
- `parent: reloadProvidersButton` -> line removed: no behavior to
  break (declarative nesting already parents the tip to the button);
  the new tooltip test asserts tip.parent on the live object and stays
  green with the line deleted.
- `SafeText.plainButtonText(...)` -> raw modelData.title:
  test_providerActionButtonRendersTitleAsPlainText goes red.
- `Components.PlainComboBox {` -> Controls.ComboBox: combo branch of
  test_descriptorFieldControlsRenderUntrustedTextAsPlain goes red (no
  PlainItemDelegate in the popup).
- `Components.PlainCheckBox {` -> Controls.CheckBox: page fails to
  load (plainText has nowhere to go); adapted swap with raw text goes
  red in the checkbox branch of the same test.
- `placeholderText: SafeText.plainTextAsRichText(...)` -> raw
  description: same test goes red.
- `function providerMapKey(providerID)` -> returns "":
  test_providerMapKeyNormalizesAliases goes red.
- `return ProviderIdentity.providerMapKey(key)` -> raw key: same test
  goes red (11labs no longer maps to elevenlabs).
- `return Guards.copyObject(item)` -> alias:
  test_copyObjectDetachesAndDropsUnsafeKeys goes red.
- `import "Guards.js" as Guards` -> dropped import:
  tst_provider_filters fails to load the page (failOnWarning).
- `ProviderIdentity.providerIconFileName(value)` -> raw value:
  test_providerIconSourceFallsBackForUnusableKeys goes red.
- `planFieldWrite(field, value, commandPath)` -> guard bypassed
  (`if (false && ...)`): test_secretFieldWriteRequiresSecurePrompt
  goes red (secret runs); value dropped:
  test_textFieldWriteRunsPlannedCommand goes red (placeholder never
  filled).
- `planSecretPrompt(field, commandPath)` -> guard bypassed:
  test_promptPathRejectsNonSecretField goes red.
- `planAction(action, commandPath)` -> guard bypassed:
  test_descriptorActionRunsOnlyAllowedCommands goes red (rm -rf runs).
- `String(payload.value.url)` -> no coercion:
  test_actionResultOpensOnlyHttpsUrls goes red (numeric URL throws).
- `safeHttpsUrl(url)` -> raw url: same test goes red (javascript:
  URL accepted).
- `function descriptorPendingFieldKey(fieldID)` -> returns "":
  test_descriptorPendingKeysQuoteAndBoundLengths goes red.
- `return JSON.stringify(value)` -> raw value: same test goes red
  (unquoted key).
- `var field = descriptorPendingFieldKey(fieldID)` -> inline trim:
  same test goes red (overlong id accepted, quotes missing).
- `["sh", "-c", shellQuote(script), "_", shellQuote(prompt)` ->
  placeholder dropped:
  test_prompt_commands_carry_positional_args_and_quote_hostile_prompts
  goes red (recorded argv shifts); prompt unquoted: same test goes red
  (positional collapse under a real shell, hostile substitution
  executes).

Notes: the checkbox/option page assertions use entity-free strings
because the org.kde.desktop native label warns on escaped entities at
render time (pre-existing behavior, found while running the new tests
under both styles in make check-qml-logic); hostile text for those
controls stays covered at the component seam in
tst_plain_text_controls.qml. Absence assertions (reject_*) stay
untouched, as do the applet-side pins outside this step's scope.

Verification: make check-refresh-nonce,
make check-security-regressions and make check-qml-logic are green, as
are check-python-test_secret_prompt_lifecycle and
check-python-test_provider_write_errors; git diff contents/ is empty.
check-python-lint is skipped locally (pyflakes absent; CI installs
it). Full `make check` left to the orchestrator.

* test(providers): replace covered list-pipeline greps with executed tests

Seven wiring tests in tests/test_secret_prompt_lifecycle.py now execute
the provider list, retry, diagnose and field-result pipeline extracted
from contents/ui/configProviders.qml. Each removed fragment was verified
by breaking the behavior it describes and confirming the executed test
goes red. No production code changes; git diff contents/ is empty.

Evidence (mutation -> red test):

- `property string selectedProviderID` -> property deleted: the page
  fails to load, tst_provider_filters goes 2 passed/4 failed.
- `selectedProviderID = firstSelectableProvider(next)` dropped from
  handleListResult: test_listResultSelectsFirstEnabledProvider goes red.
- `function shouldRetryProviderListWithoutDescriptors(...)` -> return
  false: test_descriptorUnsupportedMessageDelegatesToProtocol goes red.
- `function descriptorListUnsupportedMessage(...)` /
  `ProviderConfigProtocol.descriptorUnsupportedMessage(...)` -> return
  "": same test goes red.
- `ProviderConfigProtocol.commandError(payload)` -> "" in
  handleDiagnoseResult: test_diagnoseResultSurfacesEnvelopeError goes red.
- `ProviderConfigProtocol.normalizeProviderList(payload, ...)` -> []:
  test_listResultSelectsFirstEnabledProvider goes red.
- `function handleDiagnoseResult(...)` branches pinned by the same two
  diagnose mutations above (stores + envelope-error tests).
- `ProviderConfigProtocol.normalizeProviderDiagnostic(payload)` ->
  raw payload: test_diagnoseResultStoresNormalizedDiagnostic goes red
  (authConfigured object is not true, settingsKeys not joined).
- `function handleDescriptorFieldResult(...)` -> markFieldPending
  unlock dropped: test_fieldResultSuccessReloadsPreservingMessages and
  test_fieldResultErrorReportsWithoutReload go red.
- `function handleDescriptorActionResult(...)` -> page.reload(true)
  dropped: test_actionResultOpensOnlyHttpsUrls goes red (reloadCalls 0).
- `property bool providerDescriptorsUnavailable` -> property deleted:
  page fails to load (2 passed/4 failed); flag cleared in the retry
  branch: test_listResultRetriesWithoutDescriptorsOnOldCli goes red.
- `runProviderListCommand(false)` -> (true) in the retry branch: same
  retry test goes red (fallback still carries --descriptors).
- `descriptor.includeDescriptors === true` -> === false: same retry
  test goes red (retry never triggers).
- `page.reload(true)` in handleDescriptorFieldResult dropped: the
  field-success test goes red (reloadCalls 0); the action-result variant
  above pins the same fragment in handleDescriptorActionResult.

Verification: make check-feature-parity green; the full
tests.test_secret_prompt_lifecycle module (4 tests) green.
check-python-lint is skipped locally (pyflakes absent; CI installs it).

* test(providers): replace covered command-construction greps with executed tests

Five more wiring tests execute the command-building path extracted from
contents/ui/configProviders.qml, and the pre-existing secret-prompt argv
test pins the generated secret commands against fake binaries. Each
removed fragment was verified by breaking the behavior it describes and
confirming the executed test goes red. No production code changes.

Evidence (mutation -> red test):

- `function setApiKey(providerID)` -> kind retagged setApiKeyX: the
  secret-prompt fixture lookup finds no setApiKey command (StopIteration).
- `function loadProviderSettings(providerID)` / `diagnose --provider`
  -> diagnose2: test_loadProviderSettingsRunsRedactedDiagnose goes red.
- `--format json --redact` -> --format json: same test goes red.
- `function runProviderListCommand(includeDescriptors)` /
  `--descriptors` -> flag push dropped:
  test_listCommandCarriesDescriptorsAndJsonFlags goes red.
- `property var providerDiagnostics` / `property var providerFieldPending`
  -> property deleted: the page fails to load, tst_provider_filters goes
  2 passed/4 failed; the diagnose and field tests above exercise the
  state the properties carry.
- `field.writeCommand` guard dropped in writeDescriptorField and in
  promptDescriptorSecret, `action.command` guard dropped in
  runDescriptorAction: test_emptyWriteCommandsStaySilent goes red (the
  planner reports an unsupported command instead of staying silent).
- `kind: "descriptorField"` -> descriptorFieldX:
  test_fieldWriteRegistersDescriptorFieldKind goes red.
- `kind: "descriptorAction"` -> descriptorActionX:
  test_actionRegistersDescriptorActionKind goes red.
- `kdialog --password` -> kdialog: the argv test
  test_prompt_commands_carry_positional_args_and_quote_hostile_prompts
  goes red (recorded kdialog argv loses --password).
- `config set-api-key --provider` -> set-api-key2: same argv test goes
  red (recorded codexbar argv differs).
- `--stdin --format json --json-only` -> without --stdin: same argv
  test goes red.

Verification: make check-feature-parity green; tests
test_secret_prompt_lifecycle (4 tests) and test_provider_write_errors
green. check-python-lint skipped locally (pyflakes absent).

* test(providers): replace covered settings-page greps with executed tests

Six wiring tests execute the descriptor-row, settings-row, CLI-text,
identity and action-row helpers extracted from
contents/ui/configProviders.qml, and three real-page tests in
tests/tst_provider_filters.qml drive the kind delegates, headings and
CLI text through the live page. Each removed fragment was verified by
breaking the behavior it describes and confirming the executed test
goes red. No production code changes; git diff contents/ is empty.

Evidence (mutation -> red test):

- `function descriptorFieldRows(item)` -> always []:
  test_descriptorRowHelpersReadFieldsAndActions goes red.
- `function descriptorHasAction(item, actionID)` -> === flipped to !==:
  same test goes red.
- `function writeDescriptorField(providerID, field, value)` -> write
  guard dropped: test_emptyWriteCommandsStaySilent goes red; kind
  retagged: test_fieldWriteRegistersDescriptorFieldKind goes red.
- `function promptDescriptorSecret(providerID, field)` -> prompt guard
  dropped: same silence test goes red; --password dropped from either
  secret script: the secret-prompt argv test goes red.
- `function providerSettingsRows(item)` -> API-key-setup row dropped:
  test_providerSettingsRowsLabelAndDiagnose goes red.
- `function providerCliCommandText(item)` -> diagnose line dropped:
  test_providerCliCommandTextListsHelpfulCommands goes red, and on the
  live page test_cliCommandsTextShowsHelpfulCommands goes red.
- `!descriptorHasAction(item, "openDashboard")` guard dropped:
  test_actionRowsGuardDashboardAndOfferApiKey goes red (dashboard row
  appears despite the descriptor action).
- `Provider options` -> renamed: the kind-rows page test goes red
  (options label count 0).
- `modelData.kind === "secret"/"enum"/"boolean"` -> each flipped:
  the kind-rows page test goes red (that kind's row hides).
- `function providerDocsUrl/providerLoginUrl` -> return '':
  test_identityDelegatesToProviderTable goes red.
- `function providerCliArgument(value)` -> return value:
  same test goes red (groq no longer maps to groqcloud).
- `function supportsApiKeySetup(providerID)` -> openai case dropped:
  test_supportsApiKeySetupAllowlistsProviders goes red.
- `action: "set-api-key"` -> set-api-keyX:
  test_actionRowsGuardDashboardAndOfferApiKey goes red (no key row).
- `Settings and diagnostics` -> renamed: the labels page test and both
  toggle-driven page tests go red (toggle lookup fails).
- `Inspect redacted settings` -> renamed: the labels page test goes red.
- `Provider changes are saved by CodexBar immediately...` -> reworded:
  same labels test goes red.
- `CLI commands` -> renamed: the labels and CLI-text page tests go red.

Kept with reason (absence assertions stay):
- `reject stdoutText + "\\n" + stderrText`: no behavior to break;
  asserts the page never concatenates raw streams outside the bounded
  protocol helpers.
- `reject return field.options.length > 0 ? 0 : -1`: stale-combo
  pattern with no runtime equivalent; the enum delegate's selected index
  comes from modelData.
- `reject console.log("CodexBar"`: absence of debug logging has no
  observable runtime behavior.

Verification: make check-feature-parity, make check-qml-logic (9/9 in
tst_provider_filters), make check-security-regressions and
make check-refresh-nonce green; tests test_secret_prompt_lifecycle
(4 tests) and test_provider_write_errors green. check-python-lint is
skipped locally (pyflakes absent; CI installs it). Full make check left
to the orchestrator.

* test(providers): replace covered lifecycle and theme greps with executed tests

Eleven static assertions on contents/ui/configProviders.qml are now
covered by executed tests; three stay with a stated reason. No
production code changes; git diff contents/ is empty.

New coverage (each shown red against its mutation before removal):

- test_runCommandFallsBackToConfigTimeoutAndExpires (ProviderCommandWiring):
  a timeout-less command stays connected at launched+59s and is retired
  with the timeout message at launched+61s. Extracts expireConfigCommands,
  hasTimedConfigCommands and handleConfigCommandTimeout.
- test_configTimeoutTimerExpiresHungCommands (ProviderFilters, live page):
  the sweep Timer, shortened to 80ms, disconnects an overdue entry and
  reports the timeout. The finder keys on the repeating one-second
  interval; desktop-style hover timers forced that precision.
- test_commandPathChangeRetiresCommandsAndClearsProviders (live page):
  a cfg_commandPath change retires in-flight commands and clears the
  roster/selection.
- test_optionsNoticeFollowsDescriptorsAvailability (live page): the options
  notice tracks providerDescriptorsUnavailable both ways.
- test_providerColorKeepsBrandIdentityPerProvider (live page): branded
  providers differ, unbranded ids share one fallback, and the fallback
  differs from a real brand.
- test_providerReadableColorKeepsContrastOnDarkBackground (live page):
  replicate-on-black keeps the >=3 non-text contrast bar.

Evidence (mutation -> red test):

- `readonly property int configCommandTimeoutMs: 60000` -> 3600000:
  test_runCommandFallsBackToConfigTimeoutAndExpires goes red (the
  launched+61s sweep no longer retires). The wiring builder regex was
  relaxed to \\d+ so the value flows into the fixture and the effect
  test, not the extractor, pins it.
- `readonly property int configSecretPromptTimeoutMs:` (formula) ->
  margin +5000 dropped: the ledger-deadline test goes red (940000 < 945000).
- `id: configCommandTimeoutTimer` -> Timer element deleted:
  test_configTimeoutTimerExpiresHungCommands goes red (no timer found).
  Renaming the unreferenced id alone stays green; the element is the pin.
- `page.expireConfigCommands(Date.now())` -> onTriggered neutered, and
  separately the running binding dropped: the same timer test goes red in
  both cases (no restart() in the test, so the binding is load-bearing).
- `onCfg_commandPathChanged: handleCommandPathChanged()` -> handler
  removed: test_commandPathChangeRetiresCommandsAndClearsProviders goes red.
- `function providerColor(value)` -> always highlight, and separately
  fallback dropped: test_providerColorKeepsBrandIdentityPerProvider goes
  red in both cases.
- `function providerReadableColor(value, background)` -> passthrough:
  test_providerReadableColorKeepsContrastOnDarkBackground goes red (raw
  near-black on black ~= 1.0).
- providers `import "ThemeContrast.js" as ThemeContrast` -> dropped:
  ProviderFilters page tests fail with ReferenceError (delegates call
  page.providerReadableColor at render).
- `Editable provider options come from CodexBar` /
  `This CodexBar version does not expose editable provider options` ->
  flag negated: test_optionsNoticeFollowsDescriptorsAvailability goes red.
  Literals stay green by design (i18n-owned); the wiring behind them is pinned.
- `ProviderConfigProtocol.commandOutcome(` -> literal success forced in
  handleToggleResult: the write-errors test goes red.

Kept with reason:

- `configSecretCommandTimeoutSeconds: 60` /
  `configSecretCommandKillAfterSeconds: 5`: bumping to 61/6 keeps the
  whole secret-prompt module green. Generated scripts run against shrunk
  fixture values and the ledger-margin invariant holds for any positive
  magnitudes, so the production defaults have no failing-first pin.
- `Component.onCompleted: Qt.callLater(reload)`: deleting the line keeps
  all providers suites green; creation always coincides with the
  cfg_commandPath injection reload and callLater coalesces the pair, so no
  executed test can distinguish the duplicate.

Verification: make check-process-lifecycle, check-theme-boundaries,
check-ui-regressions, check-qml-logic (exit 0, including the desktop-style
rerun) and check-shellcheck green; test_secret_prompt_lifecycle and
test_provider_write_errors green. check-python-lint skipped locally
(pyflakes absent; CI installs it). Full make check left to the orchestrator.

* test(providers): check control visibility, not just its row

The descriptor-kind assertions matched a control by walking up to its
RowLayout and testing that row's visibility. A control hidden inside a
visible row would still have counted, so the test could pass on a field
the page never shows the user — the opposite of what it was written to
prove. The predicate now requires both, and is named for what it checks.

Also renames the extracted secret-prompt timeout constants from `secrets`
to `secret_prompt_timeouts`. They are the 60s timeout and 5s kill-after,
not credentials; the old name made CodeQL read the fixture write as
clear-text storage of a secret.

Verified: tst_provider_filters 14/14 and test_secret_prompt_lifecycle 4/4
stay green with the stricter predicate.

Raised by CodeRabbit and CodeQL on #311.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
281014ce4fSimone committed on 9/21/2026, 9:44:23 PM· committed by GitHubparent21fb0d5
8 files changedLine totals unavailable