fix(mobile): keep the opening words of local iPhone dictation#31801

merged1
voice/ios-dictation-languages
voice/ios-dictation-capture
opened7d agobyJJulius Marminge
View on GitHubTwo-way sync enabledPR last imported 7d ago

Problem

On the first on-device iPhone dictation in a language, T3 waited for Apple’s speech model before starting the microphone. Speaking immediately after tapping the mic could turn “one, two, three” into “two, three.”

Change

The iPhone records locally as soon as microphone permission and audio setup allow. Stopping waits for the model and transcribes the saved clip. Cancelling deletes the clip and releases the session even if Apple never answers. A subsequent recording can finish without waiting for the cancelled request.

Scope and approval

This is the opening-word fix from @t3dotgg’s #25808, rebuilt at the maintainer’s request. Only on-device iPhone dictation opts into early capture. Android, web and environment dictation keep their existing preparation order and consent checks. The original report, #17888, was closed with the label needs more info.

Verification

On 1ea9d03, all 63 focused iOS, Android and shared-controller tests passed. The iOS tests hold Apple’s preparation pending: capture starts, Stop waits, and Cancel deletes the file. A second recording reaches the draft before the first native request resolves. check:changed passed, including typechecking seven packages.

@jokaperes verified that the source branch kept the first word on an iPhone 17 in Brazilian Portuguese. The rebuilt head hasn’t been tested on a physical iPhone. To check it, select This device, use a language whose model isn’t installed, tap the composer mic and immediately say “one, two, three.” All three words should appear after Stop. On another cold start, cancel and dictate again; only the second clip should reach the draft.

GPT 6 Astra · Codex in T3 Code

Checks

25 successful · 13 skipped
All checks passed
1ea9d03

Added 1 commit

  • fix(mobile): keep the opening words of local iPhone dictation1ea9d03
Synced from GitHub

Approvability

Verdict: Approved at 1ea9d03

Macroscope's review found this PR approvable — Local iPhone dictation can begin capturing audio while its language model prepares, preserving opening words. Existing permission and cancellation safeguards remain, with no change to product defaults or transcription destinations.

You can add or adjust custom eligibility rules. Learn more.

Synced from GitHub

Explain Change →

Walkthrough

The changes allow live transcription to return no text and update recording completion to fall back to file transcription. iOS model preparation now runs asynchronously and exposes the supported locale. Tests cover preparation failures, cancellation, and recording during preparation.

Changes: Voice transcription

Layer / File(s)Summary
Transcription contract and recording completion
packages/client-runtime/src/voice-input/transcription.ts, packages/client-runtime/src/voice-input/controller.ts, packages/client-runtime/src/voice-input/controller.test.ts, apps/mobile/src/native/voiceTranscription.android.test.ts
PreparedVoiceTranscription.start can return null. Recording completion falls back to file transcription when the live result is absent or has no text. Tests check unavailable preparation and Android recorder preparation call counts.
iOS model preparation and dictation
apps/mobile/src/native/voiceTranscription.ios.ts, apps/mobile/src/native/voiceTranscription.ios.test.ts
iOS preparation resolves the supported locale asynchronously. Tests cover recording during preparation, cancellation and failure before or after stopping, and a subsequent successful dictation.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix

Sequence Diagram(s)

mermaid
1sequenceDiagram
2 participant VoiceInputController
3 participant Recorder
4 participant prepareVoiceTranscription
5 participant AppleSpeechModel
6 VoiceInputController->>Recorder: Start recording
7 VoiceInputController->>prepareVoiceTranscription: Start transcription preparation
8 prepareVoiceTranscription->>AppleSpeechModel: Prepare speech model and locale
9 AppleSpeechModel-->>prepareVoiceTranscription: Return supported locale
10 VoiceInputController->>Recorder: Stop recording
11 VoiceInputController->>prepareVoiceTranscription: Transcribe recording
12 prepareVoiceTranscription->>AppleSpeechModel: Transcribe with supported locale
13 AppleSpeechModel-->>VoiceInputController: Return recognized text

Possibly related PRs

Suggested labels: bugfix, size:M

Suggested reviewers: pixpmusic

Merge Risk: External image Minimal · up to 1ea9d

Local iPhone dictation now records while Apple prepares the speech model, and cancellation releases the session. No actionable merge-blocking risk was found. The only open note is a style preference about a test import path.

Pre-merge checks: Passed 4

Check nameStatusExplanation
Title checkPassedThe title clearly describes the iPhone dictation fix.
Linked Issues checkPassedCheck skipped because no linked issues were found for this pull request.
Out of Scope Changes checkPassedCheck skipped because no linked issues were found for this pull request.
Description checkPassedThe description is mostly complete. It explains the problem, change, scope, and verification. The scope section should link directly to the maintainer’s approval comment.
Finishing Touches

Generate docstrings

  • Commit to this branch
  • Create a new PR
Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
CcoderabbitaiBotreviewed7d ago
Synced from GitHub
External image Additional comments: 1

apps/mobile/src/native/voiceTranscription.ios.test.ts: 1

5-5: External image Trivial · Maintainability & Code Quality · Quick win

External image Unverified finding Verification ran but could not confirm this finding. It is shown for review, not as a verified issue.

Import the test reset helper through a package export or a shared path.

Line 5 imports resetVoiceInputGlobalsForTests with a relative path that crosses package boundaries into packages/client-runtime/src. @t3tools/client-runtime/voice-input is already imported on line 4, and the package exports it. Export the helper from voice-input/index.ts and import it from the package. This avoids a fragile path and keeps the package boundary intact.

♻️ Proposed change

diff
1-import { VoiceInputController, VoiceTranscriptionError } from "@t3tools/client-runtime/voice-input";
2-import { resetVoiceInputGlobalsForTests } from "../../../../packages/client-runtime/src/voice-input/controller";
3+import {
4+ VoiceInputController,
5+ VoiceTranscriptionError,
6+ resetVoiceInputGlobalsForTests,
7+} from "@t3tools/client-runtime/voice-input";
External image Review info

External image Run configuration

External image Commits

Reviewing files that changed from the base of the PR and between e2c1aba9de4a5a3c02ca774506340160ec828da9 and 1ea9d0312e61518ec9ba36ecc740d6f330c3166c.

External image Files selected for processing: 6

  • apps/mobile/src/native/voiceTranscription.android.test.ts
  • apps/mobile/src/native/voiceTranscription.ios.test.ts
  • apps/mobile/src/native/voiceTranscription.ios.ts
  • packages/client-runtime/src/voice-input/controller.test.ts
  • packages/client-runtime/src/voice-input/controller.ts
  • packages/client-runtime/src/voice-input/transcription.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 6 remain after this review.

I kept the reset helper out of the public voice-input API. It clears the active recording and transcription guards without stopping the underlying work; it is for isolating tests only. The Android adapter and mobile voice-session tests already import it directly from the controller. I also updated the description to distinguish the original issue from the maintainer-approved scope and to identify what was tested on this head.