README.md

tree-sitter-strata

This is Strata's presentation-only tree-sitter grammar. The compiler parser in strata-syntax is the sole syntax authority. No checker, compiler pass, or language-server semantic feature may infer meaning from this tree.

The conformance gate implements proposal D-066:

  • run the strata-syntax compiler parser test over its shared 13-file valid corpus (the CLI may add checks beyond parsing);
  • require tree-sitter to parse every compiler-accepted file without ERROR or MISSING;
  • run representative compiler-rejected files through both parsers;
  • require the presentation grammar to recover with an error for each case whose tolerance is must_error.

Run from this directory:

sh
1npm install
2npm test # static 13-file corpus conformance gate (fast)
3npm run test:fuzz # mutation fuzz-corpus differential gate (~30s)
4npm run test:all # both, in sequence

tree-sitter-cli is pinned to 0.24.7. Set TREE_SITTER to an equivalent executable path when using an already-installed binary. Generated parser files are owned by this package and must be regenerated whenever grammar.js changes.

The shared valid corpus is intentionally not copied here. The harness reads it directly from ../crates/strata-syntax/tests/fixtures, ensuring compiler parser changes and presentation conformance travel together.

Current promotion status

Zero-gap: the compiler authority gate passes all 13 shared fixtures, and the presentation grammar parses all 13 cleanly (test/expected-valid-gaps.json is now empty — kept as the regression ratchet's schema, so a newly introduced gap must be recorded explicitly rather than silently passing). All three representative invalid cases are compiler-rejected and produce tree-sitter ERROR or MISSING nodes as required.

Fuzz-corpus differential gate

npm run test:fuzz (test/fuzz-conformance.mjs) implements the D-066 fuzz-corpus half of the Phase 1 gate: it mutates the same 13-file shared corpus with the identical corpus-derived mutation strategy crates/strata-syntax/tests/fuzz_roundtrip.rs uses (same LCG shape, same delete/insert/swap/truncate moves over the same byte alphabet, independent seed), then for every mutant runs both strata parse and tree-sitter parse and checks the normative coexistence rule: whenever the compiler accepts a mutant cleanly (no diagnostics), the presentation grammar must parse it without ERROR/MISSING too.

This is a regression ratchet, not the full D-066 oracle, mirroring the same bounded-promotion-gate idiom the static corpus gate used before reaching zero gaps: test/fuzz-baseline.json records the current violation count (maxViolations) alongside the seed and per-file mutant count: the gate fails only if a run exceeds that count, and reports (without failing) when a run improves on it, so the baseline can be tightened over time. At the recorded baseline, all remaining violations trace to genuine divergence between the compiler's resilient-LL statement-boundary recovery (which silently resynchronizes past certain malformed leading tokens — e.g. a stray identifier byte-merged onto a comment line — without recording a diagnostic) and tree-sitter's MISSING/ERROR recovery (which always marks the same input visibly). Closing this fully means replicating strata-syntax's exact resilient-recovery heuristics in the presentation grammar; that is out of scope for this pass and remains the tracked Phase 1 fuzz-corpus promotion work referenced in proposal/14-toolchain.md §18.