Expand modern CSS grammar coverage

[kristofer-baxter]

Expand modern CSS grammar coverage

Coverage

This broad update adds modern at-rule, function, property, selector, and media-query coverage. The main review risk is recovery around the parenthesized regions, which gets most of the detail below.

The grammar diff adds dedicated coverage for:

• @container size, named, style(), and scroll-state() queries, including logical operators and size or scroll-state features
• @scope start and limit preludes, @starting-style, and @property
• color functions, env(), anchor(), anchor-size(), and additional math and value functions
• selector-list functional pseudo-classes, :nth-child(... of ...), :state(), and argument-specific functional pseudo-elements
• current interaction, preference, viewport-segment, dynamic-range, and related media features and values
• anchor positioning, scroll and view timelines, view transitions, containment, text, and layout properties

@property accepts digit-prefixed custom-property names. Ordinary identifiers added by this change use the stricter first-character pattern.

Everything lives in grammars/css.cson; the specs exercise complete scope arrays with deepStrictEqual.

Recovery guard

Dedicated parenthesized regions improve coverage, but an unclosed region can retain its scopes beyond the prelude. To preserve recovery, all 36 reachable regions end at { only when the rest of the line does not close the parenthesis.

A plain [^)]* is not enough. A ) inside a comment, string or escape closes nothing, and counting it suppressed recovery that origin/main performed: @container (width > 1px{ /* ) */ stayed open. Trailing comments containing parentheses are ordinary in real stylesheets.

Instead, the lexical bail-out skips complete and unterminated comments, quoted strings, and escapes while looking for a real closing parenthesis. Escapes include incomplete ones. If a line ends in an odd number of backslashes, bare or inside an unterminated string, no complete-escape alternative matches; the inherited newline-escape rule would otherwise hold the prelude open across the line break.

Every reachable nested region needs the guard because an outer end cannot be evaluated while an inner rule is active.

A recovery oracle checks every prefix of 24 representative preludes, with { and { also appended: 2,412 cases total, 1,715 of them inside the guard contract. Against origin/main, this branch has 186 improvements, 42 regressions, and 147 residual contract violations also present on main. All 42 regressions are @scope ( inputs containing an unclosed [.

A differential over legal snippets covering every guarded construct, plus Bootstrap and 176k lines of Tailwind output, found no genuine losses.

Known limitation

@scope ([data-x{ leaks, and an unterminated url(a{ inside a prelude stays open until a later ) or guard-eligible {.

That first one is main's existing attribute-selector leak: a[data-x{ also leaks there. New @scope coverage exposes it in one additional context. I tried guarding the attribute selector and reverted it because that broke a legal multi-line attribute selector covered by a pre-existing passing test.

The url() case has the same shape. url()'s argument pattern, [^'")\s]+, is byte-identical to main's and consumes the { before the region's end can be tested. That means a { background: url(x{ leaks on main too, and the region stays open across however many lines precede the next ) or guard-eligible {, not one.

I also tried tightening the class to exclude {. It splits the legal value url(https://x/a{b}) into three tokens and drops the brace out of variable.parameter.url.css. An unquoted url-token may legally contain {, so I left the pattern alone.

Both leaks are inherited rather than introduced. The new @scope and @container preludes reuse existing selector and function rules, exposing their pre-existing leaks in contexts where main never applied them.

On balance the branch removes four pre-existing leak cases while inheriting these. It does not solve every malformed-input leak, and I am not claiming it does.

Two narrower cases remain. A closing parenthesis inside a nested parenthesised construct, as in @container (width > 1px{ url("x)"), still suppresses recovery because balanced-parenthesis tracking is beyond a line-local lookahead. Separately, @document's url-prefix(), domain() and regexp() do not recover with or without the guard: their argument patterns consume the { before the region's end is tested, and main behaves identically.

There is also an @container-only case. <general-enclosed> accepts <any-value>, so @container (future: "a{b") is legal and the brace inside the string is not a body brace. The bail-out inspects only the text from the candidate brace onwards; with no string rule active, it reads the closing quote as an opening one, treats the real ) as shielded, and opens the body early.

@media and @supports are fixed by giving their condition regions a #condition-string rule, which keeps the enclosing end from being evaluated inside a string. A new test covers both. #condition-string is a condition-local copy of #string rather than #string itself, differing only in that its newline escape ends at ^ rather than the shared rule's ^(?<!\G).

The shared form cannot match while the condition is still open, so a legal @media (future: "a{\ line continuation swallowed the rest of the stylesheet. The local copy fixes that while leaving string tokenization in every other context byte-for-byte as it is on main, checked across 15 line-continuation constructs. @supports routes the same string through the shared property-value context, where main swallows it too; that case is left at parity rather than fixed.

I tried the same fix for @container and reverted it. There, an unterminated string would shield a malformed { from the bail-out and reintroduce 68 leak cases that recover today. @container therefore mis-tokenizes this legal input exactly as origin/main does. It is not a differential regression, but it is not fixed either.

One last case is specific to the new style() coverage. CSS Conditional Rules 5 defines a style query's <style-feature-value> as a <declaration-value>, so a balanced curly block may legally span lines inside it. The line-local guard treats that opening { as the container body when the ) is on a later line, so @container style(--x: { followed by a block is mis-tokenized.

This matches origin/main's generic at-rule header, so it is not a differential regression, though the dedicated style() region does not cover it. Removing the guard from style() would trade this edge case for all recovery on malformed style queries. I kept it, but welcome pushback because this is the judgment call I am least certain about.

Scope naming

color-mix() moves from meta.function.misc.css to meta.function.color.css, the scope main already gives every other color function, including rgb(). The function name keeps support.function.misc.css.

A theme targeting the meta.function.misc.css container will no longer match color-mix() or its contents. I think the reclassification is the better fit, but maintainers should push back if preserving the old container matters more.

var() and custom function trade-off

var() and custom function calls are not guarded. Both are declaration values, and <declaration-value> admits a balanced curly block that legally spans lines, as in var(--fb, { ... }) and --x: --foo({ ... });. Applying the guard truncated those legal values. origin/main scopes them correctly, so the guard was a regression there.

The legal --foo({ and malformed --foo(a{ are lexically identical within a single line; no line-local test can separate them. Legal CSS wins. main does not recover var(--x{ either, and mutation-verified tests pin both legal forms.

Related work and credit

Numbers refer to open pull requests in microsoft/vscode-css. They are written without links, bare issue notation, or @mentions, so staging this branch on a fork does not notify contributors or cross-post into upstream threads.

• upstream PR 15 by wileycoyote78 contributes @container at-rule support and is the largest overlap with this branch. This branch covers the same gap as part of broader container-query grammar, recovery, and regression coverage.
• upstream PR 32 by wesbos contributes animation-timeline, animation-range, animation-range-start, and animation-range-end. I removed those four properties from this branch and left them to upstream PR 32.
• upstream PR 43 by OverflowCat contributes digit-prefixed identifiers. This branch applies that behavior to @property custom-property names while adding six uses of the strict first-character pattern for ordinary identifiers.
• upstream PR 36 by romainmenke contributes @page handling, and upstream PR 37 contributes CSS4 system colors. I checked both; neither overlaps this branch, and this change neither duplicates nor supersedes them.
• upstream PR 50 by rhukster contributes bare top-level declarations. This branch's declaration-value region is local to style(), so it does not overlap that work.

Verification

• 3 files changed, 2,120 insertions, 90 deletions, in one commit

• 302 tests total: 296 passing, 0 failing, 6 skipped under Node 20.18.0 after a clean npm ci

• 85 it( blocks added

• origin/main also has 6 skipped tests, so this branch adds no skips

• guard-site mutation campaign, at two strengths, each applied to one of the 36 sites at a time:

  • deleting the bail-out alternative so the region can never recover: 35 of 36 caught by a failing test. The one survivor, @document's argument functions, cannot recover with or without the guard, as on main.
  • weakening the guard body to the naive [^)]*$ it replaces: only 10 of 36 caught. The 36 guard bodies are byte-identical, so this says which regions have a test exercising a shielded ), not that there are 26 differing implementations. It also means most sites would not catch a regression to the naive form on their own. The distinction is exercised directly at the sites listed above.

  These totals come from a harness that is not part of this branch, so they are not reproducible from the repository as submitted.

• malformed-input comparison: main leaks on 8 of 10 sampled cases; this branch fixes 4 and introduces 0

• backward compatibility, checked mechanically against origin/main: no CSS identifier is lost (244 gained), no scope name is removed (32 added), no #include reference dangles or goes unreferenced, and all 311 begin/end/match patterns compile with every capture index in range

• scope-loss differential over real stylesheets: across 176,011 lines of Tailwind 2.2.19 output, no character loses a scope main assigned. Bootstrap 5.3.3 differs in five constructs, all gains or corrections: prefers-reduced-motion (32), reduce (31) and no-preference (1) gain media-feature and value scopes instead of plain header text, and ::file-selector-button (6) and :placeholder-shown (4) are corrected from entity.name.tag.custom.css to a pseudo-element and a pseudo-class

• @container and @scope preludes now carry meta.at-rule.container.header.css and meta.at-rule.scope.header.css respectively, rather than the generic meta.at-rule.header.css, matching the naming main already uses for @media and @supports

• no catastrophic backtracking: six adversarial inputs, including 200 nested calc( and a 5,000-character prelude, tokenize in 244 ms or less

Performance

I loaded both grammars in one process and tokenized them alternately, taking min-of-7 per grammar, and recorded six full runs. The machine started idle at a load average of 3.2, but the benchmark drives its own load up over the sequence: sampled per-run load ranged from 3.8 to 13.2. I report every run, including the ugly ones.

Below are the median wall time per grammar, the median of the six paired per-run percentage deltas, and the full range of those paired deltas.

workload	origin/main	branch	median delta	range across 6 runs
Bootstrap 5.3.3 (12k lines)	547 ms	581 ms	+4.8%	+3.1% to +6.6%
Tailwind 2.2.19 (176k lines)	9,444 ms	9,914 ms	+5.3%	+4.2% to +7.1%
container-query corpus x1000	4,236 ms	1,484 ms	-65.2%	-65.4% to -64.1%
20 KB single line, many ;	527 ms	533 ms	+0.5%	-0.7% to +2.3%
20 KB single line, many {	3,338 ms	3,544 ms	+5.5%	-2.9% to +16.8%
minified, 52 KB single line	494 ms	492 ms	+0.1%	-0.5% to +2.3%
scroll-state, 20 KB unclosed	2 ms	16 ms	+582%	+575% to +605%

Four of the seven workloads have the same sign in all six runs and can be read as measured effects: Bootstrap +4.8%, Tailwind +5.3%, the container-query corpus -65.2%, and the scroll-state case +582%. Neither corpus contains a single @container rule or container property, so the first two measure the cost on ordinary CSS, at roughly 5%.

Three ranges cross zero and resolve nothing. The brace-heavy synthetic line is the noisiest workload measured; its +5.5% median should not be read as a cost.

The scroll-state figure is a large ratio on a small absolute base: 2 ms to 16 ms on a 20 KB file consisting of a single unterminated scroll-state() query. It reflects repeated guard evaluation against a line that never closes.

Be aware that the synthetic workloads above are generated fixtures, and they are not part of this branch. Only the Bootstrap and Tailwind rows can be reproduced from public sources.

I also compared the lexical bail-out with the simpler [^)]*$ form, holding everything else in the grammar constant, over three further interleaved runs. Every workload landed between -6.0% and +2.7%, with most deltas negative, so no workload showed the lexical form to be measurably more expensive. The roughly 5% cost on ordinary CSS comes from the added region coverage, not the guard's complexity.

Container queries come out about 3x faster because precise regions replace the generic fallback's retries.

[kristofer-baxter]

Thanks for flagging the confusing wording. I intended to acknowledge the related identifier edge case in microsoft#43, not imply that this branch contains your code, depends on your PR, or needs action from you. I have clarified the description and removed the premature direct mentions.

[kristofer-baxter]

Superseded by the upstream pull request: microsoft#57

This fork PR was staging only. All review and discussion should happen upstream.