Skip to content

DIT-13504: XLIFF source text is dropped when it carries an inline placeholder - #8

Merged
laurakoye merged 2 commits into
laura/dit-13480-capture-the-raw-literal-snapshot_text-in-resource-filefrom
laura/dit-13504-xliff-source-text-is-dropped-when-it-carries-an-inline
Aug 19, 2026
Merged

DIT-13504: XLIFF source text is dropped when it carries an inline placeholder#8
laurakoye merged 2 commits into
laura/dit-13480-capture-the-raw-literal-snapshot_text-in-resource-filefrom
laura/dit-13504-xliff-source-text-is-dropped-when-it-carries-an-inline

Conversation

@laurakoye

@laurakoye laurakoye commented Aug 18, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Closes DIT-13504.

An XLIFF <source> holding an inline placeholder produced no candidate. The <target> beside it extracted fine, so a unit kept its translation and silently lost its original.

<source>Hello <g id="n">%s</g></source>     <!-- dropped -->
<target>Bonjour <g id="n">%s</g></target>   <!-- fine -->

Why: we parse with ast-grep's HTML grammar, where <source> is a void element — it can't hold children. The grammar closes it at the placeholder and orphans </source>, so innerText finds no end_tag and returns null.

Fix: when an element has no end_tag, innerText finds the closing tag in the raw source and takes the span from there.

It lives in xml.ts, not xliff.ts — innerText is where every XML-family extractor bails, so one guard covers all callers. stringsdict.ts calls it without the source argument and is unaffected.

emitTextHit also now rejects a span containing <![CDATA[. The old CDATA guard reads the element's range, which the broken parse cuts short — without it, a CDATA <source> would fire on both the AST path and the CDATA sweep.

Tests

New xliff.test.ts: <g> / <ph> / <x/> / <xliff:g> in 1.2 and <g> in 2.0, plus the cases that shouldn't move — CDATA stays a single hit, plain <source> unchanged, a genuinely unclosed <source> still dropped.

In snapshot-text.test.ts, the pinned "yields no hit at all" test inverts, and the XLIFF fixture gains a placeholder unit so the XML_FAMILY boundary sweep covers a recovered span.

Note

A self-closing placeholder leaves a double space in the value (Hello <x/> there → "Hello there"). Existing behaviour for stripped markup, unchanged here.

Verify locally

From the repo root on this branch:

mkdir -p /tmp/xliffdemo && cat > /tmp/xliffdemo/messages.xlf <<'EOF'
<?xml version="1.0"?>
<xliff version="1.2">
  <file source-language="en" target-language="fr">
    <body>
      <trans-unit id="plain">
        <source>Hello</source>
        <target>Bonjour</target>
      </trans-unit>
      <trans-unit id="named">
        <source>Hello <g id="n">%s</g></source>
        <target>Bonjour <g id="n">%s</g></target>
      </trans-unit>
      <trans-unit id="android">
        <source>Sent <xliff:g id="amt">$5</xliff:g> to <xliff:g id="who">Ada</xliff:g></source>
      </trans-unit>
      <trans-unit id="void">
        <source>Tap <x id="icon"/> to continue</source>
      </trans-unit>
      <trans-unit id="wrapped">
        <source><![CDATA[Raw & <b>bold</b>]]></source>
      </trans-unit>
    </body>
  </file>
</xliff>
EOF

cat > xliff-check.ts <<'EOF'
import { xliffExtractor } from "./src/lang/extractors/xliff";
import { readFileSync } from "node:fs";

const source = readFileSync(process.argv[2], "utf8");

(async () => {
  const hits = await xliffExtractor.extract({ source, kind: "xml" });
  for (const h of hits) {
    const [unit, side] = h.context.identifiers;
    const found = source.includes(h.snapshotText) ? "ok " : "MISSING";
    console.log(`${found}  ${`${unit}/${side}`.padEnd(16)} ${JSON.stringify(h.value).padEnd(30)} snapshot=${JSON.stringify(h.snapshotText)}`);
  }
  console.log(`\n${hits.length} candidates, ${hits.filter((h) => !source.includes(h.snapshotText)).length} with a span not found in the file`);
})();
EOF

npx tsx ./xliff-check.ts /tmp/xliffdemo/messages.xlf

The script has to sit in the repo root — the relative import resolves from there.

Expected — 7 candidates:

ok   plain/source     "Hello"                        snapshot="Hello"
ok   plain/target     "Bonjour"                      snapshot="Bonjour"
ok   named/source     "Hello %s"                     snapshot="Hello <g id=\"n\">%s</g>"
ok   named/target     "Bonjour %s"                   snapshot="Bonjour <g id=\"n\">%s</g>"
ok   android/source   "Sent $5 to Ada"               snapshot="Sent <xliff:g id=\"amt\">$5</xliff:g> to <xliff:g id=\"who\">Ada</xliff:g>"
ok   void/source      "Tap  to continue"             snapshot="Tap <x id=\"icon\"/> to continue"
ok   /source          "Raw & <b>bold</b>"            snapshot="Raw & <b>bold</b>"

7 candidates, 0 with a span not found in the file

MISSING in the first column is the acceptance criterion — every span must be findable verbatim in the file. The CDATA unit is the one to watch for double-emission: it appears once, not twice.

Before the fix — 4 candidates. Swap in the base version of the one file and rerun:

git checkout origin/laura/dit-13480-capture-the-raw-literal-snapshot_text-in-resource-file -- src/lang/extractors/xml.ts
npx tsx ./xliff-check.ts /tmp/xliffdemo/messages.xlf
git checkout HEAD -- src/lang/extractors/xml.ts

Every <source> carrying a placeholder disappears — named, android and void — while their <target> siblings come through fine. That's the silent partial failure this fixes.

Cleanup: rm xliff-check.ts && rm -rf /tmp/xliffdemo

@laurakoye laurakoye changed the title Laura/dit 13504 xliff source text is dropped when it carries an inline DIT-13504: XLIFF source text is dropped when it carries an inline placeholder Aug 18, 2026
@laurakoye
laurakoye changed the base branch from laura/dit-13480-capture-the-raw-literal-snapshot_text-in-resource-file to laura/dit-13503-candidate-ids-hash-the-line-number-instead-of-the-occurrence August 18, 2026 21:39
@laurakoye
laurakoye changed the base branch from laura/dit-13503-candidate-ids-hash-the-line-number-instead-of-the-occurrence to laura/dit-13480-capture-the-raw-literal-snapshot_text-in-resource-file August 18, 2026 21:41
@laurakoye
laurakoye force-pushed the laura/dit-13504-xliff-source-text-is-dropped-when-it-carries-an-inline branch from 9c807f4 to 3a66a54 Compare August 18, 2026 21:41
@laurakoye
laurakoye requested a review from jholiga August 18, 2026 21:50
@laurakoye
laurakoye merged commit 3c1132d into laura/dit-13480-capture-the-raw-literal-snapshot_text-in-resource-file Aug 19, 2026
3 checks passed
@laurakoye
laurakoye deleted the laura/dit-13504-xliff-source-text-is-dropped-when-it-carries-an-inline branch August 19, 2026 15:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jholiga jholiga jholiga approved these changes

Assignees

@jholiga jholiga

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@laurakoye @jholiga