Skip to content

fix(cli, api): round trip the overall extraction instruction in ixp packages - #455

Open
anachili wants to merge 1 commit into
masterfrom
chi.heng/package-overall-extraction-instruction
Open

fix(cli, api): round trip the overall extraction instruction in ixp packages#455
anachili wants to merge 1 commit into
masterfrom
chi.heng/package-overall-extraction-instruction

Conversation

@anachili

@anachili anachili commented Aug 11, 2026

Copy link
Copy Markdown

An IXP project's overall extraction instruction is fed to the model as part of its prompt, so a project uploaded from a package was running a different prompt from the one the package was downloaded from — and scored differently in Measure as a result, with nothing to indicate anything had been dropped.

This surfaced on a customer promoting projects from a Test to a Production tenant with re package download / re package upload. The annotated data transferred exactly (same 10 documents, 120 annotations, 101 annotated values, field for field), but the project score fell from 91 to 79, concentrated on the identifier and name fields that most depend on the instruction to disambiguate them (InvoiceNumber 100 → 20, SupplierName 60 → 40, PolicyNumber 50 → 33, TaxAmount 100 → 67).

Why it was missed: the instruction lives on the default label group, not on the dataset, so it is not covered by the label defs the group is created with. Neither LabelGroup nor UpdateDataset modelled it, so it was dropped when the package was written and never sent when it was read back. Field group, field and field type instructions were already round tripped — this was the only level that was not.

It is also the last outstanding item of RE-7680, which named model config, the overall extraction instruction, and custom defined field types. Model config landed in #392 and field types in #394; this half never did.

What changed

  • LabelGroup gains instructions, so re package download captures it into the package's dataset.json.
  • UpdateDataset gains default_label_group_instructions, serialized as _default_label_group_instructions.
  • re package upload sends it for IXP projects alongside the model config it already restores.

Only sent when the packaged project actually has an instruction, so uploading a package downloaded before this change leaves the new project's own default in place rather than blanking it. It is applied in the existing update_dataset call rather than a new request — every dataset always has a default group from creation, so it does not need to be sent after the group's label defs.

Breaking (reinfer-client API): LabelGroup and UpdateDataset each gain a field, so struct literals of them need an extra initializer.

Test plan

  • Two unit tests added: the instruction deserializes off a dataset response, and UpdateDataset serializes it under the right key while omitting it entirely when unset.
  • cargo test -p reinfer-client — 41 passed.
  • cargo clippy --workspace --all-targets and cargo fmt --all --check clean.
  • Updated the label_groups literal in the test_create_update_dataset_custom integration test.

Not covered: the round trip against a live tenant. Worth a manual check that downloading a project with an instruction and uploading it into another tenant now reproduces the source's score.

Follow-ups, not in scope here

  • No CLI flag exposes the instruction on re update dataset — it is only set via package upload.
  • Field level business rules (rule_set on MoonFormFieldDef) are still dropped by packages.
  • Per choice instructions cannot reach the model at all: the platform hardcodes instructions="" when serializing field choices, so nothing the CLI sends would take effect.

RE-7680

…ackages

An IXP project's overall extraction instruction is fed to the model as
part of its prompt, so a project uploaded from a package was running a
different prompt from the one the package was downloaded from, and scored
differently in Measure as a result — with no indication anything had been
dropped.

The instruction lives on the `default` label group rather than on the
dataset, and neither `LabelGroup` nor `UpdateDataset` modelled it, so it
was lost when the package was written and never sent when it was read
back. The label defs the group is created with do not cover it.

Add it to both, and send it from `re package upload` alongside the model
config it already restores. Only send it when the packaged project has
one, so uploading an older package leaves the new project's own default
in place rather than blanking it.

Field group, field and field type instructions were already round
tripped; this was the only level that was not.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant