NeXus–NOMAD metainfo generator: Phase 3 (parser v2 + app v2)

[lukaspie]

Builds on #801 (Phases 1 + 2). This branch is fully rebased onto it. For the full plan, all phases, and the Architecture Decision Records, see the data-modeling repo: https://github.com/FAIRmat-NFDI/data-modeling/tree/main/nexus-metainfo (specifically here for Phase 3 implementation).

---

Summary

Phase 1+2 generated the static Python metainfo (base classes + applications). This PR adds a new parser (nexus_parser_v2) that reads HDF5 directly into the generated Python classes via their a_nexus_* annotations, and a new search app (nexus_app_v2) that mirrors the old app, but uses the new names in the schema. Both are purely additive, new entry points; nothing existing is touched or removed (for now).

Testing this branch

To test with the new schema, install pynxtools from this branch and add these entry points to your nomad.yaml (in addition to or instead of the existing nexus_schema/nexus_parser/nexus_app):

plugins:
  entry_points:
    include:
      - "pynxtools.nomad.metainfo:nexus_base_classes"
      - "pynxtools.nomad.metainfo:nexus_applications"
      - "pynxtools.nomad.parsers:nexus_parser_v2"
      - "pynxtools.nomad.apps.app_v2:nexus_app_v2"

Upload any NeXus (.nxs/HDF5) file as usual; nexus_parser_v2 matches it and produces archives using the Phase 1+2 generated classes (Entry, Xps, Arpes, …), browsable in the "NeXus v2" app.

---

Key decisions

1. One archive per NXentry, plus one Root archive per file

A NeXus file can hold multiple NXentry groups. Each becomes its own archive: archive.data is the Entry/Xps/Arpes/… instance directly. There is no wrapper section, unlike parser v1's Root container. A separate Root(Experiment) archive is also produced per file, populated from the file's own NXroot attributes (file_name, file_time, creator, NeXus_version, …) and linking every NXentry archive in that file as an ExperimentStep (resolved lazily via m_context + generate_entry_id, since sibling entries aren't necessarily already persisted). NXroot is mapped to basesections.Experiment in BASESECTIONS_MAP for this. Entry names are disambiguated as "{file stem} - {entry name}" so that entry/entry1-named groups (the overwhelming common case) stay distinguishable across files; the Root archive itself is named "{file stem} (NeXus file)".

2. Annotation-driven matching replaces XML/NXDL resolution at parse time

parser_v2.py does not re-derive the schema from NXDL at parse time at all. Instead, it it builds a _SectionIndex per generated Section class once, from the a_nexus_group/a_nexus_field annotations already baked into the Phase 1+2 classes (NeXusGroup.nx_class → SubSection, field concept name → Quantity), and uses NexusSchemaResolver only for HDF5-side concept matching (variadic names, NXdata signal/axes hints). Due to the new schema, paths read more naturally (e.g. data.instrument[0].energy_resolution.resolution, not data.ENTRY[0].INSTRUMENT[0].energy_resolution.resolution__field).

3. A new search app, nexus_app_v2

Separate from the existing NeXus app, but with the same layout. The columns/menu/dashboard are built against the v2 paths (no __field/__group suffixes). Locked to section_defs.definition_qualified_name = pynxtools.nomad.metainfo.base_classes.Entry, so it only ever shows v2-parsed entries (and the application definitions that inherit from Entry).

4. Link quantities and conflict-renamed quantities are populated correctly at parse time

Two parser-level fixes needed for the typed-link resolution and BaseSection-collision renaming (both already generator-side in #801) to actually produce correct values: a NeXusLink quantity whose target type was resolved to something other than str is now populated like a regular field (h5py already dereferences the link transparently) instead of storing the raw HDF5 link path as a string. And a quantity renamed for a BaseSection collision (e.g. name_quantity for an NXDL field literally named name) now also mirrors its value into the NOMAD-native quantity it shadows (e.g. name) via a _SectionIndex.shadow_map, so both the NXDL-faithful path and the BaseSection-native path (used by normalize(), search, etc.) carry the value.

5. results.material / chemical formula normalization wired up for v2 archives

Entry/Root instances now populate results.material (chemical formula, elements) during normalization, matching parser v1's behavior. This is needed for the new app's "Elements" menu (periodic table, formula filters) to return anything.

---

What is NOT in this PR (deferred, see #801's phase table)

Item	Phase
Migrating pynxtools-* plugin apps to the v2 schema/parser	4
Porting NORMALIZER_MAP logic to normalize() methods	5
metainfo_to_nxdl.py round-trip exporter	6
Removing the old schema.py/parser.py/NexusBaseSection/NORMALIZER_MAP	7