Fix matrix/TRC ICC input bypass: dead D65 gate, wrong PCS reference, no-op OETF

docs/PIPELINE.md

@@ -151,8 +151,9 @@ Here is what happens to your image. We apply these steps in order, passing the b
     The plane covers the printed frame only, because the enlarger projects the crop: a rebate or scanner surround blurred into the mask prints as a vignette the negative does not have. It is placed back at the crop, edge-replicated outside so the crop tool's full-frame preview has no seam. Hidden on the transparency transfer path, which takes no dodge/burn map. Instruments read the unmasked negative, as they already do under a dodge.
 *   **Output**: converts print density back to **scene-linear** reflectance (transmittance):
     $$I_{out} = 10^{-D}$$
-    *   **Paper Black** (`paper_black`, off): off applies black point compensation, the same idea as ICC relative-colorimetric soft-proofing. A reflection print's D-max ($2.3$) floors reflectance at $10^{-2.3} \approx 0.005$, but the adapted eye reads paper black as black, so the display should too; on preserves the paper's lifted D-max instead. With compensation, the default, each channel becomes $I_{out} = (I - t_b) / (1 - t_b)$, clamped at $0$, where $t_b = 10^{-D_b}$ and $D_b$ is the physical $D_{max}$, or $D_{max} + \text{toe}_{ch} \cdot 0.90$ when that layer's toe is negative. The curve reaches $D_{max}$ only asymptotically, so a **negative toe raises the clip point** into the shadows, which is what makes exact $0$ reachable and "negative toe deepens blacks" literal. A lifted toe and per-layer shadow casts survive because the reference is the *physical* $D_{max}$, not $D_{max,eff}$. A negative per-layer toe trim, with compensation on, tints the deepest black.
-    *   **Note**: the pipeline is **scene-linear internally**. The exposure stage emits linear light, and every creative stage (Local Contrast, Lab, Toning, Finish) operates on it. The working-space OETF, the **Adobe RGB (1998) TRC**, a pure $563/256 \approx 2.199$ power with no linear segment, is applied only as the final engine step, so it composes correctly with the Adobe RGB ICC profile at the display and export boundary. The GPU keeps a single encoded perceptual region: exposure → clahe encoded → lab decodes back to linear.
+    *   **Paper Black** (`paper_black`, off): off applies black point compensation, the same idea as ICC relative-colorimetric soft-proofing. A reflection print's D-max ($2.3$) floors reflectance at $10^{-2.3} \approx 0.005$, but the adapted eye reads paper black as black, so the display should too. On preserves the paper's lifted D-max instead. With compensation (the default), each channel becomes $I_{out} = (I - t_b) / (1 - t_b)$, clamped at $0$, where $t_b = 10^{-D_b}$ and $D_b$ is the physical $D_{max}$, or $D_{max} + \text{toe}_{ch} \cdot 0.90$ when that layer's toe is negative. The curve reaches $D_{max}$ only asymptotically, so a **negative toe raises the clip point** into the shadows. That is what makes exact $0$ reachable ("negative toe deepens blacks", literally). A lifted toe and per-layer shadow casts survive because the reference is the *physical* $D_{max}$, not $D_{max,eff}$. A negative per-layer toe trim (with compensation on) tints the deepest black.
+    *   **Note**: The pipeline is **scene-linear internally**. The exposure stage emits linear light and every creative stage (Local Contrast, Retouch, Lab, Toning, Finish) operates on it. The working-space OETF (the **Adobe RGB (1998) TRC**, a pure $563/256 \approx 2.199$ power with no linear segment) is applied only as the final engine step (the output transform), so it composes correctly with the Adobe RGB ICC profile at the display/export boundary. Retouch is a perceptual op, so the CPU brackets that stage through the OETF (encode → heal → decode); the GPU keeps a single encoded perceptual region (exposure → clahe/retouch encoded → lab decodes back to linear).
+    *   **Input ICC overrides primaries only, never the tone curve.** The rendered buffer's TRC is always this working-space OETF — never the selected Input ICC's own declared TRC, which would need the file to actually be encoded that way, and it isn't. For a **matrix/TRC** profile (`infrastructure/display/icc_profile.py`), the boundary transform extracts just its primaries (Bradford-adapted to D65, via its `chad` tag when present) and applies them as a plain matrix; the profile's declared TRC is inert. Concretely, `sRGB-*-g10.icc` (linear TRC) and `sRGB-*-srgbtrc.icc` (sRGB TRC) — identical primaries, different declared TRC — now render **identically** as Input ICC. A **LUT** (A2B0/B2A0) profile instead runs through the full CMS transform, so its authored input curves *are* honoured — that's where a profile like the bundled narrowband `RGBScan.icc` applies its compensation, which is authored against this exact boundary encoding.
 
 ### Automatic helpers
 

docs/USER_GUIDE.md

@@ -797,7 +797,7 @@ The primary **Export** action. Its chevron menu picks the scope: current frame (
 
 *   **Format**: `JPEG`, `TIFF`, `PNG`, `JPEG XL`, or `WebP`, with quality or effort options per format. TIFF is always zlib-compressed. **JPEG XL supports only `sRGB`, `P3 D65`, `Rec 2020` or `Greyscale`** for Color Space: it tags color with compact enumerated values rather than an embedded ICC profile, and NegPy's JXL encoder cannot carry an arbitrary one, so `Adobe RGB`, `ProPhoto RGB` and a custom Output ICC are rejected with an error. Pick a supported space or a different format.
 *   **Color Space**: `Same as Source`, `sRGB`, `Adobe RGB`, `ProPhoto RGB`, `P3 D65`, `Rec 2020`, or `Greyscale` (true B&W output).
-*   **Input / Output ICC**: soft-proof against, and optionally embed, an ICC profile. Output is the destination profile (default); Input treats the profile as the source, for when a scan's profile is known but untagged. Not available for JPEG XL output; see the Format note above.
+*   **Input / Output ICC**: soft-proof against, and optionally embed, an ICC profile. Output is the destination profile (default); Input treats the profile as the source, for when a scan's profile is known but untagged. Not available for JPEG XL output; see the Format note above. Input overrides **primaries only** — the tone curve is always the pipeline's own, so a matrix-style profile's declared TRC is ignored (two profiles with identical primaries but different TRCs render identically); a LUT-style profile's own input curves are still honoured.
 *   **Paper Aspect Ratio**: final print ratio, or *Original* (no resize).
 *   **Resolution**: *Original* (full RAW resolution), *Print* (long-edge **Size** in cm plus **DPI**), or *Pixels* (long-edge **px**; the short side follows the paper ratio).
 *   **Destination**: **Filename Pattern** (a Jinja2 template with export settings plus Metadata fields such as roll, camera and film; see [TEMPLATING.md](TEMPLATING.md)), an **Overwrite** toggle, and the output location (subfolder of source, same as source, or an absolute **Export Path** with a browse button). Destination applies to all three output intents: with **Linear** selected, Format, Size and Color hide (a raw dump has no use for them) and Destination stays.

negpy/infrastructure/display/icc_profile.py

@@ -0,0 +1,135 @@
+"""Low-level ICC profile parsing for matrix/TRC detection and primaries extraction.
+
+Reads the tag table and specific tag payloads using pure struct parsing —
+no dependency on PIL.ImageCms or lcms2 for the introspection itself.
+"""
+
+import struct
+from typing import Optional
+
+import numpy as np
+
+# ICC's PCS (profile connection space) is always D50-relative, so every conformant
+# matrix/TRC profile's rXYZ/gXYZ/bXYZ colorant tags are D50-adapted regardless of the
+# profile's actual native illuminant (D65 for sRGB/Adobe RGB, D60 for ACES, D50 for
+# ProPhoto, ...). Recovering D65-referenced primaries — needed to combine with this
+# codebase's D65-referenced working-space math — is therefore always a two-step
+# chromatic adaptation: PCS(D50) -> native white -> D65. `chad`, when present, gives
+# the exact native white (inverting it alone only reaches D65 by coincidence, for a
+# D65-native profile); its absence means the native white is unknown and D50 is the
+# only assumption available.
+_D50_XYZ = np.array([0.9642, 1.0000, 0.8249], dtype=np.float64)
+_D65_XYZ = np.array([0.95047, 1.00000, 1.08883], dtype=np.float64)
+
+# Bradford cone-response matrix (Lindbloom), used to build a chromatic adaptation
+# transform between any two reference whites.
+_BRADFORD_CONE = np.array(
+    [
+        [0.8951000, 0.2664000, -0.1614000],
+        [-0.7502000, 1.7135000, 0.0367000],
+        [0.0389000, -0.0685000, 1.0296000],
+    ],
+    dtype=np.float64,
+)
+
+
+def _bradford_adaptation(src_white: np.ndarray, dst_white: np.ndarray) -> np.ndarray:
+    """3x3 Bradford chromatic adaptation matrix mapping XYZ relative to ``src_white``
+    to XYZ relative to ``dst_white``."""
+    lms_src = _BRADFORD_CONE @ src_white
+    lms_dst = _BRADFORD_CONE @ dst_white
+    scale = np.diag(lms_dst / lms_src)
+    return np.linalg.inv(_BRADFORD_CONE) @ scale @ _BRADFORD_CONE
+
+
+def _read_tag_table(data: bytes) -> dict[bytes, tuple[int, int]]:
+    """Parse the ICC tag table into {signature: (offset, size)}."""
+    if len(data) < 132:
+        return {}
+    tag_count = struct.unpack_from(">I", data, 128)[0]
+    tags: dict[bytes, tuple[int, int]] = {}
+    for i in range(tag_count):
+        base = 132 + i * 12
+        if base + 12 > len(data):
+            break
+        sig = data[base : base + 4]
+        offset, size = struct.unpack_from(">II", data, base + 4)
+        tags[sig] = (offset, size)
+    return tags
+
+
+def _read_xyz_tag(data: bytes, offset: int, size: int) -> Optional[np.ndarray]:
+    """Read an XYZType tag (ICC spec §10.31) → (3,) float64 array."""
+    if size < 20:
+        return None
+    x = struct.unpack_from(">i", data, offset + 8)[0] / 65536.0
+    y = struct.unpack_from(">i", data, offset + 12)[0] / 65536.0
+    z = struct.unpack_from(">i", data, offset + 16)[0] / 65536.0
+    return np.array([x, y, z], dtype=np.float64)
+
+
+def _read_chad_tag(data: bytes, offset: int, size: int) -> Optional[np.ndarray]:
+    """Read a chromaticAdaptationTag (s15Fixed16ArrayType, ICC spec §10.8) → (3, 3) float64."""
+    if size < 8 + 9 * 4:
+        return None
+    vals = struct.unpack_from(">9i", data, offset + 8)
+    return np.array(vals, dtype=np.float64).reshape(3, 3) / 65536.0
+
+
+def is_matrix_trc_profile(data: bytes) -> bool:
+    """True when the profile is a matrix/TRC (shaper-matrix) type.
+
+    Requires rXYZ/gXYZ/bXYZ colorant tags and at least one TRC tag,
+    and must NOT have A2B0/B2A0 LUT tags.
+    """
+    tags = _read_tag_table(data)
+    has_colorants = all(sig in tags for sig in (b"rXYZ", b"gXYZ", b"bXYZ"))
+    has_trc = any(sig in tags for sig in (b"rTRC", b"gTRC", b"bTRC"))
+    has_lut = any(sig in tags for sig in (b"A2B0", b"B2A0"))
+    return has_colorants and has_trc and not has_lut
+
+
+def extract_primaries_matrix(data: bytes) -> Optional[np.ndarray]:
+    """Extract the 3x3 D65-referenced RGB→XYZ matrix from rXYZ/gXYZ/bXYZ colorant tags.
+
+    The raw tag values are PCS-relative (D50-adapted, per the ICC spec). `chad`
+    records the profile's native-white -> PCS(D50) adaptation actually used, so
+    `inv(chad)` recovers the *native* reference — D65 only by coincidence, for a
+    D65-native profile (sRGB, Adobe RGB, ...). For anything else (ACES/ACEScg: D60,
+    ProPhoto: D50, ...) that native reference then needs its own adaptation to D65.
+    Without `chad` (typical of v2 profiles) the native white is unknown, so D50 is
+    assumed. Either way the result always lands on D65 — see
+    `test_extracted_primaries_are_d65_referenced` for the profiles this matters for.
+
+    Returns a (3, 3) float64 array where each column is one primary's XYZ
+    tristimulus, or **None if the colorant tags are missing** (malformed size, or
+    absent entirely — e.g. a GRAY-space profile, which has no rXYZ/gXYZ/bXYZ to
+    extract). This is a load-bearing part of the contract, not incidental: callers
+    (`ImageProcessor._try_matrix_bypass`) rely on `None` here to fall through to the
+    full-CMS path rather than crash or fabricate a matrix — `is_matrix_trc_profile`
+    is expected to have already filtered out non-RGB profiles first, but this
+    function must stay safe to call regardless.
+    """
+    tags = _read_tag_table(data)
+    cols = []
+    for sig in (b"rXYZ", b"gXYZ", b"bXYZ"):
+        entry = tags.get(sig)
+        if entry is None:
+            return None
+        xyz = _read_xyz_tag(data, *entry)
+        if xyz is None:
+            return None
+        cols.append(xyz)
+    m_pcs = np.column_stack(cols)
+
+    m_native, native_white = m_pcs, _D50_XYZ
+    chad_entry = tags.get(b"chad")
+    if chad_entry is not None:
+        chad = _read_chad_tag(data, *chad_entry)
+        if chad is not None:
+            try:
+                chad_inv = np.linalg.inv(chad)
+                m_native, native_white = chad_inv @ m_pcs, chad_inv @ _D50_XYZ
+            except np.linalg.LinAlgError:
+                pass
+    return _bradford_adaptation(native_white, _D65_XYZ) @ m_native

negpy/kernel/image/logic.py

@@ -182,6 +182,37 @@ def working_oetf_decode(img: np.ndarray) -> np.ndarray:
 _LAB_KAPPA = 7.787
 
 
+@parallel_njit(cache=True, fastmath=True)
+def _matmul_3x3_kernel(px: np.ndarray, m: np.ndarray) -> np.ndarray:
+    """Row-parallel 3x3 matrix multiply over an (N, 3) pixel array."""
+    n = px.shape[0]
+    out = np.empty((n, 3), dtype=np.float32)
+    for i in prange(n):
+        r, g, b = px[i, 0], px[i, 1], px[i, 2]
+        out[i, 0] = m[0, 0] * r + m[0, 1] * g + m[0, 2] * b
+        out[i, 1] = m[1, 0] * r + m[1, 1] * g + m[1, 2] * b
+        out[i, 2] = m[2, 0] * r + m[2, 1] * g + m[2, 2] * b
+    return out
+
+
+def apply_primaries_transform(img: np.ndarray, src_to_xyz: np.ndarray) -> np.ndarray:
+    """Apply a primaries-only colour transform (no TRC re-decode from the input profile).
+
+    By the time export calls this, the buffer is already encoded with the
+    *working-space* OETF (not the input profile's own TRC — decoding it with
+    the wrong one is the double-TRC bug this bypass exists to avoid). The
+    XYZ_to_working @ src_to_XYZ matrix is a linear-light operator, so it must
+    run on decoded values: decode, matrix-multiply, re-encode.
+    """
+    m_total = np.ascontiguousarray((_XYZ_TO_WORKING.astype(np.float64) @ src_to_xyz).astype(np.float32))
+    linear = working_oetf_decode(img)
+    h, w = linear.shape[:2]
+    flat = linear.reshape(-1, 3).astype(np.float32, copy=False)
+    out = _matmul_3x3_kernel(flat, m_total)
+    out = np.clip(out.reshape(h, w, 3), 0.0, 1.0)
+    return working_oetf_encode(out)
+
+
 @parallel_njit(cache=True, fastmath=True)
 def _rgb_to_lab_kernel(px: np.ndarray, m: np.ndarray, white: np.ndarray, eps: float, kappa: float) -> np.ndarray:
     """Row-parallel linear working RGB -> CIELAB (D65) over an (N, 3) pixel list."""

negpy/services/rendering/image_processor.py

@@ -982,6 +982,10 @@ def _encode_export(
 
         is_greyscale = color_space == ColorSpace.GREYSCALE.value
 
+        buffer, bypassed = self._try_matrix_bypass(buffer, icc_input)
+        if bypassed:
+            icc_input = None
+
         if fmt == ExportFormat.TIFF:
             if is_greyscale:
                 img_int = float_to_uint_luma(np.ascontiguousarray(buffer), bit_depth=16)
@@ -1239,6 +1243,36 @@ def _get_target_icc_bytes(self, color_space: str, icc_path: Optional[str]) -> Op
                 return f.read()
         return None
 
+    @staticmethod
+    def _try_matrix_bypass(buffer: np.ndarray, input_icc_path: Optional[str]) -> Tuple[np.ndarray, bool]:
+        """Apply a primaries-only transform if the input ICC is a matrix/TRC profile.
+
+        Returns (transformed_buffer, True) when the bypass fired, so the caller
+        can clear icc_input and let the normal working→output CMS path run.
+        Returns (buffer, False) unchanged for LUT-based profiles.
+        """
+        if not input_icc_path or not os.path.exists(input_icc_path):
+            return buffer, False
+        try:
+            with open(input_icc_path, "rb") as f:
+                icc_data = f.read()
+            from negpy.infrastructure.display.icc_profile import (
+                extract_primaries_matrix,
+                is_matrix_trc_profile,
+            )
+
+            if not is_matrix_trc_profile(icc_data):
+                return buffer, False
+            src_to_xyz = extract_primaries_matrix(icc_data)
+            if src_to_xyz is None:
+                return buffer, False
+            from negpy.kernel.image.logic import apply_primaries_transform
+
+            return apply_primaries_transform(buffer, src_to_xyz), True
+        except Exception as e:
+            logger.warning("Matrix-TRC bypass failed, falling back to full CMS: %s", e)
+            return buffer, False
+
     @staticmethod
     def _has_custom_icc(input_icc_path: Optional[str], output_icc_path: Optional[str]) -> bool:
         """True when an input or output ICC override file is present."""
@@ -1492,6 +1526,11 @@ def soft_proof_preview(
             # littleCMS needs RGB against the RGB working/output profiles.
             if pil_img.mode != "RGB":
                 pil_img = pil_img.convert("RGB")
+            buf_f32 = np.asarray(pil_img, dtype=np.float32) / 255.0
+            buf_f32, bypassed = ImageProcessor._try_matrix_bypass(buf_f32, input_icc_path)
+            if bypassed:
+                pil_img = Image.fromarray(np.clip(buf_f32 * 255.0 + 0.5, 0, 255).astype(np.uint8))
+                input_icc_path = None
             p_src = ImageProcessor._resolve_src_profile(working_color_space, input_icc_path)
             # Custom output profile, or the working space when only an input is set.
             p_dst = ImageProcessor._resolve_dst_profile(working_color_space, output_icc_path)