diff --git a/docs/PIPELINE.md b/docs/PIPELINE.md index 754efaa7..6b1e03cf 100644 --- a/docs/PIPELINE.md +++ b/docs/PIPELINE.md @@ -151,8 +151,9 @@ Here is what happens to your image. We apply these steps in order, passing the b The plane covers the printed frame only, because the enlarger projects the crop: a rebate or scanner surround blurred into the mask prints as a vignette the negative does not have. It is placed back at the crop, edge-replicated outside so the crop tool's full-frame preview has no seam. Hidden on the transparency transfer path, which takes no dodge/burn map. Instruments read the unmasked negative, as they already do under a dodge. * **Output**: converts print density back to **scene-linear** reflectance (transmittance): $$I_{out} = 10^{-D}$$ - * **Paper Black** (`paper_black`, off): off applies black point compensation, the same idea as ICC relative-colorimetric soft-proofing. A reflection print's D-max ($2.3$) floors reflectance at $10^{-2.3} \approx 0.005$, but the adapted eye reads paper black as black, so the display should too; on preserves the paper's lifted D-max instead. With compensation, the default, each channel becomes $I_{out} = (I - t_b) / (1 - t_b)$, clamped at $0$, where $t_b = 10^{-D_b}$ and $D_b$ is the physical $D_{max}$, or $D_{max} + \text{toe}_{ch} \cdot 0.90$ when that layer's toe is negative. The curve reaches $D_{max}$ only asymptotically, so a **negative toe raises the clip point** into the shadows, which is what makes exact $0$ reachable and "negative toe deepens blacks" literal. A lifted toe and per-layer shadow casts survive because the reference is the *physical* $D_{max}$, not $D_{max,eff}$. A negative per-layer toe trim, with compensation on, tints the deepest black. - * **Note**: the pipeline is **scene-linear internally**. The exposure stage emits linear light, and every creative stage (Local Contrast, Lab, Toning, Finish) operates on it. The working-space OETF, the **Adobe RGB (1998) TRC**, a pure $563/256 \approx 2.199$ power with no linear segment, is applied only as the final engine step, so it composes correctly with the Adobe RGB ICC profile at the display and export boundary. The GPU keeps a single encoded perceptual region: exposure → clahe encoded → lab decodes back to linear. + * **Paper Black** (`paper_black`, off): off applies black point compensation, the same idea as ICC relative-colorimetric soft-proofing. A reflection print's D-max ($2.3$) floors reflectance at $10^{-2.3} \approx 0.005$, but the adapted eye reads paper black as black, so the display should too. On preserves the paper's lifted D-max instead. With compensation (the default), each channel becomes $I_{out} = (I - t_b) / (1 - t_b)$, clamped at $0$, where $t_b = 10^{-D_b}$ and $D_b$ is the physical $D_{max}$, or $D_{max} + \text{toe}_{ch} \cdot 0.90$ when that layer's toe is negative. The curve reaches $D_{max}$ only asymptotically, so a **negative toe raises the clip point** into the shadows. That is what makes exact $0$ reachable ("negative toe deepens blacks", literally). A lifted toe and per-layer shadow casts survive because the reference is the *physical* $D_{max}$, not $D_{max,eff}$. A negative per-layer toe trim (with compensation on) tints the deepest black. + * **Note**: The pipeline is **scene-linear internally**. The exposure stage emits linear light and every creative stage (Local Contrast, Retouch, Lab, Toning, Finish) operates on it. The working-space OETF (the **Adobe RGB (1998) TRC**, a pure $563/256 \approx 2.199$ power with no linear segment) is applied only as the final engine step (the output transform), so it composes correctly with the Adobe RGB ICC profile at the display/export boundary. Retouch is a perceptual op, so the CPU brackets that stage through the OETF (encode → heal → decode); the GPU keeps a single encoded perceptual region (exposure → clahe/retouch encoded → lab decodes back to linear). + * **Input ICC overrides primaries only, never the tone curve.** The rendered buffer's TRC is always this working-space OETF — never the selected Input ICC's own declared TRC, which would need the file to actually be encoded that way, and it isn't. For a **matrix/TRC** profile (`infrastructure/display/icc_profile.py`), the boundary transform extracts just its primaries (Bradford-adapted to D65, via its `chad` tag when present) and applies them as a plain matrix; the profile's declared TRC is inert. Concretely, `sRGB-*-g10.icc` (linear TRC) and `sRGB-*-srgbtrc.icc` (sRGB TRC) — identical primaries, different declared TRC — now render **identically** as Input ICC. A **LUT** (A2B0/B2A0) profile instead runs through the full CMS transform, so its authored input curves *are* honoured — that's where a profile like the bundled narrowband `RGBScan.icc` applies its compensation, which is authored against this exact boundary encoding. ### Automatic helpers diff --git a/docs/USER_GUIDE.md b/docs/USER_GUIDE.md index 849c645c..fc728d22 100644 --- a/docs/USER_GUIDE.md +++ b/docs/USER_GUIDE.md @@ -797,7 +797,7 @@ The primary **Export** action. Its chevron menu picks the scope: current frame ( * **Format**: `JPEG`, `TIFF`, `PNG`, `JPEG XL`, or `WebP`, with quality or effort options per format. TIFF is always zlib-compressed. **JPEG XL supports only `sRGB`, `P3 D65`, `Rec 2020` or `Greyscale`** for Color Space: it tags color with compact enumerated values rather than an embedded ICC profile, and NegPy's JXL encoder cannot carry an arbitrary one, so `Adobe RGB`, `ProPhoto RGB` and a custom Output ICC are rejected with an error. Pick a supported space or a different format. * **Color Space**: `Same as Source`, `sRGB`, `Adobe RGB`, `ProPhoto RGB`, `P3 D65`, `Rec 2020`, or `Greyscale` (true B&W output). -* **Input / Output ICC**: soft-proof against, and optionally embed, an ICC profile. Output is the destination profile (default); Input treats the profile as the source, for when a scan's profile is known but untagged. Not available for JPEG XL output; see the Format note above. +* **Input / Output ICC**: soft-proof against, and optionally embed, an ICC profile. Output is the destination profile (default); Input treats the profile as the source, for when a scan's profile is known but untagged. Not available for JPEG XL output; see the Format note above. Input overrides **primaries only** — the tone curve is always the pipeline's own, so a matrix-style profile's declared TRC is ignored (two profiles with identical primaries but different TRCs render identically); a LUT-style profile's own input curves are still honoured. * **Paper Aspect Ratio**: final print ratio, or *Original* (no resize). * **Resolution**: *Original* (full RAW resolution), *Print* (long-edge **Size** in cm plus **DPI**), or *Pixels* (long-edge **px**; the short side follows the paper ratio). * **Destination**: **Filename Pattern** (a Jinja2 template with export settings plus Metadata fields such as roll, camera and film; see [TEMPLATING.md](TEMPLATING.md)), an **Overwrite** toggle, and the output location (subfolder of source, same as source, or an absolute **Export Path** with a browse button). Destination applies to all three output intents: with **Linear** selected, Format, Size and Color hide (a raw dump has no use for them) and Destination stays. diff --git a/negpy/infrastructure/display/icc_profile.py b/negpy/infrastructure/display/icc_profile.py new file mode 100644 index 00000000..9216bfc1 --- /dev/null +++ b/negpy/infrastructure/display/icc_profile.py @@ -0,0 +1,135 @@ +"""Low-level ICC profile parsing for matrix/TRC detection and primaries extraction. + +Reads the tag table and specific tag payloads using pure struct parsing — +no dependency on PIL.ImageCms or lcms2 for the introspection itself. +""" + +import struct +from typing import Optional + +import numpy as np + +# ICC's PCS (profile connection space) is always D50-relative, so every conformant +# matrix/TRC profile's rXYZ/gXYZ/bXYZ colorant tags are D50-adapted regardless of the +# profile's actual native illuminant (D65 for sRGB/Adobe RGB, D60 for ACES, D50 for +# ProPhoto, ...). Recovering D65-referenced primaries — needed to combine with this +# codebase's D65-referenced working-space math — is therefore always a two-step +# chromatic adaptation: PCS(D50) -> native white -> D65. `chad`, when present, gives +# the exact native white (inverting it alone only reaches D65 by coincidence, for a +# D65-native profile); its absence means the native white is unknown and D50 is the +# only assumption available. +_D50_XYZ = np.array([0.9642, 1.0000, 0.8249], dtype=np.float64) +_D65_XYZ = np.array([0.95047, 1.00000, 1.08883], dtype=np.float64) + +# Bradford cone-response matrix (Lindbloom), used to build a chromatic adaptation +# transform between any two reference whites. +_BRADFORD_CONE = np.array( + [ + [0.8951000, 0.2664000, -0.1614000], + [-0.7502000, 1.7135000, 0.0367000], + [0.0389000, -0.0685000, 1.0296000], + ], + dtype=np.float64, +) + + +def _bradford_adaptation(src_white: np.ndarray, dst_white: np.ndarray) -> np.ndarray: + """3x3 Bradford chromatic adaptation matrix mapping XYZ relative to ``src_white`` + to XYZ relative to ``dst_white``.""" + lms_src = _BRADFORD_CONE @ src_white + lms_dst = _BRADFORD_CONE @ dst_white + scale = np.diag(lms_dst / lms_src) + return np.linalg.inv(_BRADFORD_CONE) @ scale @ _BRADFORD_CONE + + +def _read_tag_table(data: bytes) -> dict[bytes, tuple[int, int]]: + """Parse the ICC tag table into {signature: (offset, size)}.""" + if len(data) < 132: + return {} + tag_count = struct.unpack_from(">I", data, 128)[0] + tags: dict[bytes, tuple[int, int]] = {} + for i in range(tag_count): + base = 132 + i * 12 + if base + 12 > len(data): + break + sig = data[base : base + 4] + offset, size = struct.unpack_from(">II", data, base + 4) + tags[sig] = (offset, size) + return tags + + +def _read_xyz_tag(data: bytes, offset: int, size: int) -> Optional[np.ndarray]: + """Read an XYZType tag (ICC spec §10.31) → (3,) float64 array.""" + if size < 20: + return None + x = struct.unpack_from(">i", data, offset + 8)[0] / 65536.0 + y = struct.unpack_from(">i", data, offset + 12)[0] / 65536.0 + z = struct.unpack_from(">i", data, offset + 16)[0] / 65536.0 + return np.array([x, y, z], dtype=np.float64) + + +def _read_chad_tag(data: bytes, offset: int, size: int) -> Optional[np.ndarray]: + """Read a chromaticAdaptationTag (s15Fixed16ArrayType, ICC spec §10.8) → (3, 3) float64.""" + if size < 8 + 9 * 4: + return None + vals = struct.unpack_from(">9i", data, offset + 8) + return np.array(vals, dtype=np.float64).reshape(3, 3) / 65536.0 + + +def is_matrix_trc_profile(data: bytes) -> bool: + """True when the profile is a matrix/TRC (shaper-matrix) type. + + Requires rXYZ/gXYZ/bXYZ colorant tags and at least one TRC tag, + and must NOT have A2B0/B2A0 LUT tags. + """ + tags = _read_tag_table(data) + has_colorants = all(sig in tags for sig in (b"rXYZ", b"gXYZ", b"bXYZ")) + has_trc = any(sig in tags for sig in (b"rTRC", b"gTRC", b"bTRC")) + has_lut = any(sig in tags for sig in (b"A2B0", b"B2A0")) + return has_colorants and has_trc and not has_lut + + +def extract_primaries_matrix(data: bytes) -> Optional[np.ndarray]: + """Extract the 3x3 D65-referenced RGB→XYZ matrix from rXYZ/gXYZ/bXYZ colorant tags. + + The raw tag values are PCS-relative (D50-adapted, per the ICC spec). `chad` + records the profile's native-white -> PCS(D50) adaptation actually used, so + `inv(chad)` recovers the *native* reference — D65 only by coincidence, for a + D65-native profile (sRGB, Adobe RGB, ...). For anything else (ACES/ACEScg: D60, + ProPhoto: D50, ...) that native reference then needs its own adaptation to D65. + Without `chad` (typical of v2 profiles) the native white is unknown, so D50 is + assumed. Either way the result always lands on D65 — see + `test_extracted_primaries_are_d65_referenced` for the profiles this matters for. + + Returns a (3, 3) float64 array where each column is one primary's XYZ + tristimulus, or **None if the colorant tags are missing** (malformed size, or + absent entirely — e.g. a GRAY-space profile, which has no rXYZ/gXYZ/bXYZ to + extract). This is a load-bearing part of the contract, not incidental: callers + (`ImageProcessor._try_matrix_bypass`) rely on `None` here to fall through to the + full-CMS path rather than crash or fabricate a matrix — `is_matrix_trc_profile` + is expected to have already filtered out non-RGB profiles first, but this + function must stay safe to call regardless. + """ + tags = _read_tag_table(data) + cols = [] + for sig in (b"rXYZ", b"gXYZ", b"bXYZ"): + entry = tags.get(sig) + if entry is None: + return None + xyz = _read_xyz_tag(data, *entry) + if xyz is None: + return None + cols.append(xyz) + m_pcs = np.column_stack(cols) + + m_native, native_white = m_pcs, _D50_XYZ + chad_entry = tags.get(b"chad") + if chad_entry is not None: + chad = _read_chad_tag(data, *chad_entry) + if chad is not None: + try: + chad_inv = np.linalg.inv(chad) + m_native, native_white = chad_inv @ m_pcs, chad_inv @ _D50_XYZ + except np.linalg.LinAlgError: + pass + return _bradford_adaptation(native_white, _D65_XYZ) @ m_native diff --git a/negpy/kernel/image/logic.py b/negpy/kernel/image/logic.py index b2853eb6..5fdb023e 100644 --- a/negpy/kernel/image/logic.py +++ b/negpy/kernel/image/logic.py @@ -182,6 +182,37 @@ def working_oetf_decode(img: np.ndarray) -> np.ndarray: _LAB_KAPPA = 7.787 +@parallel_njit(cache=True, fastmath=True) +def _matmul_3x3_kernel(px: np.ndarray, m: np.ndarray) -> np.ndarray: + """Row-parallel 3x3 matrix multiply over an (N, 3) pixel array.""" + n = px.shape[0] + out = np.empty((n, 3), dtype=np.float32) + for i in prange(n): + r, g, b = px[i, 0], px[i, 1], px[i, 2] + out[i, 0] = m[0, 0] * r + m[0, 1] * g + m[0, 2] * b + out[i, 1] = m[1, 0] * r + m[1, 1] * g + m[1, 2] * b + out[i, 2] = m[2, 0] * r + m[2, 1] * g + m[2, 2] * b + return out + + +def apply_primaries_transform(img: np.ndarray, src_to_xyz: np.ndarray) -> np.ndarray: + """Apply a primaries-only colour transform (no TRC re-decode from the input profile). + + By the time export calls this, the buffer is already encoded with the + *working-space* OETF (not the input profile's own TRC — decoding it with + the wrong one is the double-TRC bug this bypass exists to avoid). The + XYZ_to_working @ src_to_XYZ matrix is a linear-light operator, so it must + run on decoded values: decode, matrix-multiply, re-encode. + """ + m_total = np.ascontiguousarray((_XYZ_TO_WORKING.astype(np.float64) @ src_to_xyz).astype(np.float32)) + linear = working_oetf_decode(img) + h, w = linear.shape[:2] + flat = linear.reshape(-1, 3).astype(np.float32, copy=False) + out = _matmul_3x3_kernel(flat, m_total) + out = np.clip(out.reshape(h, w, 3), 0.0, 1.0) + return working_oetf_encode(out) + + @parallel_njit(cache=True, fastmath=True) def _rgb_to_lab_kernel(px: np.ndarray, m: np.ndarray, white: np.ndarray, eps: float, kappa: float) -> np.ndarray: """Row-parallel linear working RGB -> CIELAB (D65) over an (N, 3) pixel list.""" diff --git a/negpy/services/rendering/image_processor.py b/negpy/services/rendering/image_processor.py index 607a4deb..719682af 100644 --- a/negpy/services/rendering/image_processor.py +++ b/negpy/services/rendering/image_processor.py @@ -982,6 +982,10 @@ def _encode_export( is_greyscale = color_space == ColorSpace.GREYSCALE.value + buffer, bypassed = self._try_matrix_bypass(buffer, icc_input) + if bypassed: + icc_input = None + if fmt == ExportFormat.TIFF: if is_greyscale: img_int = float_to_uint_luma(np.ascontiguousarray(buffer), bit_depth=16) @@ -1239,6 +1243,36 @@ def _get_target_icc_bytes(self, color_space: str, icc_path: Optional[str]) -> Op return f.read() return None + @staticmethod + def _try_matrix_bypass(buffer: np.ndarray, input_icc_path: Optional[str]) -> Tuple[np.ndarray, bool]: + """Apply a primaries-only transform if the input ICC is a matrix/TRC profile. + + Returns (transformed_buffer, True) when the bypass fired, so the caller + can clear icc_input and let the normal working→output CMS path run. + Returns (buffer, False) unchanged for LUT-based profiles. + """ + if not input_icc_path or not os.path.exists(input_icc_path): + return buffer, False + try: + with open(input_icc_path, "rb") as f: + icc_data = f.read() + from negpy.infrastructure.display.icc_profile import ( + extract_primaries_matrix, + is_matrix_trc_profile, + ) + + if not is_matrix_trc_profile(icc_data): + return buffer, False + src_to_xyz = extract_primaries_matrix(icc_data) + if src_to_xyz is None: + return buffer, False + from negpy.kernel.image.logic import apply_primaries_transform + + return apply_primaries_transform(buffer, src_to_xyz), True + except Exception as e: + logger.warning("Matrix-TRC bypass failed, falling back to full CMS: %s", e) + return buffer, False + @staticmethod def _has_custom_icc(input_icc_path: Optional[str], output_icc_path: Optional[str]) -> bool: """True when an input or output ICC override file is present.""" @@ -1492,6 +1526,11 @@ def soft_proof_preview( # littleCMS needs RGB against the RGB working/output profiles. if pil_img.mode != "RGB": pil_img = pil_img.convert("RGB") + buf_f32 = np.asarray(pil_img, dtype=np.float32) / 255.0 + buf_f32, bypassed = ImageProcessor._try_matrix_bypass(buf_f32, input_icc_path) + if bypassed: + pil_img = Image.fromarray(np.clip(buf_f32 * 255.0 + 0.5, 0, 255).astype(np.uint8)) + input_icc_path = None p_src = ImageProcessor._resolve_src_profile(working_color_space, input_icc_path) # Custom output profile, or the working space when only an input is set. p_dst = ImageProcessor._resolve_dst_profile(working_color_space, output_icc_path) diff --git a/tests/test_icc_trc_bypass.py b/tests/test_icc_trc_bypass.py new file mode 100644 index 00000000..725293ef --- /dev/null +++ b/tests/test_icc_trc_bypass.py @@ -0,0 +1,471 @@ +"""Tests for the matrix/TRC ICC profile bypass. + +Validates: +- Profile-type detection (matrix/TRC vs LUT-based) +- Primaries matrix extraction (PCS-D50 tag values -> D65-referenced result) +- TRC independence (the actual bug being fixed) +""" + +import struct +from typing import Optional + +import numpy as np +import pytest + +from negpy.infrastructure.display.icc_profile import ( + extract_primaries_matrix, + is_matrix_trc_profile, +) +from negpy.kernel.image.logic import _WORKING_TO_XYZ, apply_primaries_transform + + +def _s15fixed16(val: float) -> bytes: + return struct.pack(">i", int(round(val * 65536.0))) + + +def _xyz_tag(x: float, y: float, z: float) -> bytes: + return b"XYZ " + b"\x00" * 4 + _s15fixed16(x) + _s15fixed16(y) + _s15fixed16(z) + + +def _trc_tag_gamma(gamma: float) -> bytes: + """A curveType TRC with a single gamma value (count=1).""" + return b"curv" + b"\x00" * 4 + struct.pack(">I", 1) + struct.pack(">H", int(round(gamma * 256.0))) + b"\x00\x00" + + +def _chad_tag(m: np.ndarray) -> bytes: + """An s15Fixed16ArrayType chromaticAdaptationTag (ICC spec §10.8).""" + return b"sf32" + b"\x00" * 4 + b"".join(_s15fixed16(v) for v in m.flatten()) + + +def _build_matrix_trc_icc( + r_xyz: tuple[float, float, float], + g_xyz: tuple[float, float, float], + b_xyz: tuple[float, float, float], + wtpt: tuple[float, float, float] = (0.9642, 1.0000, 0.8249), + gamma: float = 2.2, + add_a2b0: bool = False, + chad: Optional[np.ndarray] = None, +) -> bytes: + """Build a minimal ICC profile with matrix/TRC structure. ``wtpt`` defaults to the + D50 PCS value real profiles carry; pass ``chad`` (a v4 profile always has one) to + exercise the inv(chad) adaptation path instead of the generic Bradford fallback.""" + tag_data: list[tuple[bytes, bytes]] = [] + tag_data.append((b"rXYZ", _xyz_tag(*r_xyz))) + tag_data.append((b"gXYZ", _xyz_tag(*g_xyz))) + tag_data.append((b"bXYZ", _xyz_tag(*b_xyz))) + tag_data.append((b"wtpt", _xyz_tag(*wtpt))) + trc = _trc_tag_gamma(gamma) + tag_data.append((b"rTRC", trc)) + tag_data.append((b"gTRC", trc)) + tag_data.append((b"bTRC", trc)) + if chad is not None: + tag_data.append((b"chad", _chad_tag(chad))) + if add_a2b0: + tag_data.append((b"A2B0", b"mft2" + b"\x00" * 40)) + + tag_count = len(tag_data) + tag_table_size = tag_count * 12 + header_size = 128 + 4 + tag_table_size + offset = header_size + offsets: list[tuple[int, int]] = [] + for _, payload in tag_data: + padded = len(payload) + if padded % 4: + padded += 4 - padded % 4 + offsets.append((offset, len(payload))) + offset += padded + total_size = offset + + header = bytearray(128) + struct.pack_into(">I", header, 0, total_size) + header[36:40] = b"acsp" + header[12:16] = b"mntr" + header[16:20] = b"RGB " + header[40:44] = b"APPL" + + tag_table = struct.pack(">I", tag_count) + for i, (sig, _) in enumerate(tag_data): + tag_table += sig + struct.pack(">II", offsets[i][0], offsets[i][1]) + + body = b"" + for _, payload in tag_data: + padded = len(payload) + if padded % 4: + payload += b"\x00" * (4 - padded % 4) + body += payload + + return bytes(header) + tag_table + body + + +def _build_gray_icc() -> bytes: + """Build a minimal GRAY-space ICC profile: desc/cprt/wtpt/kTRC/bkpt, no colorant tags + at all — matches the real structure of a bundled/downloadable grayscale profile.""" + tag_data: list[tuple[bytes, bytes]] = [ + (b"wtpt", _xyz_tag(0.9642, 1.0, 0.8249)), + (b"bkpt", _xyz_tag(0.0, 0.0, 0.0)), + (b"kTRC", _trc_tag_gamma(2.2)), + ] + + tag_count = len(tag_data) + tag_table_size = tag_count * 12 + header_size = 128 + 4 + tag_table_size + offset = header_size + offsets: list[tuple[int, int]] = [] + for _, payload in tag_data: + padded = len(payload) + if padded % 4: + padded += 4 - padded % 4 + offsets.append((offset, len(payload))) + offset += padded + total_size = offset + + header = bytearray(128) + struct.pack_into(">I", header, 0, total_size) + header[36:40] = b"acsp" + header[12:16] = b"mntr" + header[16:20] = b"GRAY" + header[40:44] = b"APPL" + + tag_table = struct.pack(">I", tag_count) + for i, (sig, _) in enumerate(tag_data): + tag_table += sig + struct.pack(">II", offsets[i][0], offsets[i][1]) + + body = b"" + for _, payload in tag_data: + padded = len(payload) + if padded % 4: + payload += b"\x00" * (4 - padded % 4) + body += payload + + return bytes(header) + tag_table + body + + +# PCS-D50-relative sRGB primaries, as an ICC v4 file actually stores them +# (extracted from a real sRGB-primaries profile) — real profiles never store +# native-D65 values directly, since the PCS is always D50-relative. +_SRGB_R = (0.43603516, 0.22248840, 0.01391602) +_SRGB_G = (0.38511658, 0.71690369, 0.09706116) +_SRGB_B = (0.14305115, 0.06060791, 0.71392822) +_D50_WP = (0.9642, 1.0000, 0.8249) +_D65_WP = (0.9505, 1.0000, 1.0889) + +# The chad tag from the same real profile that _SRGB_R/G/B were extracted from — the +# exact D65->D50 adaptation lcms2 applied when writing the file, as opposed to a +# generic Bradford assumption. +_SRGB_CHAD = np.array( + [ + [1.04788208, 0.0229187, -0.05021667], + [0.02958679, 0.99047852, -0.01707458], + [-0.00924683, 0.01507568, 0.75167847], + ] +) + +# Known native-D65-referenced sRGB primaries (Lindbloom), what extract_primaries_matrix +# should recover from the PCS-D50 fixture above. +_SRGB_R_D65 = (0.4124564, 0.2126729, 0.0193339) +_SRGB_G_D65 = (0.3575761, 0.7151522, 0.1191920) +_SRGB_B_D65 = (0.1804375, 0.0721750, 0.9503041) + +# PCS-D50 colorants + chad extracted from a real ACES (AP0) profile, D60-native — inverting +# chad alone lands on D60, not D65, unless the native white is itself re-adapted to D65. +_ACES_R = (0.9908905, 0.3618927, -0.00271606) +_ACES_G = (0.01223755, 0.72251892, 0.008255) +_ACES_B = (-0.03892517, -0.08441162, 0.81936646) +_ACES_CHAD = np.array( + [ + [1.03416443, 0.01681519, -0.03747559], + [0.0216217, 0.99223328, -0.01272583], + [-0.00694275, 0.01132202, 0.8130188], + ] +) + +# PCS-D50 colorants + chad extracted from a real ProPhoto-primaries profile, D50-native — +# chad is near-identity here, so inv(chad) alone is nearly a no-op and stays at D50. +_PROPHOTO_R = (0.79771423, 0.28805542, 0.0) +_PROPHOTO_G = (0.13516235, 0.71186829, 1.526e-05) +_PROPHOTO_B = (0.03132629, 7.629e-05, 0.82489014) +_PROPHOTO_CHAD = np.array( + [ + [9.99954220e-01, -3.05200000e-05, -3.05200000e-05], + [-4.57800000e-05, 1.00004578e00, -1.52600000e-05], + [-1.52600000e-05, 1.52600000e-05, 9.99740600e-01], + ] +) + +_D65_XYZ = np.array([0.95047, 1.0, 1.08883]) + + +class TestProfileDetection: + def test_matrix_trc_profile_detected(self): + icc = _build_matrix_trc_icc(_SRGB_R, _SRGB_G, _SRGB_B) + assert is_matrix_trc_profile(icc) + + def test_lut_profile_not_detected_as_matrix(self): + icc = _build_matrix_trc_icc(_SRGB_R, _SRGB_G, _SRGB_B, add_a2b0=True) + assert not is_matrix_trc_profile(icc) + + def test_real_bundled_profiles(self): + import os + + icc_dir = os.path.join(os.path.dirname(os.path.dirname(__file__)), "icc") + rgbscan = os.path.join(icc_dir, "RGBScan.icc") + if os.path.exists(rgbscan): + with open(rgbscan, "rb") as f: + data = f.read() + assert not is_matrix_trc_profile(data), "RGBScan.icc is LUT-based, must not be detected as matrix/TRC" + + def test_gray_profile_not_detected_as_matrix(self): + """A GRAY-space profile (class=mntr, space=GRAY: desc/cprt/wtpt/kTRC/bkpt, no + rXYZ/gXYZ/bXYZ) has no colorant tags at all — must not be mistaken for matrix/TRC, + and extraction must return None rather than raise, so the bypass safely no-ops and + the profile falls through to full CMS.""" + icc = _build_gray_icc() + assert not is_matrix_trc_profile(icc) + assert extract_primaries_matrix(icc) is None + + from negpy.services.rendering.image_processor import ImageProcessor + + import tempfile + import os as _os + + with tempfile.NamedTemporaryFile(suffix=".icc", delete=False) as f: + f.write(icc) + path = f.name + try: + img = np.random.RandomState(2).rand(8, 8, 3).astype(np.float32) + out, bypassed = ImageProcessor._try_matrix_bypass(img, path) + assert not bypassed + np.testing.assert_array_equal(out, img) + finally: + _os.unlink(path) + + +class TestPrimariesExtraction: + def test_extract_srgb_primaries_adapts_via_chad(self): + """With a chad tag present (the v4 case), extraction inverts it exactly rather + than assuming generic Bradford — recovers the D65-native sRGB matrix tightly.""" + icc = _build_matrix_trc_icc(_SRGB_R, _SRGB_G, _SRGB_B, chad=_SRGB_CHAD) + m = extract_primaries_matrix(icc) + assert m is not None + expected = np.array([_SRGB_R_D65, _SRGB_G_D65, _SRGB_B_D65], dtype=np.float64).T + np.testing.assert_allclose(m, expected, atol=5e-4) + + def test_extract_srgb_primaries_falls_back_to_bradford_without_chad(self): + """Without a chad tag (typical of v2 profiles), extraction still recovers + D65-native primaries via generic Bradford D50->D65, just less precisely.""" + icc = _build_matrix_trc_icc(_SRGB_R, _SRGB_G, _SRGB_B) + m = extract_primaries_matrix(icc) + assert m is not None + expected = np.array([_SRGB_R_D65, _SRGB_G_D65, _SRGB_B_D65], dtype=np.float64).T + np.testing.assert_allclose(m, expected, atol=2e-3) + + def test_returns_none_without_tags(self): + header = bytearray(128) + struct.pack_into(">I", header, 0, 132) + header[36:40] = b"acsp" + data = bytes(header) + struct.pack(">I", 0) + assert extract_primaries_matrix(data) is None + + @pytest.mark.parametrize( + "r,g,b,chad,label", + [ + (_SRGB_R, _SRGB_G, _SRGB_B, _SRGB_CHAD, "sRGB (D65-native)"), + (_ACES_R, _ACES_G, _ACES_B, _ACES_CHAD, "ACES AP0 (D60-native)"), + (_PROPHOTO_R, _PROPHOTO_G, _PROPHOTO_B, _PROPHOTO_CHAD, "ProPhoto (D50-native)"), + ], + ) + def test_extracted_primaries_are_d65_referenced(self, r, g, b, chad, label): + """The whole point of extract_primaries_matrix is to hand back primaries expressed + against D65 (to combine with this codebase's D65-native _XYZ_TO_WORKING). That must + hold regardless of the profile's own native white point: R+G+B XYZ must sum to D65. + inv(chad) alone only satisfies this for D65-native profiles (sRGB here) — for a + D60-native (ACES) or D50-native (ProPhoto) profile it lands on the profile's own + native white instead, which is the bug this test exists to catch.""" + icc = _build_matrix_trc_icc(r, g, b, chad=chad) + m = extract_primaries_matrix(icc) + assert m is not None + white = m.sum(axis=1) + np.testing.assert_allclose(white, _D65_XYZ, atol=1e-3, err_msg=f"{label}: white did not land on D65") + + def test_extracted_primaries_are_d65_referenced_without_chad(self): + """Bradford-fallback path (no chad tag) must also land on D65 regardless of the + profile's actual native white — it has no way to know it, so it always assumes D50.""" + icc = _build_matrix_trc_icc(_SRGB_R, _SRGB_G, _SRGB_B) + m = extract_primaries_matrix(icc) + assert m is not None + np.testing.assert_allclose(m.sum(axis=1), _D65_XYZ, atol=1e-3) + + +class TestTrcIndependence: + """The core test: identical primaries with different TRCs must produce + identical output through the matrix-only path.""" + + def test_different_trcs_same_result(self): + icc_g18 = _build_matrix_trc_icc(_SRGB_R, _SRGB_G, _SRGB_B, gamma=1.8) + icc_g24 = _build_matrix_trc_icc(_SRGB_R, _SRGB_G, _SRGB_B, gamma=2.4) + assert is_matrix_trc_profile(icc_g18) + assert is_matrix_trc_profile(icc_g24) + + m1 = extract_primaries_matrix(icc_g18) + m2 = extract_primaries_matrix(icc_g24) + assert m1 is not None and m2 is not None + np.testing.assert_array_equal(m1, m2) + + img = np.random.RandomState(42).rand(64, 64, 3).astype(np.float32) + out1 = apply_primaries_transform(img, m1) + out2 = apply_primaries_transform(img, m2) + np.testing.assert_array_equal(out1, out2) + + +class TestApplyPrimariesTransform: + def test_identity_for_working_space(self): + """When the input primaries match the working space, the transform is identity.""" + img = np.random.RandomState(7).rand(32, 32, 3).astype(np.float32) * 0.8 + 0.1 + out = apply_primaries_transform(img, _WORKING_TO_XYZ.astype(np.float64)) + np.testing.assert_allclose(out, img, atol=1e-4) + + def test_output_clipped_to_01(self): + img = np.ones((2, 2, 3), dtype=np.float32) * 2.0 + out = apply_primaries_transform(img, _WORKING_TO_XYZ.astype(np.float64)) + assert out.max() <= 1.0 + assert out.min() >= 0.0 + + def test_shape_preserved(self): + img = np.random.RandomState(0).rand(100, 50, 3).astype(np.float32) + out = apply_primaries_transform(img, _WORKING_TO_XYZ.astype(np.float64)) + assert out.shape == img.shape + assert out.dtype == np.float32 + + +class TestImageProcessorBypass: + def test_matrix_profile_bypasses(self): + from negpy.services.rendering.image_processor import ImageProcessor + + icc_data = _build_matrix_trc_icc(_SRGB_R, _SRGB_G, _SRGB_B) + import tempfile + import os + + with tempfile.NamedTemporaryFile(suffix=".icc", delete=False) as f: + f.write(icc_data) + path = f.name + try: + img = np.random.RandomState(1).rand(16, 16, 3).astype(np.float32) + out, bypassed = ImageProcessor._try_matrix_bypass(img, path) + assert bypassed + assert out.shape == img.shape + finally: + os.unlink(path) + + def test_lut_profile_does_not_bypass(self): + from negpy.services.rendering.image_processor import ImageProcessor + + icc_data = _build_matrix_trc_icc(_SRGB_R, _SRGB_G, _SRGB_B, add_a2b0=True) + import tempfile + import os + + with tempfile.NamedTemporaryFile(suffix=".icc", delete=False) as f: + f.write(icc_data) + path = f.name + try: + img = np.random.RandomState(1).rand(16, 16, 3).astype(np.float32) + out, bypassed = ImageProcessor._try_matrix_bypass(img, path) + assert not bypassed + np.testing.assert_array_equal(out, img) + finally: + os.unlink(path) + + def test_bypasses_regardless_of_whitepoint_tag(self): + """Real profiles carry a PCS-D50 wtpt; a D65-literal wtpt is only ever seen in + hand-built fixtures. Either way, whitepoint is not part of the bypass gate.""" + from negpy.services.rendering.image_processor import ImageProcessor + + import tempfile + import os + + for wtpt in (_D50_WP, _D65_WP): + icc_data = _build_matrix_trc_icc(_SRGB_R, _SRGB_G, _SRGB_B, wtpt=wtpt) + with tempfile.NamedTemporaryFile(suffix=".icc", delete=False) as f: + f.write(icc_data) + path = f.name + try: + img = np.random.RandomState(1).rand(16, 16, 3).astype(np.float32) + out, bypassed = ImageProcessor._try_matrix_bypass(img, path) + assert bypassed + finally: + os.unlink(path) + + def test_no_path_does_not_bypass(self): + from negpy.services.rendering.image_processor import ImageProcessor + + img = np.random.RandomState(1).rand(16, 16, 3).astype(np.float32) + out, bypassed = ImageProcessor._try_matrix_bypass(img, None) + assert not bypassed + + +class TestRealBundledProfiles: + """Regression tests against the app's own shipped profiles, not synthetic fixtures.""" + + @staticmethod + def _icc_path(name: str) -> str: + import os + + return os.path.join(os.path.dirname(os.path.dirname(__file__)), "icc", name) + + def test_working_profile_is_identity(self): + """Selecting the app's own working-space profile as Input ICC must be a near-no-op: + its primaries are the working space's primaries, so the bypass matrix is I + eps + (eps from chad-inversion + s15Fixed16 quantization, not exactly I). Tolerance is a + few 16-bit LSB — measured max diff across seeds is ~2e-4 (~14 LSB).""" + from negpy.services.rendering.image_processor import ImageProcessor + + path = self._icc_path("AdobeCompat-v4.icc") + img = np.random.RandomState(3).rand(32, 32, 3).astype(np.float32) * 0.8 + 0.1 + out, bypassed = ImageProcessor._try_matrix_bypass(img, path) + assert bypassed + np.testing.assert_allclose(out, img, atol=5e-4) + + def test_rgbscan_not_bypassable(self): + """RGBScan.icc (spac/Lab PCS, A2B0-only) is the shipped Narrowband Scan default. + It must never take the matrix-only shortcut — its A2B0 curves are authored + against the full-CMS decode and must keep running through it unchanged.""" + from negpy.services.rendering.image_processor import ImageProcessor + + path = self._icc_path("RGBScan.icc") + assert not is_matrix_trc_profile(open(path, "rb").read()) + img = np.random.RandomState(4).rand(16, 16, 3).astype(np.float32) + out, bypassed = ImageProcessor._try_matrix_bypass(img, path) + assert not bypassed + np.testing.assert_array_equal(out, img) + + def test_boundary_encoding_contract(self): + """The buffer arriving at the boundary is pure-power ~2.2 encoded (working_oetf's + 563/256, not exactly 2.2, but within a fraction of an LSB). RGBScan.icc's CLUT + input domain is sRGB piecewise. The A2B0 input curve is the transcode between the + two — decode pure-power, re-encode sRGB — which is what this pins, so a change to + the finish-step OETF fails this loudly instead of silently breaking the shipped + narrowband-scan default.""" + import struct + + from negpy.kernel.image.logic import working_oetf_decode + + path = self._icc_path("RGBScan.icc") + data = open(path, "rb").read() + tag_count = struct.unpack_from(">I", data, 128)[0] + tags = {} + for i in range(tag_count): + base = 132 + i * 12 + sig = data[base : base + 4] + off, sz = struct.unpack_from(">II", data, base + 4) + tags[sig] = (off, sz) + off, _ = tags[b"A2B0"] + assert data[off : off + 4] == b"mft2" + n_in_entries = struct.unpack_from(">H", data, off + 48)[0] + r_curve = np.array(struct.unpack_from(f">{n_in_entries}H", data, off + 52), dtype=np.float64) / 65535.0 + + x = np.linspace(0.0, 1.0, n_in_entries) + linear = working_oetf_decode(x.astype(np.float32)).astype(np.float64) + srgb_encoded = np.where(linear <= 0.0031308, linear * 12.92, 1.055 * linear ** (1 / 2.4) - 0.055) + + # The profile's curve was built against a plain gamma=2.2 assumption, not the + # pipeline's exact 563/256 (2.19921875) — a known, tiny, sub-8-bit-LSB gap. + assert np.max(np.abs(r_curve - srgb_encoded)) < 2e-4