diff --git a/CLAUDE.md b/CLAUDE.md index cb30c864..0dd46324 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -214,6 +214,7 @@ locally and fail in CI. | [dol_issue16_design.md](misc/docs/dol_issue16_design.md) | Issue #16 design: optional key-path write-through / autovivification — opt-in `create_missing`, contextual per-level factory, the `path_set_writeback` boundary engine + persistent-store write-back protocol, scoped plan. Design-only (no code yet). | | [dol_issue18_design.md](misc/docs/dol_issue18_design.md) | Issue #18 design: `self`-not-wrapped delegation trap — `wrapped_self` (shipped) now, is-a wrapping (deferred, major) later. | | [dol_issue83_design.md](misc/docs/dol_issue83_design.md) | Issue #83 design study: the **inverse** of #18 — a delegated method *receives* the unmapped key. Two delegation routes (a fix for one is a no-op on the other), a 13-package census (mostly **latent**; 12 claims refuted), and options A–F with verified costs: `wrapped_self` has its own silent hole (degrades with no live strong reference), chain-walking free functions break on non-`Store` layers, and the only form correct *by construction* is routing the capability through `__getitem__` as a sibling store. **§5 is the carry-forward list for a future redesign.** | +| [dol_issue86_design.md](misc/docs/dol_issue86_design.md) | Discussion #86 design study: **Option G** — spec-carried boundary codecs on a flat proxy (wrapt lessons, KT/VT-annotated interface specs, flatten-and-compile codec stacks). Prototype in `dol/_interface_wrap.py` (private, experimental). Headline: is-a does **not** fix #83 for backend-direct method bodies — F and G serve disjoint populations; codec laws (§2.5) are the boundary invariant's fine print; flat-model guarantees are scoped to pure-codec stacks (filters/caches still nest). Open question 0: who owns the wrap_kvs endgame. | | [dol_issue10_design.md](misc/docs/dol_issue10_design.md) | Issues #10 + #2 (paired) design: recursive wrapping of nested stores (`recursive_wrap`) + a flat `KvReader`/`KvPersister` view (`flat_store`), sharing one `(path,key,value)` descent frontier and reusing the #16 `path_set_writeback` engine. Load-bearing fix: the recursion read-surface and the write-back boundary must be **different** objects (naive `boundary=self` infinite-loops). Model-2 read + write-into-existing in scope; persistent creation deferred to P3. Design-only (no code yet). | | [frontend_dol_ideas.md](misc/docs/frontend_dol_ideas.md) | `zoddal` design: TypeScript KV interface, adapters, Zod bridge, zod-collection-ui integration | diff --git a/dol/_interface_wrap.py b/dol/_interface_wrap.py new file mode 100644 index 00000000..b812bdf5 --- /dev/null +++ b/dol/_interface_wrap.py @@ -0,0 +1,880 @@ +"""Spec-carried boundary codecs on a flat proxy (Option G prototype). + +EXPERIMENTAL — private module, not exported from ``dol``. Design study: +``misc/docs/dol_issue86_design.md`` (companion to discussions #86 and the +#83/#18 design docs). + +The idea, in one paragraph: a wrap is ``(leaf, spec, stack)`` where *spec* +declares, per method, at which argument/return paths the "types of interest" +(KT, VT, ... — any TypeVar "role") occur, and *stack* is a flat sequence of +codec layers, each mapping ``role -> Codec(encoder, decoder)``. Wrapping an +already-wrapped object **extends the stack over the same leaf** — there is +never a wrapper-of-wrapper. At wrap time the stack is compiled: per role, the +encoder pipeline (outer→inner) and decoder pipeline (inner→outer) are fused +into single callables; per method, a boundary plan binds parameters and return +paths to them. Method bodies always run against the **leaf's own public +interface** (``self`` is the leaf), so a method that is correct on the bare +leaf stays correct under any codec stack, and internal ``self.x()`` calls stay +below the boundary — transforms are applied exactly once, at the boundary. + +What this deliberately does NOT do: it does not change what ``self`` is inside +leaf methods (Issue #18's outer-domain methods are the other half of the +problem, served by ``wrapped_self`` today), it does not rebind methods (the +rejected option D), and it is not (yet) ``wrap_kvs``. + +>>> from typing import Protocol, TypeVar, Iterator, Iterable +>>> KT, VT = TypeVar('KT'), TypeVar('VT') +>>> class KvInterface(Protocol[KT, VT]): +... def __getitem__(self, k: KT) -> VT: ... +... def __setitem__(self, k: KT, v: VT) -> None: ... +... def __iter__(self) -> Iterator[KT]: ... +... def __len__(self) -> int: ... +... def __contains__(self, k: KT) -> bool: ... +>>> d = {'a.json': '1'} +>>> codecs = dict( +... KT=Codec(encoder=lambda k: k + '.json', decoder=lambda k: k[:-5]), +... VT=Codec(encoder=str, decoder=int), +... ) + +Loud by default: a public leaf attribute the spec doesn't cover refuses at +wrap time (it would be served with unmapped keys/values — the #83 bug class): + +>>> interface_wrap(d, spec=KvInterface, codecs=codecs) # doctest: +ELLIPSIS +Traceback (most recent call last): +... +dol._interface_wrap.UndeclaredAttributeError: The leaf exposes public... + +Every escape is explicit — extend the spec, forward verbatim, or hide: + +>>> s = interface_wrap(d, spec=KvInterface, codecs=codecs, undeclared='exclude') +>>> s['a'] +1 +>>> s['b'] = 2 +>>> d +{'a.json': '1', 'b.json': '2'} +>>> list(s) +['a', 'b'] +>>> 'a' in s +True +""" + +from collections.abc import Iterable as _IterableABC, Iterator as _IteratorABC +from dataclasses import dataclass, field +from functools import cached_property +import inspect +import typing +from typing import ( + Any, + Callable, + Mapping, + NamedTuple, + Optional, + TypeVar, + Union, + get_args, + get_origin, + get_type_hints, +) + +__all__ = [ + 'Codec', + 'InterfaceSpec', + 'InterfaceProxy', + 'interface_wrap', + 'InterfaceWrapError', + 'UnsupportedSpecShape', + 'UnderAnnotatedSpecError', + 'UndeclaredAttributeError', +] + + +# --------------------------------------------------------------------------- +# Errors — loud by default (No Silent Failures) + + +class InterfaceWrapError(Exception): + """Base for all errors raised by this module.""" + + +class UnsupportedSpecShape(InterfaceWrapError, TypeError): + """An annotation contains a role TypeVar at a path we cannot map. + + Raised at wrap (compile) time, never at call time: refusing early beats + guessing (dol_issue83_design.md §5.7). + """ + + +class UnderAnnotatedSpecError(InterfaceWrapError, TypeError): + """A spec'd method has a parameter with no annotation at all. + + An unannotated parameter is indistinguishable from a deliberately + non-role parameter, which is exactly the silence-by-omission failure mode + this mechanism exists to kill — one level down (a key parameter the spec + author forgot to annotate would silently receive OUTER keys). Annotate + every named parameter of a spec method: with a role TypeVar if it carries + a role, with a concrete type (or ``Any``) to state it does not. + """ + + +class UndeclaredAttributeError(InterfaceWrapError, AttributeError): + """A public attribute of the leaf is not covered by the spec. + + Raised at wrap time under the default ``undeclared='raise'`` policy — the + census's failure mode is silence-by-omission (s3dol ADR-0011 D5), so an + attribute that would be served with unmapped keys/values must be + explicitly passed through, excluded, or added to the spec. + """ + + +# --------------------------------------------------------------------------- +# Codec + + +@dataclass(frozen=True) +class Codec: + """An encoder/decoder pair for one role (type of interest). + + ``encoder`` maps outer→inner (the direction of arguments going in); + ``decoder`` maps inner→outer (the direction of results coming out). + Same field semantics as ``dol.trans.Codec``; redefined here to keep this + leaf module dependency-free within dol. + """ + + encoder: Callable[[Any], Any] + decoder: Callable[[Any], Any] + + def __iter__(self): + return iter((self.encoder, self.decoder)) + + +def _identity(x): + return x + + +def _fuse(funcs): + """Compose single-argument functions left-to-right into one callable.""" + funcs = [f for f in funcs if f is not _identity] + if not funcs: + return _identity + if len(funcs) == 1: + return funcs[0] + + def fused(x, _funcs=tuple(funcs)): + for f in _funcs: + x = f(x) + return x + + return fused + + +# --------------------------------------------------------------------------- +# Spec introspection: find role TypeVars at paths inside annotations + +# A path is a tuple of steps; each step is (origin, arg_index). The empty path +# means the annotation IS the role ("bare"). + + +class _RoleSite(NamedTuple): + role: str # TypeVar name + path: tuple # ((origin, arg_index), ...) + + +def _find_role_sites(ann, roles, path=()): + """Yield ``_RoleSite`` for each occurrence of a role TypeVar in ``ann``. + + ``roles`` maps TypeVar *name* -> TypeVar. Matching is by name, not + identity, so a user's ``KT = TypeVar('KT')`` matches a spec authored with + a different-but-same-named TypeVar (e.g. ``typing.KT`` re-exported by + dol). Name collisions across genuinely different roles are the user's + responsibility — roles are names here. + """ + if isinstance(ann, TypeVar): + if ann.__name__ in roles: + yield _RoleSite(ann.__name__, path) + return + origin = get_origin(ann) + if origin is None: + return + for i, arg in enumerate(get_args(ann)): + if isinstance(arg, list): + # Callable[[X, Y], R]: the parameter list arrives as a list. + # Descend so a role inside it is SEEN (and then refused by the + # path transformer — origin Callable is unmappable) rather than + # silently ignored. + for el in arg: + yield from _find_role_sites(el, roles, path + ((origin, i),)) + else: + yield from _find_role_sites(arg, roles, path + ((origin, i),)) + + +def _transformer_for_path(ann, path, role_func, *, where): + """Build f(value) applying ``role_func`` at ``path`` inside ``value``. + + ``ann`` is the (sub-)annotation the path descends into — carried along so + each container case can inspect its own type arguments. Supports the + container shapes we can map faithfully; anything else raises + ``UnsupportedSpecShape`` at compile time. + """ + if not path: + return role_func + (origin, index), rest = path[0], path[1:] + sub_ann = get_args(ann)[index] if get_args(ann) else Any + inner = _transformer_for_path(sub_ann, rest, role_func, where=where) + + if origin is list: + return lambda v: [inner(x) for x in v] + if origin in (set, frozenset): + return lambda v, _o=origin: _o(inner(x) for x in v) + if origin is tuple: + args = get_args(ann) + if len(args) == 2 and args[1] is Ellipsis: + # Variadic tuple[X, ...]: map every element. + return lambda v: tuple(inner(x) for x in v) + + def map_tuple(v, _i=index, _inner=inner): + return tuple(_inner(x) if j == _i else x for j, x in enumerate(v)) + + return map_tuple + if origin is dict: + if index == 0: + return lambda v: {inner(k): x for k, x in v.items()} + return lambda v: {k: inner(x) for k, x in v.items()} + if origin is _IteratorABC: + # Iterators are one-shot by contract: map lazily (streaming preserved). + return lambda v: map(inner, v) + if origin is _IterableABC: + # Iterable implies RE-iterable: materialize, because handing a + # one-shot map to code that iterates twice (or len()s) silently + # yields an empty second pass. + return lambda v: [inner(x) for x in v] + if origin is Union: + args = get_args(ann) + non_none = [a for a in args if a is not type(None)] + if len(non_none) == 1: + # Optional[X]: map non-None, pass None through. + return lambda v: v if v is None else inner(v) + raise UnsupportedSpecShape( + f'Cannot map a role inside a non-Optional Union (in {where}: ' + f'{ann!r}): there is no reliable runtime discrimination between ' + f'union arms. Declare separate methods or use an explicit spec.' + ) + raise UnsupportedSpecShape( + f'Cannot map role inside {origin!r} (in {where}: {ann!r}). ' + f'Supported containers: list, set, frozenset, tuple, dict, ' + f'Iterable, Iterator, Optional. ' + f'Add an explicit method override or exclude the method.' + ) + + +@dataclass(frozen=True) +class InterfaceSpec: + """Per-method role placement, compiled from an annotated class. + + ``source`` is typically a ``Protocol`` class whose methods are annotated + with role TypeVars (KT, VT, ...). ``methods`` maps method name to a dict: + ``{param_name_or_'return': [(role, path), ...]}``. You can also build an + ``InterfaceSpec`` directly from that dict form (``from_dict``) when + annotations are unavailable — same mechanism, no typing required. + """ + + methods: Mapping[str, Mapping[str, tuple]] + signatures: Mapping[str, inspect.Signature] + source: Any = None + + @classmethod + def from_annotated(cls, source, *, roles=None): + """Compile a spec from an annotated (Protocol) class. + + ``roles``: iterable of role names to look for; default = names of the + TypeVars in ``source.__parameters__``, else {'KT', 'VT'}. + """ + if roles is None: + params = getattr(source, '__parameters__', ()) + roles = ( + {p.__name__: p for p in params if isinstance(p, TypeVar)} + or {'KT': None, 'VT': None} + ) + else: + roles = {name: None for name in roles} + methods = {} + signatures = {} + for name, func in _spec_functions(source): + hints = _resolved_hints(func, source) + sig = inspect.signature(func) + # Loudness one level down (s3dol ADR-0011 D5): every named param + # of a spec method must be annotated, or a forgotten role is + # silent. (*args/**kwargs stay conventional passthrough.) + for p in sig.parameters.values(): + if p.name in ('self', 'cls'): + continue + if p.kind in ( + inspect.Parameter.VAR_POSITIONAL, + inspect.Parameter.VAR_KEYWORD, + ): + continue + if p.annotation is inspect.Parameter.empty: + raise UnderAnnotatedSpecError( + f'{source.__name__}.{name}: parameter {p.name!r} has ' + f'no annotation. Annotate it with a role TypeVar if ' + f'it carries keys/values, or with a concrete type ' + f'(or Any) to declare it role-free.' + ) + sites = {} + for pname, ann in hints.items(): + found = list(_find_role_sites(ann, roles)) + if found: + # Validate each path is mappable now (refuse early). + for site in found: + _transformer_for_path( + ann, + site.path, + _identity, + where=f'{source.__name__}.{name}({pname})', + ) + sites[pname] = tuple(((s.role, s.path, ann) for s in found)) + methods[name] = sites + signatures[name] = sig + return cls(methods=methods, signatures=signatures, source=source) + + @classmethod + def from_dict(cls, methods, *, signatures=None, source=None): + """Build a spec from the explicit dict form. + + ``methods``: ``{method_name: {param_or_'return': [(role, path)] | role_str}}`` + where a bare role string means "the whole value has this role". + """ + def norm_occurrence(occ): + # Accept the user 2-tuple (role, path), the normalized 3-tuple + # (role, path, ann) — __reduce__ round-trips the normalized form + # back through here — and a bare role string. + if isinstance(occ, str): + return (occ, (), Any) + if len(occ) == 2: + role, path = occ + return (role, tuple(path), Any) + role, path, ann = occ + return (role, tuple(path), ann) + + norm = {} + for mname, params in methods.items(): + norm[mname] = { + p: ( + (norm_occurrence(v),) + if isinstance(v, str) + else tuple(norm_occurrence(occ) for occ in v) + ) + for p, v in params.items() + } + return cls(methods=norm, signatures=signatures or {}, source=source) + + +def _spec_functions(source): + """Yield (name, function) for the spec's declared methods (incl. dunders). + + Only the spec class's OWN plain functions count (inherited Protocol + methods carry the base's TypeVars, whose substitution is future work). + Members a spec cannot host refuse loudly instead of vanishing silently. + """ + for name, member in vars(source).items(): + if name in ( + '__init__', + '__subclasshook__', + '__init_subclass__', + '__class_getitem__', + ): + continue + if isinstance(member, (property, staticmethod, classmethod)): + raise UnsupportedSpecShape( + f'{source.__name__}.{name}: {type(member).__name__} members ' + f'are not supported in interface specs (yet) — they would ' + f'be silently skipped otherwise. Remove it or use a plain ' + f'method.' + ) + if inspect.isfunction(member): + yield name, member + + +def _resolved_hints(func, owner): + """``get_type_hints`` with the owner's module globals, resolving strings.""" + module = inspect.getmodule(owner) + globalns = getattr(module, '__dict__', {}) + return get_type_hints(func, globalns=globalns) + + +# --------------------------------------------------------------------------- +# The flat codec stack + + +def _fused_role_funcs(stack, *, direction): + """Fuse a stack of ``{role: Codec}`` layers into ``{role: callable}``. + + ``stack`` is ordered innermost-first (append order: the first layer + applied to the leaf is index 0). Encoders run outer->inner (reversed + stack order); decoders run inner->outer (stack order). + """ + roles = set() + for layer in stack: + roles.update(layer) + out = {} + for role in roles: + if direction == 'encode': + funcs = [ + layer[role].encoder for layer in reversed(stack) if role in layer + ] + else: + funcs = [layer[role].decoder for layer in stack if role in layer] + out[role] = _fuse(funcs) + return out + + +# --------------------------------------------------------------------------- +# Method-plan compilation + + +def _compile_method_plan(name, sites, leaf_method, sig, encoders, decoders): + """Compile one boundary method: encode role args, call leaf, decode result. + + ``sites``: {param_name_or_'return': ((role, path, ann), ...)}. + Returns a callable(*args, **kwargs) with the leaf method baked in. + """ + # Build per-parameter transformers (outer -> inner). Integer site keys + # mean positional index (dict-form specs), resolved against the outer + # signature when one exists; when none does (builtin slots like + # dict.__getitem__ have no text signature on 3.10), compile a purely + # positional plan instead. + positional_only_plan = False + if any(isinstance(p, int) for p in sites): + if sig is None: + positional_only_plan = True + else: + param_names_by_index = list(sig.parameters) + sites = { + (param_names_by_index[p] if isinstance(p, int) else p): v + for p, v in sites.items() + } + + param_transforms = {} # pname -> callable + return_transform = None + for pname, occurrences in sites.items(): + funcs = [] + for role, path, ann in occurrences: + role_func = (encoders if pname != 'return' else decoders).get( + role, _identity + ) + if role_func is _identity: + continue + funcs.append( + _transformer_for_path(ann, path, role_func, where=name) + ) + if not funcs: + continue + fused = _fuse(funcs) + if pname == 'return': + return_transform = fused + continue + param = sig.parameters.get(pname) if sig is not None else None + if param is not None: + if param.kind is inspect.Parameter.VAR_KEYWORD: + raise UnsupportedSpecShape( + f'{name}: role on a **kwargs parameter ({pname!r}) is ' + f'not supported — keyword names as keys have no ' + f'annotation channel.' + ) + if param.kind is inspect.Parameter.VAR_POSITIONAL: + # bound.arguments holds a TUPLE for *args: map elementwise + # (a bare-role transform applied to the tuple itself would + # silently corrupt). + fused = (lambda f: lambda tup: tuple(f(x) for x in tup))(fused) + if param.default is None: + # A None default lives in the LEAF's domain and, on 3.10, + # get_type_hints implicitly wraps `x: KT = None` in Optional. + # Normalize both: never transform None. + fused = (lambda f: lambda v: v if v is None else f(v))(fused) + param_transforms[pname] = fused + + if not param_transforms and return_transform is None: + # Spec'd but no active roles in this stack: plain passthrough. + return leaf_method + + if positional_only_plan: + # No signature to resolve against (3.10 builtin slots): transform by + # positional index; keyword calls of role'd params are refused loudly. + idx_transforms = { + p: t for p, t in param_transforms.items() if isinstance(p, int) + } + + def plan(*args, **kwargs): + args = tuple( + idx_transforms[i](a) if i in idx_transforms else a + for i, a in enumerate(args) + ) + if len(args) <= max(idx_transforms, default=-1): + raise TypeError( + f'{name}: role-bearing positional argument(s) ' + f'{sorted(idx_transforms)} must be passed positionally ' + f'(no signature is available to resolve keyword calls).' + ) + result = leaf_method(*args, **kwargs) + if return_transform is not None: + result = return_transform(result) + return result + + return plan + + # Fast path: single role'd parameter, first positional slot with no + # default. Keyword calls of that param are handled by name (POSITIONAL_ + # OR_KEYWORD contracts include them), without a Signature.bind per call. + param_names = list(sig.parameters) if sig is not None else [] + _first = sig.parameters[param_names[0]] if param_names else None + if ( + sig is not None + and param_names + and set(param_transforms) <= {param_names[0]} + and _first.default is inspect.Parameter.empty + and _first.kind + in ( + inspect.Parameter.POSITIONAL_ONLY, + inspect.Parameter.POSITIONAL_OR_KEYWORD, + ) + ): + first_transform = param_transforms.get(param_names[0]) + first_name = param_names[0] + + if first_transform is None: + + def plan(*args, **kwargs): + return return_transform(leaf_method(*args, **kwargs)) + + else: + + def plan(*args, **kwargs): + if args: + args = (first_transform(args[0]),) + args[1:] + elif first_name in kwargs: + # Re-emit positionally: the caller used the SPEC's name, + # which the leaf's own parameter may not share. + kwargs = dict(kwargs) + args = (first_transform(kwargs.pop(first_name)),) + result = leaf_method(*args, **kwargs) + if return_transform is not None: + result = return_transform(result) + return result + + return plan + + if sig is None: + raise UnsupportedSpecShape( + f'Method {name!r} has role-bearing named parameters but no ' + f'inspectable signature to bind against.' + ) + + def plan(*args, **kwargs): + bound = sig.bind(*args, **kwargs) + for pname, transform in param_transforms.items(): + if pname in bound.arguments: + bound.arguments[pname] = transform(bound.arguments[pname]) + result = leaf_method(*bound.args, **bound.kwargs) + if return_transform is not None: + result = return_transform(result) + return result + + return plan + + +def _outer_signature(leaf, name, spec): + """The signature calls are bound against — the OUTER contract. + + The spec's signature is authoritative when it exists: the caller talks to + the interface the spec declares, and the leaf's own parameter *names* are + irrelevant (dict names its key ``key``; the spec may say ``k``). Calls are + re-emitted positionally/keyword exactly as bound, so the leaf receives + them as the caller sent them. Falls back to the leaf's signature for + dict-form specs without signatures. + """ + sig = spec.signatures.get(name) + if sig is None: + func = getattr(type(leaf), name, None) or getattr(leaf, name, None) + try: + sig = inspect.signature(func) + except (TypeError, ValueError): + return None + params = list(sig.parameters.values()) + if params and params[0].name in ('self', 'cls'): + params = params[1:] + sig = sig.replace(parameters=params) + return sig + + +# --------------------------------------------------------------------------- +# The proxy + + +class InterfaceProxy: + """Base class for generated flat proxies. Instances hold the whole wrap. + + State (all under proxy-private names, wrapt's ``_self_`` lesson): + ``_self_leaf`` (strong ref, the innermost object — also ``__wrapped__``), + ``_self_spec``, ``_self_stack`` (tuple of {role: Codec}, innermost-first), + ``_self_plans`` (compiled boundary callables), plus the policy fields. + """ + + _self_passthrough = frozenset() + + def __init__(self, *args, **kwargs): + # Two construction modes: wrapping an existing instance (internal, + # via interface_wrap) or constructing the leaf (class-wrap mode). + raise TypeError( + 'InterfaceProxy subclasses are built via interface_wrap(...)' + ) + + @property + def __wrapped__(self): + return self._self_leaf + + def _encode_role(self, role, value): + """Map an outer-domain value of ``role`` to the leaf domain (total).""" + return self._self_encoders.get(role, _identity)(value) + + def _decode_role(self, role, value): + """Map a leaf-domain value of ``role`` outward (the inverse walk).""" + return self._self_decoders.get(role, _identity)(value) + + def __repr__(self): + return ( + f'<{type(self).__name__} of {self._self_leaf!r} ' + f'with {len(self._self_stack)} codec layer(s)>' + ) + + def __reduce__(self): + spec_ref = self._self_spec.source or dict(self._self_spec.methods) + return ( + _rebuild_interface_wrap, + ( + self._self_leaf, + spec_ref, + tuple(self._self_stack), + self._self_undeclared, + tuple(sorted(self._self_passthrough)), + ), + ) + + def __getattr__(self, name): + # Only reached when normal lookup fails: plans and passthroughs first. + if name.startswith('__') and name.endswith('__'): + # Explicit dunder access must NOT escape to the leaf: forwarding + # would hand out raw leaf-bound methods (s.__contains__('outer_k') + # silently answering in the wrong key domain). Plain + # AttributeError keeps hasattr-style duck typing honest. + raise AttributeError(name) + if name.startswith('_'): + return getattr( + object.__getattribute__(self, '_self_leaf'), name + ) + if name in object.__getattribute__(self, '_self_passthrough'): + return getattr(object.__getattribute__(self, '_self_leaf'), name) + raise UndeclaredAttributeError( + f'{name!r} is not in the interface spec of this wrap. ' + f"Add it to the spec, or pass passthrough={{'{name}'}} to " + f'interface_wrap to forward it verbatim (unmapped keys/values!).' + ) + + +def _rebuild_interface_wrap(leaf, spec_ref, stack, undeclared, passthrough): + """Pickle reconstructor: recompile the wrap from values (no dynamic class).""" + return interface_wrap( + leaf, + spec=spec_ref, + _stack=stack, + undeclared=undeclared, + passthrough=passthrough, + ) + + +_DUNDER_METHOD_TEMPLATE = ''' +def {name}(self, *args, **kwargs): + return self._self_plans[{name!r}](*args, **kwargs) +''' + +_proxy_class_cache = {} + + +def _build_proxy_class(leaf_type, spec, method_names, class_name=None): + """Generate (and cache) the proxy class for (leaf_type, spec, surface). + + The class namespace holds one dispatching method per spec'd method the + leaf actually has — dunders included, so implicit special-method lookup + works. Capability mirroring: a method the leaf lacks is NOT given to the + proxy class. + """ + # Cache only source-backed specs: the source class is a stable, hashable + # key the cache holds strongly. Dict-form specs (no source) build a fresh + # class — caching them by id() risks collisions after GC id-reuse and + # unbounded growth otherwise. + key = None + if spec.source is not None: + key = (leaf_type, spec.source, tuple(method_names)) + cached = _proxy_class_cache.get(key) + if cached is not None: + return cached + ns = {} + for name in method_names: + exec(_DUNDER_METHOD_TEMPLATE.format(name=name), {}, ns) + if '__getitem__' in ns and '__iter__' not in ns: + # Without this, Python's legacy sequence protocol would invent + # iteration from __getitem__(0), __getitem__(1), ... — feeding int + # keys through the key encoder, silently. Louder to refuse. + def __iter__(self): + raise TypeError( + f'{type(self).__name__} is not iterable: __iter__ is not in ' + f'its interface spec (and sequence-protocol fallback over ' + f'__getitem__ would silently feed integer keys through the ' + f'key codec).' + ) + + ns['__iter__'] = __iter__ + ns['__module__'] = __name__ + cls_name = class_name or f'{leaf_type.__name__}InterfaceProxy' + cls = type(cls_name, (InterfaceProxy,), ns) + if key is not None: + _proxy_class_cache[key] = cls + return cls + + +# --------------------------------------------------------------------------- +# Public entry point + + +def interface_wrap( + obj, + *, + spec, + codecs: Optional[Mapping[str, Codec]] = None, + undeclared: str = 'raise', + passthrough: _IterableABC = (), + _stack: Optional[tuple] = None, +): + """Wrap ``obj`` (an instance) with boundary codecs per an interface spec. + + :param obj: the object to wrap, or an existing ``InterfaceProxy`` (in + which case the new codec layer extends the flat stack over the SAME + leaf — wrapping never nests). + :param spec: an annotated (Protocol) class, an ``InterfaceSpec``, or the + explicit dict form accepted by ``InterfaceSpec.from_dict``. + :param codecs: one codec layer: ``{role_name: Codec(encoder, decoder)}``. + :param undeclared: policy for public leaf attributes absent from the + spec: ``'raise'`` (default — loud, at wrap time), ``'passthrough'`` + (forward verbatim; documents itself as unmapped), ``'exclude'`` + (hide: touching them raises at use time). + :param passthrough: explicit names to forward verbatim regardless. + """ + if undeclared not in ('raise', 'passthrough', 'exclude'): + raise ValueError( + f"undeclared must be 'raise', 'passthrough' or 'exclude', " + f'got {undeclared!r}' + ) + # --- normalize the spec + if isinstance(spec, InterfaceSpec): + spec_obj = spec + elif isinstance(spec, type): + spec_obj = InterfaceSpec.from_annotated(spec) + elif isinstance(spec, Mapping): + spec_obj = InterfaceSpec.from_dict(spec) + else: + raise TypeError(f'Cannot interpret spec: {spec!r}') + + # --- normalize the stack + if isinstance(obj, InterfaceProxy): + leaf = obj._self_leaf + base_stack = tuple(obj._self_stack) + passthrough = frozenset(passthrough) | obj._self_passthrough + else: + leaf = obj + base_stack = tuple(_stack or ()) + if isinstance(leaf, type): + raise TypeError( + 'interface_wrap wraps instances in this prototype; ' + 'class-wrapping is future work (see the design doc).' + ) + # Mixed-architecture stacks: flat-model guarantees (encode/decode + # totality, __wrapped__ = raw backend, pickle uniformity) are scoped to + # pure interface_wrap stacks. Wrapping a legacy dol Store is allowed but + # the Store chain below is opaque to us — say so. + try: + from dol.base import Store as _LegacyStore + + if isinstance(leaf, _LegacyStore): + import warnings + + warnings.warn( + 'interface_wrap over a legacy dol Store: the Store (and its ' + '.store chain) is treated as an opaque leaf — __wrapped__ ' + 'is the Store, not the raw backend, and flat-stack ' + 'guarantees apply only to the layers above it.', + stacklevel=2, + ) + except ImportError: # pragma: no cover - dol.base always importable here + pass + stack = base_stack + ((dict(codecs),) if codecs else ()) + + # --- role sanity (loudness): every codec role must appear in the spec + used_roles = { + role + for sites in spec_obj.methods.values() + for occurrences in sites.values() + for role, _path, _ann in occurrences + } + for layer in stack: + unknown = set(layer) - used_roles + if unknown: + raise InterfaceWrapError( + f'Codec layer names role(s) {sorted(unknown)} that occur ' + f'nowhere in the spec (spec roles: {sorted(used_roles)}). ' + f'A codec that can never apply is almost certainly a mistake.' + ) + + # --- undeclared-surface policy (wrap time, loud by default) + passthrough = frozenset(passthrough) + public_attrs = { + n for n in dir(leaf) if not n.startswith('_') + } + undeclared_names = public_attrs - set(spec_obj.methods) - passthrough + if undeclared_names and undeclared == 'raise': + raise UndeclaredAttributeError( + f'The leaf exposes public attributes not covered by the spec: ' + f'{sorted(undeclared_names)}. Methods among these would receive ' + f'UNMAPPED keys/values through this wrap (the #83 bug class). ' + f"Add them to the spec, or pass undeclared='passthrough' / " + f"'exclude', or list them in passthrough=... explicitly." + ) + if undeclared == 'passthrough': + passthrough = passthrough | undeclared_names + + # --- compile + encoders = _fused_role_funcs(stack, direction='encode') + decoders = _fused_role_funcs(stack, direction='decode') + present = [ + name for name in spec_obj.methods if hasattr(leaf, name) + ] + plans = {} + for name in present: + sig = _outer_signature(leaf, name, spec_obj) + plans[name] = _compile_method_plan( + name, + spec_obj.methods[name], + getattr(leaf, name), + sig, + encoders, + decoders, + ) + + cls = _build_proxy_class(type(leaf), spec_obj, tuple(present)) + proxy = object.__new__(cls) + object.__setattr__(proxy, '_self_leaf', leaf) + object.__setattr__(proxy, '_self_spec', spec_obj) + object.__setattr__(proxy, '_self_stack', stack) + object.__setattr__(proxy, '_self_plans', plans) + object.__setattr__(proxy, '_self_encoders', encoders) + object.__setattr__(proxy, '_self_decoders', decoders) + object.__setattr__(proxy, '_self_undeclared', undeclared) + object.__setattr__(proxy, '_self_passthrough', passthrough) + return proxy diff --git a/dol/tests/test_interface_wrap.py b/dol/tests/test_interface_wrap.py new file mode 100644 index 00000000..c4ebb682 --- /dev/null +++ b/dol/tests/test_interface_wrap.py @@ -0,0 +1,652 @@ +"""Tests for the Option G prototype: dol/_interface_wrap.py. + +Covers the #83/#86 census shapes (scalar key methods, iterable-of-keys, +key-value iterators), flat stacking (wrap-of-wrap extends the stack, never +nests), the no-double-apply guarantees (internal self-calls; prefix-owning +leaves), the pickle matrix that today's delegation machinery fails, the +undeclared-surface loudness policy, and lazy iterator mapping. +""" + +import pickle +import pytest +from typing import Callable, Iterable, Iterator, Optional, Protocol, TypeVar + +from dol._interface_wrap import ( + Codec, + InterfaceSpec, + InterfaceProxy, + InterfaceWrapError, + UndeclaredAttributeError, + UnsupportedSpecShape, + interface_wrap, +) + +KT = TypeVar('KT') +VT = TypeVar('VT') + + +# --- module-level (picklable) codec functions ------------------------------- + + +def add_json(k): + return k + '.json' + + +def strip_json(k): + return k[:-5] + + +def prefix_x(k): + return 'x/' + k + + +def strip_x(k): + return k[2:] + + +def int_to_str(v): + return str(v) + + +def str_to_int(v): + return int(v) + + +json_key_codec = Codec(encoder=add_json, decoder=strip_json) +x_key_codec = Codec(encoder=prefix_x, decoder=strip_x) +value_codec = Codec(encoder=int_to_str, decoder=str_to_int) + + +# --- a leaf in the shape of the #83 census ---------------------------------- + + +class Bucket: + """A backend with keyed non-Mapping methods — the #83 census shape. + + Owns its own prefix arithmetic (like s3dol's S3BucketReader): public keys + are relative; wire keys are prefixed. Methods use the leaf's own public + interface internally, so they are correct on the bare leaf — the property + the boundary model must preserve. + """ + + def __init__(self, wire=None, *, prefix='logs/'): + self.wire = wire if wire is not None else {} + self.prefix = prefix + + # internal (wire-domain) helpers + def _wire_key(self, k): + return self.prefix + k + + # the Mapping-ish surface + def __getitem__(self, k): + return self.wire[self._wire_key(k)] + + def __setitem__(self, k, v): + self.wire[self._wire_key(k)] = v + + def __delitem__(self, k): + del self.wire[self._wire_key(k)] + + def __iter__(self): + p = self.prefix + return (w[len(p):] for w in self.wire if w.startswith(p)) + + def __contains__(self, k): + return self._wire_key(k) in self.wire + + # the census shapes + def url_for(self, k, *, expires_in=3600): + return f'https://x.example/{self._wire_key(k)}?e={expires_in}' + + def delete_many(self, keys): + for k in keys: + del self[k] # internal self-call: stays below the boundary + + def items_page(self): + for k in self: + yield (k, self[k]) + + def copy_key(self, src, dst): + self[dst] = self[src] + + +class BucketInterface(Protocol[KT, VT]): + def __getitem__(self, k: KT) -> VT: ... + def __setitem__(self, k: KT, v: VT) -> None: ... + def __delitem__(self, k: KT) -> None: ... + def __iter__(self) -> Iterator[KT]: ... + def __contains__(self, k: KT) -> bool: ... + def url_for(self, k: KT, *, expires_in: int = 3600) -> str: ... + def delete_many(self, keys: Iterable[KT]) -> None: ... + def items_page(self) -> Iterator[tuple[KT, VT]]: ... + def copy_key(self, src: KT, dst: KT) -> None: ... + + +def mk_bucket(): + return Bucket({'logs/a.json': '1', 'logs/b.json': '2'}) + + +def wrap_bucket(bucket=None, **kwargs): + # `wire` and `prefix` are public data attributes of the leaf; the loudness + # policy makes forwarding them an explicit choice (this is the designed + # gesture, not a workaround). + kwargs.setdefault('passthrough', {'wire', 'prefix'}) + return interface_wrap( + bucket if bucket is not None else mk_bucket(), + spec=BucketInterface, + codecs=dict(KT=json_key_codec, VT=value_codec), + **kwargs, + ) + + +# --- the boundary invariant on the census shapes ---------------------------- + + +def test_mapping_surface(): + s = wrap_bucket() + assert s['a'] == 1 + assert sorted(s) == ['a', 'b'] + assert 'a' in s and 'zzz' not in s + s['c'] = 3 + assert s.__wrapped__.wire['logs/c.json'] == '3' + del s['c'] + assert 'c' not in s + + +def test_scalar_key_method_gets_mapped_key(): + """#83's minimal repro shape: url_for must see the leaf-domain key.""" + s = wrap_bucket() + assert s.url_for('a') == 'https://x.example/logs/a.json?e=3600' + # non-role kwargs pass through untouched + assert s.url_for('a', expires_in=60).endswith('?e=60') + + +def test_iterable_of_keys_arg(): + s = wrap_bucket() + s.delete_many(['a']) + assert sorted(s) == ['b'] + + +def test_iterator_of_pairs_return_is_lazy_and_mapped(): + s = wrap_bucket() + pages = s.items_page() + assert iter(pages) is pages # stayed an iterator (lazy) + assert sorted(pages) == [('a', 1), ('b', 2)] + + +def test_two_key_params(): + s = wrap_bucket() + s.copy_key('a', 'target') + assert s['target'] == 1 + assert 'logs/target.json' in s.__wrapped__.wire + + +def test_internal_self_calls_do_not_double_apply(): + """delete_many calls del self[k] internally; that call must stay below + the boundary (single application of the key codec).""" + s = wrap_bucket() + s.delete_many(['a', 'b']) + assert s.__wrapped__.wire == {} + + +def test_prefix_owning_leaf_composes(): + """The leaf's own prefix arithmetic composes with the stack: encoder maps + outer->leaf-public, leaf maps leaf-public->wire. No double-apply.""" + s = wrap_bucket() + # outer 'a' -> leaf 'a.json' -> wire 'logs/a.json' + assert s.url_for('a').split('/', 3)[-1] == 'logs/a.json?e=3600' + + +# --- flat stacking ---------------------------------------------------------- + + +def test_wrap_of_wrap_extends_stack_not_nests(): + s1 = wrap_bucket() + s2 = interface_wrap( + s1, spec=BucketInterface, codecs=dict(KT=x_key_codec) + ) + assert isinstance(s2, InterfaceProxy) + assert s2.__wrapped__ is s1.__wrapped__ # SAME leaf: no nesting + assert len(s2._self_stack) == 2 + # outer key 'a' -> +'x/' is OUTER-most? No: second wrap is outer. + # Encoders run outer->inner: x_key first? Layer order: stack is + # innermost-first, so layer0=json, layer1=x. Encode: x then json. + assert s2._encode_role('KT', 'a') == 'x/a.json' + # inverse mapping (the missing primitive of #83 §5.4): decoder walk + assert s2._decode_role('KT', 'x/a.json') == 'a' + + +def test_stacked_read_write_roundtrip(): + leaf = Bucket({}) + s1 = wrap_bucket(leaf) + s2 = interface_wrap(s1, spec=BucketInterface, codecs=dict(KT=x_key_codec)) + s2['a'] = 7 + assert leaf.wire == {'logs/x/a.json': '7'} + assert s2['a'] == 7 + assert list(s2) == ['a'] + assert s2.url_for('a').split('/', 3)[-1].startswith('logs/x/a.json') + + +def test_deep_stack_flat_cost_object_graph(): + """Six layers: still ONE proxy, one leaf, six stack entries.""" + s = wrap_bucket() + for _ in range(5): + s = interface_wrap(s, spec=BucketInterface, codecs=dict(KT=x_key_codec)) + assert isinstance(s.__wrapped__, Bucket) # not a proxy: no nesting + assert len(s._self_stack) == 6 + assert s._encode_role('KT', 'a') == 'x/x/x/x/x/a.json' + + +# --- pickling --------------------------------------------------------------- + + +def test_pickle_roundtrip_instance(): + s = wrap_bucket() + s2 = pickle.loads(pickle.dumps(s)) + assert s2['a'] == 1 + assert sorted(s2) == ['a', 'b'] + assert s2.url_for('a') == 'https://x.example/logs/a.json?e=3600' + + +def test_pickle_roundtrip_stacked(): + """Stacked wraps pickle — the case today's delegation machinery fails + (anonymous intermediate classes in the reduce payload).""" + s = interface_wrap( + wrap_bucket(), spec=BucketInterface, codecs=dict(KT=x_key_codec) + ) + s2 = pickle.loads(pickle.dumps(s)) + assert len(s2._self_stack) == 2 + assert s2._encode_role('KT', 'a') == 'x/a.json' + + +def test_pickle_preserves_transform_behavior_after_write(): + s = pickle.loads(pickle.dumps(wrap_bucket())) + s['new'] = 9 + assert s.__wrapped__.wire['logs/new.json'] == '9' + + +def test_pickle_with_lambda_codec_fails_loudly(): + s = interface_wrap( + mk_bucket(), + spec=BucketInterface, + codecs=dict(KT=Codec(encoder=lambda k: k, decoder=lambda k: k)), + passthrough={'wire', 'prefix'}, + ) + with pytest.raises(Exception): # PicklingError or AttributeError + pickle.dumps(s) + + +# --- loudness policies ------------------------------------------------------ + + +class Leaky: + def __getitem__(self, k): + return 42 + + def surprise_delete(self, k): + """A public keyed method the spec forgot.""" + + +class MinimalGet(Protocol[KT, VT]): + def __getitem__(self, k: KT) -> VT: ... + + +def test_undeclared_public_method_raises_at_wrap_time(): + with pytest.raises(UndeclaredAttributeError) as exc: + interface_wrap( + Leaky(), spec=MinimalGet, codecs=dict(KT=json_key_codec) + ) + assert 'surprise_delete' in str(exc.value) + + +def test_undeclared_passthrough_is_explicit_and_works(): + s = interface_wrap( + Leaky(), + spec=MinimalGet, + codecs=dict(KT=json_key_codec), + undeclared='passthrough', + ) + assert s.surprise_delete is not None # forwarded verbatim, by choice + + +def test_undeclared_exclude_hides_and_raises_on_use(): + s = interface_wrap( + Leaky(), + spec=MinimalGet, + codecs=dict(KT=json_key_codec), + undeclared='exclude', + ) + with pytest.raises(UndeclaredAttributeError): + _ = s.surprise_delete + + +def test_unknown_codec_role_raises(): + with pytest.raises(InterfaceWrapError): + interface_wrap( + Leaky(), + spec=MinimalGet, + codecs=dict(QT=json_key_codec), # QT occurs nowhere in the spec + undeclared='exclude', + ) + + +def test_unsupported_shape_refuses_at_wrap_time(): + class Bad(Protocol[KT, VT]): + def weird(self, k: Callable[[KT], int]) -> None: ... + + with pytest.raises(UnsupportedSpecShape): + InterfaceSpec.from_annotated(Bad) + + +# --- capability mirroring --------------------------------------------------- + + +def test_missing_leaf_method_not_resurrected(): + """A spec'd method the leaf lacks must NOT appear on the proxy (contrast + _filt_iter's __len__-resurrection bug).""" + + class NoLen: + def __getitem__(self, k): + return 1 + + class SpecWithLen(Protocol[KT, VT]): + def __getitem__(self, k: KT) -> VT: ... + def __len__(self) -> int: ... + + s = interface_wrap( + NoLen(), spec=SpecWithLen, codecs=dict(KT=json_key_codec), + undeclared='exclude', + ) + with pytest.raises(TypeError): + len(s) + + +# --- generalization beyond KT/VT -------------------------------------------- + + +QT = TypeVar('QT') + + +def test_arbitrary_role_lane(): + """The mechanism is role-generic: any TypeVar name is a codec lane.""" + + class Queryable(Protocol[QT]): + def search(self, q: QT) -> list: ... + + class Engine: + def search(self, q): + return [q] + + s = interface_wrap( + Engine(), + spec=Queryable, + codecs=dict(QT=Codec(encoder=str.upper, decoder=str.lower)), + ) + assert s.search('hello') == ['HELLO'] + + +# --- dict-form spec (no typing required) ------------------------------------ + + +def test_dict_form_spec(): + # Integer keys = positional parameter index (no typing, no name coupling + # to the leaf's own parameter names — dict names its params key/value). + spec = { + '__getitem__': {0: 'KT', 'return': 'VT'}, + '__setitem__': {0: 'KT', 1: 'VT'}, + } + d = {} + s = interface_wrap( + d, + spec=spec, + codecs=dict(KT=json_key_codec, VT=value_codec), + undeclared='exclude', + ) + s['a'] = 5 + assert d == {'a.json': '5'} + assert s['a'] == 5 + + +# --- optional / laziness edge cases ---------------------------------------- + + +def test_optional_key_param(): + class OptGet(Protocol[KT, VT]): + def find(self, k: Optional[KT]) -> Optional[VT]: ... + + class L: + def find(self, k): + return None if k is None else '7' + + s = interface_wrap( + L(), spec=OptGet, codecs=dict(KT=json_key_codec, VT=value_codec), + undeclared='exclude', + ) + assert s.find(None) is None + assert s.find('a') == 7 + + +# --- panel-driven hardening tests (refute round 1) --------------------------- + + +from dol._interface_wrap import UnderAnnotatedSpecError + + +def test_var_positional_role_maps_elementwise(): + """*keys: KT must encode each element, not the tuple (was a silent bug).""" + + class VarSpec(Protocol[KT]): + def delete(self, *keys: KT) -> None: ... + + class L: + def delete(self, *keys): + self.got = keys + + s = interface_wrap(L(), spec=VarSpec, codecs=dict(KT=json_key_codec), + undeclared='exclude') + s.delete('a', 'b') + assert s.__wrapped__.got == ('a.json', 'b.json') + + +def test_var_keyword_role_refuses(): + class KwSpec(Protocol[VT]): + def update_all(self, **kv: VT) -> None: ... + + class L: + def update_all(self, **kv): ... + + with pytest.raises(UnsupportedSpecShape): + interface_wrap(L(), spec=KwSpec, codecs=dict(VT=value_codec), + undeclared='exclude') + + +def test_unannotated_param_in_spec_method_refuses(): + """A spec method with an unannotated param is silence-by-omission one + level down — refuse at compile time.""" + + class Sloppy(Protocol[KT]): + def url_for(self, k) -> str: ... # forgot the annotation + + with pytest.raises(UnderAnnotatedSpecError): + InterfaceSpec.from_annotated(Sloppy) + + +def test_property_in_spec_refuses_not_vanishes(): + class WithProp(Protocol[KT]): + def __getitem__(self, k: KT) -> str: ... + rootdir = property(lambda self: '/') + + with pytest.raises(UnsupportedSpecShape): + InterfaceSpec.from_annotated(WithProp) + + +def test_none_default_never_encoded(): + """`k: KT = None`: the None default is leaf-domain; never encode it. + Also converges the 3.10 (implicit-Optional) vs 3.11+ compilation.""" + + class DefSpec(Protocol[KT]): + def latest(self, k: KT = None) -> str: ... + + class L: + def latest(self, k=None): + return f'got:{k}' + + s = interface_wrap(L(), spec=DefSpec, codecs=dict(KT=json_key_codec), + undeclared='exclude') + assert s.latest() == 'got:None' # default: untouched + assert s.latest(None) == 'got:None' # explicit None: untouched + assert s.latest('a') == 'got:a.json' # real key: encoded + + +def test_in_flight_iterator_survives_stack_extension(): + """Wrapping is copy-not-mutate: an iterator obtained before a new wrap + keeps the pipelines it was compiled with.""" + s1 = wrap_bucket() + it = iter(sorted(s1)) + first = it if isinstance(it, str) else None # noqa: just consume below + got_first = next(iter(sorted(s1))) + s2 = interface_wrap(s1, spec=BucketInterface, codecs=dict(KT=x_key_codec)) + # s1's own iteration is unaffected by s2's existence + assert sorted(s1) == ['a', 'b'] + assert len(s1._self_stack) == 1 and len(s2._self_stack) == 2 + assert got_first == 'a' + + +def test_unspecced_dunder_absent_not_leaked(): + """dict's __or__ must NOT be silently mirrored raw (the basepy-verified + transform-bypass leak): a dunder outside the spec simply doesn't exist + on the proxy — loud TypeError, no silent raw data.""" + d = {'a.json': '1'} + s = interface_wrap( + d, + spec={'__getitem__': {0: 'KT', 'return': 'VT'}}, + codecs=dict(KT=json_key_codec, VT=value_codec), + undeclared='exclude', + ) + with pytest.raises(TypeError): + s | {'b': 2} + + +def test_nested_iterator_of_iterators(): + class NestSpec(Protocol[KT]): + def batches(self) -> Iterator[Iterator[KT]]: ... + + class L: + def batches(self): + yield iter(['a.json']) + yield iter(['b.json']) + + s = interface_wrap(L(), spec=NestSpec, codecs=dict(KT=json_key_codec), + undeclared='exclude') + assert [list(b) for b in s.batches()] == [['a'], ['b']] + + +def test_wrapping_legacy_store_warns(): + from dol import wrap_kvs + + legacy = wrap_kvs({'a.json': '1'}, obj_of_data=str) + with pytest.warns(UserWarning, match='legacy dol Store'): + interface_wrap( + legacy, + spec={'__getitem__': {0: 'KT'}}, + codecs=dict(KT=json_key_codec), + undeclared='passthrough', + ) + + +# --- independent code-review round (findings 1-9) ---------------------------- + + +import copy + + +def test_dict_form_spec_pickles_and_copies(): + """Finding 2: __reduce__ round-trips the normalized 3-tuple form through + from_dict; copy.copy rides the same path.""" + spec = {'__getitem__': {0: 'KT', 'return': 'VT'}} + d = {'a.json': '1'} + s = interface_wrap(d, spec=spec, + codecs=dict(KT=json_key_codec, VT=value_codec), + undeclared='exclude') + s2 = pickle.loads(pickle.dumps(s)) + assert s2['a'] == 1 + s3 = copy.copy(s) + assert s3['a'] == 1 + assert s3.__wrapped__ is not None # shallow copy shares nothing broken + + +def test_explicit_dunder_access_is_loud(): + """Finding 3: s.__contains__ / s.__or__ must not silently hand out the + leaf's raw bound method — AttributeError keeps duck typing honest.""" + s = wrap_bucket() + with pytest.raises(AttributeError): + s.__or__ + # spec'd dunders remain accessible (they live on the class) + assert s.__contains__('a') is True + + +def test_no_sequence_protocol_iteration_leak(): + """A __getitem__-only spec must not let iter() invent integer-key + iteration via the legacy sequence protocol.""" + s = interface_wrap( + {'a.json': '1'}, + spec={'__getitem__': {0: 'KT', 'return': 'VT'}}, + codecs=dict(KT=json_key_codec, VT=value_codec), + undeclared='exclude', + ) + with pytest.raises(TypeError): + iter(s) + + +def test_keyword_call_of_specd_method(): + """Finding 4: POSITIONAL_OR_KEYWORD contracts include keyword calls; + the spec's name is the contract even when the leaf names it differently.""" + + class KwSpec(Protocol[KT]): + def url_for(self, key: KT) -> str: ... + + class L: + def url_for(self, target): # leaf uses a DIFFERENT param name + return 'u/' + target + + s = interface_wrap(L(), spec=KwSpec, codecs=dict(KT=json_key_codec), + undeclared='exclude') + assert s.url_for('a') == 'u/a.json' + assert s.url_for(key='a') == 'u/a.json' + + +def test_iterable_arg_survives_reiteration(): + """Finding 5: Iterable (re-iterable contract) args are materialized; + a leaf that iterates twice sees both passes.""" + + class TwoPass(Protocol[KT]): + def pairs(self, keys: Iterable[KT]) -> list: ... + + class L: + def pairs(self, keys): + return [list(keys), list(keys)] + + s = interface_wrap(L(), spec=TwoPass, codecs=dict(KT=json_key_codec), + undeclared='exclude') + assert s.pairs(['a']) == [['a.json'], ['a.json']] + + +def test_invalid_undeclared_policy_refuses(): + with pytest.raises(ValueError): + wrap_bucket(undeclared='riase') # typo must not silently mean exclude + + +def test_dict_kv_return_shape(): + """Review gap 9a: dict[KT, VT] returns map both keys and values.""" + + class BulkSpec(Protocol[KT, VT]): + def bulk(self, ks: list[KT]) -> dict[KT, VT]: ... + + class L: + def bulk(self, ks): + return {k: '7' for k in ks} + + s = interface_wrap(L(), spec=BulkSpec, + codecs=dict(KT=json_key_codec, VT=value_codec), + undeclared='exclude') + assert s.bulk(['a']) == {'a': 7} diff --git a/misc/docs/dol_issue86_design.md b/misc/docs/dol_issue86_design.md new file mode 100644 index 00000000..efe333a6 --- /dev/null +++ b/misc/docs/dol_issue86_design.md @@ -0,0 +1,392 @@ +# dol Discussion #86 — Option G: spec-carried boundary codecs on a flat proxy + +> Companion to [dol_issue83_design.md](dol_issue83_design.md) (options A–F) and +> [dol_issue18_design.md](dol_issue18_design.md) (the is-a plan). Responds to the +> maintainer's proposal (relayed in-session, 2026-08-10; summarized in §1 and posted to +> discussion #86): wrapt-inspired object proxies + KT/VT-annotated interface specs + +> flatten-and-compile codec stacks. Prototype: `dol/_interface_wrap.py` (private, +> additive, stdlib-only) + `dol/tests/test_interface_wrap.py`, branch +> `claude/option-g-interface-codecs`. Every claim below marked **[verified]** has +> running-code evidence (§12); this doc survived one adversarial panel round (4 lenses), +> which broke several first-draft claims — the corrections are folded in and flagged. + +## TL;DR + +The proposal decomposes into three mechanisms. Two survive adversarial review with +stated preconditions; the third inverts into a set of lessons: + +1. **A typed interface spec** — per method, *where* the types-of-interest (KT, VT, any + TypeVar "role") occur in arguments and returns, compiled from annotations (or an + explicit dict form) at wrap time. This is the "wrapper must be able to express *this + method takes a key*" capability that #83 §5.2 demands, and it **replaces** the + verified-broken `ingoing_key_methods`/`outcoming_key_methods`. **Sound, prototyped, + with three refusal layers making omission loud** (§2.3). Its honest weak edge: spec + authoring is real work, and loudness degrades one keyword at a time (§7). +2. **A flat codec stack, compiled once** (the "two lists" idea). The structurally + load-bearing part: re-wrapping *extends a list* instead of *nesting an object*, so — + **within pure-codec stacks** — the chain-walking family dissolves: `inner_most_key` + becomes a total fold, the missing **inverse** key mapping (#83 §5.4) exists by + construction, the leaf is a strong structural reference (#83 §5.6), and the measured + 420–510 ns/layer delegation tax collapses to one boundary hop (**3.6× faster at + depth 6, [verified]**). The scope limit is real and must be said plainly: `filt_iter` + (key-*set* change) and `cached_keys` (stateful source) are **not codecs**, so mixed + compositions still nest (§10). +3. **A wrapt-style transparent proxy carrier — take the lessons, not the goal.** + Running wrapt 2.3.0 **[verified]**: its proxy has *exactly* dol's #18 hole (inside a + wrapped method, `self` is the wrapped object — the proxy intercepts only the first + hop); pickling refuses loudly by default; the `__iter__` mistake is a ten-year + cautionary tale about eagerly-defined dunders poisoning duck typing; and on modern + CPython the pure-Python proxy measured *faster* than the C extension (83 vs 445 ns + attr reads) — a dependency-free carrier costs nothing. Universal transparency is a + tar pit; the prototype exposes **only the spec'd surface** and makes everything else + loud (§2.4). + +**The headline finding** (new evidence, corrects an overclaim in #86's own option F +verdict): **is-a wrapping does not fix #83 for backend-direct method bodies.** +A method like `cosmodol.replace` or `pydrivedol.get_url` that passes its key straight +to the backend still receives the outer key under is-a — hooks on the MRO don't help a +body that never routes through them **[verified, panel probe p4]**. Boundary +transformation is the only mechanism in the A–G space that serves that population, and +it serves it with a strong invariant: **a method correct on the bare leaf stays correct +under the stack** — *provided the codec laws hold* (§3.1). So Option G is not an +interim stand-in for is-a; the two serve disjoint populations (§8), and "both #83 and +#18 disappear under F" should be retired from the corpus. + +## 1. The proposal, restated + +The maintainer's proposal (2026-08-10, in-session): (a) study wrapt's object-proxy +design; (b) wrap incoming/outgoing keys and values in *all* methods of an object, not +just the Mapping dunders, with codecs hooked up automatically per a specification — +e.g. a Protocol class annotated with KT/VT; (c) generalize beyond keys/values to any +small set of "types of interest"; (d) accumulate wrapping layers in two lists (encoders, +decoders) and compile them for speed and validation; (e) take special care that a +method calling another method does not double-apply transforms. + +Point (e) turned out to be the crux, and the answer is structural rather than careful: +apply codecs **only at the proxy boundary** and keep `self` inside method bodies bound +to the leaf. Then internal `self.x()` calls never cross the boundary, and transforms +apply exactly once **[verified: counting-encoder probe, 1 invocation through a spec'd +method that internally calls another spec'd method]**. + +## 2. The mechanism (as prototyped) + +A wrap is `(leaf, spec, stack)` — one proxy object, however many layers. + +### 2.1 The spec + +```python +KT, VT = TypeVar('KT'), TypeVar('VT') + +class BucketInterface(Protocol[KT, VT]): + def __getitem__(self, k: KT) -> VT: ... # Mapping dunders are ordinary + def __iter__(self) -> Iterator[KT]: ... # spec entries — no privileged + def __contains__(self, k: KT) -> bool: ... # surface (#83 §5.2) + def url_for(self, k: KT, *, expires_in: int = 3600) -> str: ... + def delete_many(self, keys: Iterable[KT]) -> None: ... + def items_page(self) -> Iterator[tuple[KT, VT]]: ... +``` + +Compilation walks `get_type_hints` + `signature` per method, recording the *paths* at +which role TypeVars occur. Supported shapes: bare, `list/set/frozenset/tuple/dict[...]`, +`Iterable/Iterator[...]` (lazily mapped, nesting included), `Optional[...]`, +`*args: KT` (elementwise). Everything else **refuses at wrap time** +(`UnsupportedSpecShape`) — including roles inside `Callable[[KT], …]` (contravariant +positions would hand inner keys to outer callbacks) and `**kwargs: VT` (keyword names +as keys have no annotation channel). Roles are matched **by TypeVar name, not +identity** — deliberate: dol itself ships two distinct `KT` objects (`dol.KT` is +`typing.KT`; `dol.caching.KT` is its own) **[verified]**, so identity matching would +silently classify a user's same-named `KT` as "not a key" — the exact silent hole the +mechanism exists to kill. The residual (two *different* roles sharing a name) is the +user's naming responsibility. + +The **spec's signature is the outer contract**: calls bind against it, so the leaf's +own parameter names are irrelevant (`dict` calls its key `key`; the spec may say `k`). +A dict form exists for annotation-free use — `{'__getitem__': {0: 'KT', 'return': +'VT'}}`, integer keys = positional index — same compiled algebra, same loudness rules. +(Yes, this is #14's rejected Option-B notation as a *fallback input format*; the +difference from B is everything in §2.3 and §7.) + +### 2.2 The stack + +`stack` is a tuple of layers, each `{role: Codec(encoder, decoder)}`. Wrapping an +already-wrapped proxy builds a **new** proxy whose stack is a copied-and-extended tuple +over the **same leaf** — never a wrapper-of-wrapper, and never mutation: an in-flight +lazy iterator keeps the fused pipelines it was compiled with **[verified]**. Per role, +encoders fuse outer→inner and decoders inner→outer into single callables; per method, a +plan binds parameters and return paths to them (fast path for the dunder-shaped common +case; `Signature.bind` for the general case). + +What this buys, each previously a named open problem — **scoped to pure-codec stacks**: + +- `inner_most_key(w, k)` ≡ `w._encode_role('KT', k)` — total, no `.store` walk, no + non-`Store`-layer hazard, because there *are* no layers at runtime. +- The **inverse mapping** (#83 §5.4 — needed by anything returning keys: `prefixes`, + `walk`, listings) is `w._decode_role('KT', k)`. It exists by construction; today it + does not exist at all. +- The leaf is a strong structural reference, `w.__wrapped__` (#83 §5.6) — no weakref + registry for the innermost direction. **Correction from the panel**: this is *not* + the #16/#10 write-back boundary. That engine needs the raw-*scalar* surface — leaf + *plus the value-codec slice* of the stack, below only path/view layers **[verified: + `path_set_writeback` against the raw leaf raises `PathCreationError`; against the + value-codec surface it works]**. A flat design can provide that as a *stack slice* + (a derived proxy over the same leaf with a stack prefix) — future work, and the + honest statement is that the outermost/innermost directionality clash of #83 §5 + becomes two accessors *plus a slicing operation*, not two accessors. +- Per-op cost is one boundary call at any depth: **[measured]** getitem 675 ns at 6 + layers vs 2426 ns nested (and 309 vs 383 ns at depth 1); iteration 206 µs vs 700 µs + per 1000 keys at depth 6. + +### 2.3 Loudness — three refusal layers + +The census's failure mode is silence-by-omission (ADR-0011 D5), so omission is loud at +every level the mechanism can see: + +1. **Undeclared public attributes** of the leaf (`undeclared='raise'`, the default): + wrap-time refusal naming the attributes; escapes are explicit + (`'passthrough'` / `'exclude'` / `passthrough={...}` per name). +2. **Unannotated parameters inside spec'd methods** (`UnderAnnotatedSpecError`): + a spec author who writes `url_for(self, k)` forgot the annotation, and `k` would + silently receive outer keys — refused at compile time. (Panel-found hole, closed.) +3. **Unsupported shapes** (`UnsupportedSpecShape`): refuse rather than guess (#83 + §5.7) — including properties/classmethods in specs, which would otherwise vanish + silently. + +Two honest caveats the panel established. First, **the raise default is noisy on real +leaves** — a bare `dict` drags 11 MutableMapping publics; `dol.Files` ~20; a boto3 +client 122 — so users will reach for `'passthrough'`, and after that, omission is +silent again. The guard therefore needs **two desks**: wrap-time loudness at the +user's desk (this mechanism), plus an adapter-side reflective conformance test at the +author's desk (ADR-0011 D5's shape — unchanged and still recommended). Second, +**parameterless under-declaration survives**: `keys: Iterable` (author forgot `[KT]`) +is indistinguishable from a deliberately role-free iterable. Layer 2 catches the +*unannotated* case; the *under-parameterized* case remains a review-time judgement. + +### 2.4 The carrier — and what it refuses to be + +A generated class per `(leaf type, spec, surface)`, cached; instances hold +`(leaf, spec, stack, compiled plans)` under `_self_`-prefixed names (wrapt's lesson). +The class namespace contains **exactly the spec'd methods the leaf actually has** — +capability mirroring (a leaf without `__len__` yields a proxy without `__len__`; +contrast `_filt_iter`'s verified `__len__`-resurrection bug), and **nothing else**: + +- A dunder outside the spec does not exist on the proxy — `proxy | other` on a + dict-leaf wrap raises `TypeError` **[verified]** instead of silently returning raw + inner data the way today's class-wrap `DelegatedAttribute`s do (`__or__`/`copy`/ + `fromkeys` leak, basepy-verified). Loud beats transparent. +- `__eq__`/`__hash__` are **deliberately not defined** in the prototype (object + identity semantics). The panel is right that no member of {eq, hash, + len-under-filtering} is derivable from role mapping; each is a policy decision. + Today's Store is itself incoherent here (eq compares outer views, hash hashes the + inner store) — the redesign should decide this family *explicitly* rather than + inherit the incoherence. Open question §11.4. +- `__class__` transparency (isinstance-as-leaf) is **not** implemented: wrapt shows + it's feasible, but it changes capability detection and interacts with #5 (wrapper- + class control) — a policy to co-design, not a default to sneak in. +- Pickling: `__reduce__` → `(rebuild, (leaf, spec_source, stack, policy))` — no + dynamic class in the payload; the rebuild recompiles. **Honest scope [panel- + corrected]**: this repairs *by construction* the anonymous-intermediate-class + failure (today's stacked wraps, matrix case g) and keeps instance wraps working; + it does **not** repair the `Files`/decorator-form failures, whose root cause is the + *leaf's own class* being shadowed and by-name unreachable — that needs the name- + shadowing fix in dol itself, and `Files` is route-2 (`mk_relative_path_store`) + anyway, untouched by any wrap_kvs-side change. Constraints: codecs and spec source + must be module-level/picklable; lambdas fail loudly **[verified]**. + +### 2.5 The codec laws (the invariant's fine print) + +"A method correct on the bare leaf stays correct under any codec stack" was **refuted +as stated** by the panel and is hereby restated with its preconditions. It holds iff: + +1. **The key decoder is total and injective on the leaf's actually-occurring keys.** + dol's own `prefixed`/`suffixed` codecs violate this on out-of-band keys (decode + `'z.txt'` → `''` — ADR-0006's corruption family), and the flat model inherits that + corruption exactly as today's model does. Filtering first (ADR-0006's + `Pipe(filt_iter.prefixes(p), KeyCodecs.prefixed(p))`) remains the blessed guard — + and filters are not codec layers, so this composition is a mixed stack (§10). +2. **Encoder and decoder are mutual inverses on both domains.** dol permits one-sided + key transforms today (`kv_wrap.outcoming_keys`); under a one-sided codec a method + that returns its own key argument returns a *different* key. Typed/total codecs + (the two in-tree TODOs asking for inspectable codec types) graduate from + nice-to-have to the enforcement hook for these laws — promoted to §11.3. +3. **Container-shaped role arguments are copied at the boundary** (encode = build a + new list/dict/tuple), so a leaf method that *mutates* its argument in place loses + that side channel silently. Rare, real, now documented. + +Lazy iterator mapping adds a temporal caveat: a data-dependent decode failure raises +at *consumption* time, arbitrarily far from the call — loud but late. An eager/strict +option is cheap future work. + +## 3. What the panel verified as HOLDING + +- **No-double-apply is structural** — spec'd method calling another spec'd method via + `self`: encoder fires exactly once (counting probe). +- **The s3dol composition** — a leaf owning its own prefix arithmetic (`url_for` + routing through `_id_of_key`) under a boundary key codec: byte-identical to ground + truth, at one and two stacked layers; the unmapped-key counterfactual reproduces + #83, the rebind counterfactual double-applies. +- **P0 additivity** — nothing in the prototype touches existing modules. +- **Layer metadata half-plumbed** — every Wrap class already carries `_class_trans` + through `__reduce__`; a layer-list accumulator can ride that channel in P2. +- **G survives is-a landing** (§8) — the one steelman that failed to kill it. + +## 4. What it does not cover (complete inventory) + +From the s3dol shape census, the shapes the spec vocabulary cannot express — these +**must remain sibling stores, handles, or free functions** (see the decision rule, +§7): key-space-shifting returns (`sub(prefix)`, `mkdir` returning a sub-store, the +trailing-slash `__getitem__` overload) — this is #10's territory; cross-keyspace +values (`S3ClientDol.__setitem__(bucket, mapping-in-another-keyspace)`); keys embedded +in returned records (`info()` → ObjectInfo carrying its key) and in exception payloads +(`S3PartialFailure.succeeded/.failures` — laziness compounds this: the raise site has +no boundary frame); keys nested at data paths (`object_list_pages()` → +`page['Contents'][i]['Key']`); untyped operation batches (`cosmodol.batch`); +view-dependent no-arg operations (`sshdol.sync_to` — no key argument to transform, +yet it violates a filtered outer view's contract); replace-the-write strategies +(multipart uploads); filter/prefix pushdown (#24's fast-op hooks — the spec is a +natural registry for them later, but that is co-design, not coverage). Also not in +the prototype: `postget`/`preset` (key-aware value transforms), `__missing__` +routing (which has *two* paths with different key domains — leaf-native dict +`__missing__` sees inner keys, Store-level routing sees outer keys **[verified]**), +class-wrapping, and inherited-Protocol TypeVar substitution. + +## 5. The population map (refined from the draft's dichotomy) + +Method bodies split three ways, not two **[panel-corrected]**: + +- **Leaf-domain bodies** (adapters: `url_for`, `replace`, `delete_many`, and mixed + bodies like `sshdol.mkdir` that combine raw-backend calls with `self[...]` — these + stay coherently leaf-domain): **Option G's population.** Boundary transformation is + correct for them under the §2.5 laws. +- **Outer-domain bodies** (user extension methods over the wrapped view — every + `wrapped_self` site in the census: `xdol`, `unbox`, `lexis`; all keyless): + `wrapped_self` today, is-a later. G deliberately does not touch them. +- **View-blind bodies** (`sync_to`: no keys in the signature, semantics still depend + on the outer view): **no arg/return mechanism can serve these.** They are the + strongest surviving argument for #86 §4's rung 1 — have fewer such methods — and + for sibling stores. + +## 6. The rebind family, re-examined honestly + +The draft claimed the flat carrier voids two of the three rebind-rejection premises. +The panel refuted both sub-claims **[probes p2, p3]**, and the corrected statement is: +under a flat proxy, rebinding `self` to the proxy makes internal leaf→leaf self-calls +**re-cross the boundary → double-encode** (replacing the innermost-binding defect with +a violation of G's own headline invariant); write-through `__setattr__` *relocates* +state divergence (writes fail on dict/slotted leaves; two proxies alias state through +one leaf; wrapper-domain writes land where leaf methods read them). Reason 2 +(`super()`/descriptor TypeErrors in bodies) was never in doubt. **The rejection +stands, on stronger grounds than the draft gave it.** Do not re-propose. + +## 7. Placement in the option space — answering B's verdict + +G's spec core **is Option B, replaced rather than built on** — exactly what ADR-0011 +D5 predicted a real mechanism would be. The #86 verdict on B was "the guard is worth +more than the registry", and G's answer is direct: **in G the guard is the +mechanism** — three refusal layers at wrap/compile time (§2.3) instead of a test the +author must remember to write — and the registry is *derived* (from annotations) +rather than *maintained*. What B's verdict got right and G keeps: the wrap-time guard +serves the wrong desk alone; the adapter-side reflective conformance test stays. + +The **sibling-store decision rule** (new, resolves the two-blessed-patterns tension): +*spec expressibility is the line.* A capability whose shape the spec can express +(scalar KT, Iterable[KT], tuple[KT, VT], …) MAY be a method iff spec'd; a shape the +spec refuses (§4's inventory) MUST be a sibling store, handle, or free function. This +preserves #86 §4's rung 1 incentive — every keyed method is a declared, reviewed +liability — while giving the ones that earn their place a correctness mechanism. + +During P0–P1 nothing protects users of the *legacy* wrap path: a spec is honored only +by the new engine, so `cosmodol.replace` under today's `KeyCodecs` stays exactly as +broken as the census found it. A cheap P1 mitigation worth considering: teach +`wrap_kvs` to *warn* when key-wrapping a store whose class declares an interface spec. + +## 8. Relation to is-a (option F) — and open question 0 + +New evidence for the F conversation: (a) **is-a does not fix backend-direct keyed +bodies** (§TL;DR) — F's "both #83 and #18 disappear" holds only for bodies that route +through `self[...]`/hooks; (b) **hook-name collision**: a leaf that owns `_id_of_key` +(s3dol's `S3BucketReader`, base.py:201-shape) gets its prefix arithmetic *shadowed* by +a naive is-a hook → `KeyError` **[probe p4]** — a constraint on F's Phase-3 mixin +design that neither prior doc records. So F and G are complements: F serves +outer-domain bodies (#18), G serves leaf-domain bodies (#83), and both leave §5's +third population to design pressure (fewer keyed methods). + +This raises **open question 0**, which the staged plans currently answer differently: +*what does `wrap_kvs` compile to at the endgame — an is-a subclass (#18 doc, Phase 3) +or a flat proxy (this doc, P2)?* They contradict; the maintainer should own the call. +A plausible synthesis: `wrap_kvs`'s *codec semantics* compile to the flat engine, while +*class-decorator* usage (the #18 population's home) gets is-a — but that is a proposal, +not a decision. + +## 9. Migration (rescoped after the panel) + +- **P0 (this branch)**: the private prototype + tests. Additive; no exports; no + behavior change anywhere. +- **P1**: census-family adapters (`cosmodol`, `pydrivedol`, `sshdol`…) adopt + `interface_wrap` for spec-expressible capability surfaces; sibling stores per the + §7 rule. Add the adapter-side conformance helper. Consider the legacy-path warning. +- **P2 — rescoped**: a wrap_kvs facade over the engine is possible **only for + flat-equivalent configurations** — stacks where no inner layer *observes its + domain*. A layer-i `wants_self` transform receives the layer-i wrapper today + **[verified]**; a layer-i `postget` sees intermediate keys; both are observable API + a fused pipeline never materializes. The facade must detect these and fall back to + nesting, loudly. Route 2 (`mk_relative_path_store` → `Files`) is **not** wrap_kvs + and migrates separately or not at all — say so wherever "two routes unified" is + claimed. A P2 compatibility appendix must answer, before code: does the product + subclass `Store`; what does `.store` return (≥51 lines in dol core read it); is the + #6 signature graft kept; does the engine register the `wrapped_self` backref + (today's blessed #18 fix silently degrades to identity on engine-built wraps + otherwise **[verified]**). +- **P3**: answer open question 0 with #5 and #10 at the table; extend the layer + vocabulary (a filter-layer kind for `filt_iter`; `cached_keys` decided separately) + or permanently scope the flat model to codec stacks. + +## 10. Mixed stacks (old/new) — the standing rule + +Nothing prevents `KeyCodecs.prefixed('x')(g_proxy)` (legacy Store over a G proxy) or +`interface_wrap(legacy_store, …)` today, and in both directions the flat-model +guarantees silently degrade (`inner_most_key` under-resolves; `__wrapped__` is a +wrapper, not the backend) **[probe p1]**. The prototype's policy: wrapping a legacy +`Store` **warns** and treats it as an opaque leaf; the guarantees are explicitly +scoped to the layers above it. The eventual policy (absorb known layer types? refuse?) +belongs with P2's compatibility appendix. + +## 11. Open questions for the maintainer + +0. **Who owns the wrap_kvs endgame** — flat engine, is-a, or the split synthesis (§8)? +1. Naming: `interface_wrap`? role-lane spelling (`codecs=dict(KT=…)`)? Should the + built-in Mapping spec ship now, and with which of the mixin methods (each needs a + vocabulary decision: `KeysView[KT]`, `update(**kw)`, `get`'s default)? +2. Loudness defaults: keep `undeclared='raise'` knowing real leaves are noisy and + passthrough becomes habit — or default to `'exclude'` (loud at use, quiet at wrap)? +3. **Typed codecs**: adopt encoded/decoded type tags on `Codec` (the two in-tree + TODOs) so stack composition can check adjacency and the §2.5 laws become + enforceable rather than documentary? +4. The eq/hash/len policy family (§2.4) — decide explicitly, including whether to fix + today's Store eq/hash incoherence in the same breath. +5. `__class__` transparency — co-design with #5, or drop permanently? + +## 12. Verification log + +| # | Claim | How verified | +|---|---|---| +| 1 | wrapt proxy has the #18 hole; pickle refuses; pure-Python faster than C ext on 3.12 | wrapt 2.3.0 installed, probes run | +| 2 | today: pickle fails for decorator-form/Files/stacked wraps, one root cause (name shadowing); instance + single class-wrap work | 7-case matrix, exact tracebacks | +| 3 | today: ~420–510 ns/layer getitem; 34–42× dict iteration at 1 layer; 2 delegation objects per instance wrap | timeit + object-graph probes | +| 4 | prototype: no-double-apply on internal self-calls | counting encoder, 1 invocation | +| 5 | prototype: s3dol prefix-owning leaf composes; 2-layer stack byte-identical | faithful simulation + counterfactuals | +| 6 | is-a does not fix backend-direct #83 bodies; leaf `_id_of_key` shadowed by naive is-a hook | panel probe p4 | +| 7 | rebinding on a flat proxy double-encodes internal self-calls; write-through relocates state divergence | panel probes p2, p3 | +| 8 | `__wrapped__` ≠ #16/#10 boundary (raw-scalar surface needed) | real `path_set_writeback` run both ways | +| 9 | boundary invariant fails without codec laws (out-of-band keys, one-sided codecs, arg mutation) | panel probe a, three shapes | +| 10 | flat 3.6×/3.4× faster than nested at depth 6 (getitem/iter) | benchmark, this branch | +| 11 | layer-1 `wants_self` transform receives the layer-1 wrapper (fusion not semantics-preserving) | panel probe A1 | +| 12 | engine wraps don't populate the `wrapped_self` registry | panel probe C2 | +| 13 | prototype hardening: `*keys: KT` elementwise, unannotated-param refusal, None-default rule, no raw-dunder leak, in-flight iterator safety, nested iterators, 3.10 parity | `dol/tests/test_interface_wrap.py` + 3.10 doctest run | +| 14 | independent code review round: 3.10 builtin-slot signature failure (positional plan fallback), dict-form spec pickle/copy crash, explicit-dunder escape (`s.__contains__` answering in the wrong key domain), sequence-protocol iteration leak, fast-path keyword calls, `Iterable`-arg one-shot downgrade, policy-typo silence, id()-keyed class-cache collisions — all fixed with regression tests (39 total) | independent reviewer probes + `dol/tests/test_interface_wrap.py`; accepted follow-ups: fast path does not enforce the outer signature's arity (generic path does); dict-form proxy classes are not cached | + +## References + +Issues/discussions: #86 (option space) · #83 (census + carry-forward list) · #18 +(is-a plan, rebind rejection) · #10, #16 (boundary engine) · #5, #6 · #24 (fast-op +hooks) · s3dol#14, s3dol ADR-0011, ADR-0006. In-tree: `dol/_interface_wrap.py`, +`dol/tests/test_interface_wrap.py`, sibling docs in `misc/docs/`.