Audit (dol#83 census): couchdol is NOT affected — the `clear()` scope-mismatch claim is refuted

[thorwhalen]

Summary

This is a negative result filed for the record as part of the i2mint/dol#83 census of the key-scope delegation bug class (root cause: i2mint/dol#18).

An unverified survey alleged that CouchDbPersister.clear() (couchdol/__init__.py:60) and CouchDbStore.clear() (couchdol/__init__.py:179) "wipe the leaf ignoring outer scoping" — a destructive scope mismatch.

Both allegations are refuted. Neither method deletes anything: both raise NotImplementedError. There is no data loss, no scope mismatch, and — after a full sweep of the package's method surface — no instance of the dol#83 bug class in couchdol at all.

One genuinely minor, non-bug-class nit was found along the way (a degraded error message) and is noted at the end.

The bug class, briefly

dol wraps stores by delegation (has-a), not inheritance. When a store is wrapped with a key transform, the wrapper maps keys correctly for __getitem__ / __setitem__ / __delitem__ / __contains__ / __iter__ — but every other (non-dunder) method is handed the outer, unmapped key. Two delegation routes:

• Route A — instance wraps and mk_relative_path_store subclasses: Store.__getattr__ (dol/base.py:742) returns getattr(self.store, attr), i.e. the leaf-bound method.
• Route B — class wraps: delegate_to (dol/base.py:416) installs a DelegatedAttribute descriptor for every attr in dir(wrapped); DelegatedAttribute.__get__ (dol/base.py:252) also returns the leaf-bound method.

Either way, a leaf method like url_for(k) receives the outer key that the leaf's own namespace knows nothing about. There is a value-side mirror too: methods that read/write values directly on the leaf bypass obj_of_data / data_of_obj.

Why couchdol is clean

1. No key-codec wrapping anywhere. Grepping the whole repo for mk_relative_path_store, KeyCodecs, prefixless_view, filt_iter, wrap_kvs, PrefixRelativizationMixin, KeyTemplate, Pipe returns zero hits. The package ships three classes in one module and wraps nothing.

2. The one real key transform has nothing to hand off. CouchDbTupleKeyStore (couchdol/__init__.py:188) does define _id_of_key / _key_of_id (tuple ↔ dict), so Route A delegation is genuinely active in the package. But the leaf's entire method surface is:

CouchDbPersister member	kind	takes a key?	reachable via delegation?
ID_REPLACE, REV_REPLACE, SPECIAL_KEYS	class constants	no	yes, harmlessly
clear	method	no	shadowed by CouchDbStore.clear
__getitem__, __setitem__, __delitem__, __iter__, __len__	dunders	yes	key-mapped correctly by Store
__get_item_internal, __recover_internals, __replace_internals, __get_doc_filter	name-mangled private (_CouchDbPersister__*)	some	not public API; nothing calls them from outside
_key_fields	data attribute	no	read deliberately via self.store._key_fields

There is no public non-dunder method on the leaf that accepts a key. Inherited get / pop / setdefault / update / keys / values / items / head all resolve on the MRO (Mapping / MutableMapping / KvReader), so Store.__getattr__ never fires for them and they route through the outer, key-mapped dunders.

Findings

Symbol	Location	Verdict	Severity
CouchDbPersister.clear	couchdol/__init__.py:60	refuted	none (it is a protective override)
CouchDbStore.clear	couchdol/__init__.py:179	refuted	none (resolves to dol's disabled clear)
CouchDbStore.clear — degraded safety message	couchdol/__init__.py:179	confirmed-live	cosmetic

CouchDbPersister.clear — refuted

The body is a bare raise, a hand-copy of dol's own protective message. It never touches self._cdb:

def clear(self):
    raise NotImplementedError(
        "clear is disabled by default, for your own protection! "
        "Loop and delete if you really want to."
    )

CouchDbStore.clear — refuted

class CouchDbStore(Store):
    def clear(self):
        super().clear()

super() starts at Store, and Store does not define clear — it is commented out at dol/base.py:831. The MRO walk lands on KvPersister.clear = _disabled_clear_method (dol/base.py:225), which raises NotImplementedError (dol/util.py:516). Nothing is deleted.

Worth adding: even in the counterfactual where this had inherited MutableMapping.clear, that implementation loops self.popitem() through the outer __iter__ / __getitem__ / __delitem__ and would have been scope-correct. clear is one of the few MutableMapping mixin methods that is structurally safe under key-transforming delegation.

Repro — SYNTHETIC

The couchdb SDK is not installable in the audit environment, so couchdol cannot be imported. The block below mirrors the exact class shapes and inheritance of couchdol/__init__.py against real dol, with the two clear bodies copied verbatim. It is synthetic, clearly labelled as such, and reproduces the resolution path rather than the CouchDB I/O.

from dol.base import Persister, Store
from dol.util import lazyprop, _disabled_clear_method

# ---- mirror of CouchDbPersister (couchdol/__init__.py:8) --------------------
class FakeCouchDbPersister(Persister):
    def clear(self):                             # verbatim, couchdol/__init__.py:60
        raise NotImplementedError(
            "clear is disabled by default, for your own protection! "
            "Loop and delete if you really want to."
        )
    def __init__(self, key_fields=("_id",)):
        self._backend = {}                       # stands in for self._cdb
        self._key_fields = key_fields
    def __getitem__(self, k): return self._backend[tuple(sorted(k.items()))]
    def __setitem__(self, k, v): self._backend[tuple(sorted(k.items()))] = v
    def __delitem__(self, k): del self._backend[tuple(sorted(k.items()))]
    def __iter__(self):
        for kk in list(self._backend): yield dict(kk)
    def __len__(self): return len(self._backend)

# ---- mirror of CouchDbStore (couchdol/__init__.py:178) ----------------------
class FakeCouchDbStore(Store):
    def clear(self): super().clear()             # verbatim, couchdol/__init__.py:179
    def __init__(self, *a, **kw): super().__init__(FakeCouchDbPersister(*a, **kw))

# ---- mirror of CouchDbTupleKeyStore (couchdol/__init__.py:188) --------------
class FakeCouchDbTupleKeyStore(FakeCouchDbStore):
    @lazyprop
    def _key_fields(self): return self.store._key_fields
    def _id_of_key(self, k): return {f: v for f, v in zip(self._key_fields, k)}
    def _key_of_id(self, _id): return tuple(_id[x] for x in self._key_fields)

p = FakeCouchDbPersister(); p[{"_id": "a"}] = {"v": 1}
try: p.clear()
except NotImplementedError: print("persister: raised; len still", len(p))

s = FakeCouchDbStore(); s[{"_id": "a"}] = {"v": 1}
try: s.clear()
except NotImplementedError as e: print("store:     raised; leaf len still", len(s.store))

t = FakeCouchDbTupleKeyStore(key_fields=("_id", "user")); t[(1234, "bob")] = {"age": 42}
print("outer keys:", list(t), "| leaf keys:", list(t.store))
try: t.clear()
except NotImplementedError: print("tuple key store: raised; leaf len still", len(t.store))

print("Store defines clear?", "clear" in vars(Store))                    # False
print("KvPersister.clear disabled?",
      vars(Persister)["clear"] is _disabled_clear_method)                # True

Output:

persister: raised; len still 1
store:     raised; leaf len still 1
outer keys: [(1234, 'bob')] | leaf keys: [{'_id': 1234, 'user': 'bob'}]
tuple key store: raised; leaf len still 1
Store defines clear? False
KvPersister.clear disabled? True

User-visible consequence

None. No data is destroyed, no key is mis-scoped, no value bypasses a codec. clear() on either class raises NotImplementedError and leaves the CouchDB database untouched.

The one real (cosmetic) nit

_disabled_clear_method builds its message as f"Instance of {type(self)}: {self.clear.__doc__}" (dol/util.py:538). Because CouchDbStore.clear overrides the disabled method with an undocumented passthrough, self.clear.__doc__ is None, and the user sees:

NotImplementedError: Instance of <class 'couchdol.CouchDbStore'>: None

instead of dol's informative "here's how to delete everything if you really mean it" text. The override adds nothing — deleting CouchDbStore.clear entirely (couchdol/__init__.py:179-180) restores the useful message, since Store inherits the disabled method already. Optionally give CouchDbPersister.clear the same treatment, or keep its hardcoded string, which is already informative.

Forward-looking guidance (nothing to fix today)

couchdol is clean because its leaf has no public key-taking method, not because delegation is safe here — CouchDbTupleKeyStore's tuple↔dict transform means the hazard mechanism is one method away. If a CouchDB-specific accessor is ever added to CouchDbPersister (doc_url(k), rev_of(k), attachments(k), …), it will be handed the outer, unmapped key through Store.__getattr__.

Two options at that point:

1. Preferred — keep the leaf surface key-free. Return a handle object keyed at construction (the pattern azuredol uses with BlobHandle) rather than adding per-key methods to the persister. This sidesteps the delegation problem structurally instead of patching each method.

2. If a key-taking method is unavoidable, resolve the key through the whole wrapper chain:

   from dol import wrapped_self
   from dol.dig import inner_most_key   # NOT exported from dol's top level
   
   def doc_url(self, k):
       inner_k = inner_most_key(wrapped_self(self), k)
       if not isinstance(inner_k, str):   # mandatory: returns None silently
           inner_k = self._id_of_key(k)   # fall back; do NOT compose the two
       ...

   Two traps. inner_most_key walks the entire chain including the leaf's own _id_of_key — it therefore replaces self._id_of_key(k) and must never be composed with it, or the key gets double-transformed. And it returns None silently when no layer defines _id_of_key, so the type check is not optional. (Note couchdol's keys are dicts/tuples, not str, so the guard would check for the expected key type, not str.)

Verification method

Full source read of couchdol/__init__.py (223 lines, the entire package), AST enumeration of every class member, MRO resolution checked programmatically against real dol, repo-wide grep for key-codec wrappers (0 hits), and the synthetic repro above executed end to end.

---

Note on in-flight upstream fixes (added when filing)

Two dol PRs are open and change details referenced above:

• i2mint/dol#84 — inner_most_key and unravel_key
  become importable from dol directly (no more from dol.dig import ...), and
  inner_most_key now raises instead of returning None when no layer of the chain
  defines _id_of_key. If you write a local shim, the isinstance(_id, str) guard becomes
  unnecessary once that lands — but the "it replaces _id_of_key, never composes with it" trap
  still applies.
• i2mint/dol#85 — fixes dol.content_url to
  resolve the key through wrapping layers, and makes mk_relative_path_store install
  key-mapping is_valid_key/validate_key. Any finding above that is inherited from
  dol.filesys.Files is repaired by #85 with no change needed in this repo — this issue will
  be closed with verification once it merges.

Design context for why the ecosystem-wide answer is not "sprinkle wrapped_self everywhere":
i2mint/s3dol#14 and
s3dol ADR-0011.
Short version: wrapped_self is a best-effort guardrail with its own silent failure mode (it
degrades to the raw leaf when nothing holds a reference to the wrapper), so the durable fix is
to have no key-taking methods rather than to harden each one.