Ideas for extensions of dol -- namely when we will re-architecture it

[thorwhalen]

Include hooks

Example: filt_iter

What I really wish for is a way to control the way that filter(filt, iterable) will work via the iterable.
That way, when there's a different, perhaps more optimized way, to apply filt to the iterable, we can use that.
For example, say you want express your filt to be a mongo query that will be sent to the remote DB to filter the items closer to the data. That's much less wasteful than asking for the items to be sent one by one over the wire, only to filter out most of them!

But there's apparently no way to control builtin filter from an iterable (that same way, for example, that you can control what iter or len does, via the __iter__ and __len__ methods of the iterable.

That said, we can control what we wrote: Namely filt_iter. We can tell it to first see if the wrapped mapping has a _filter_ method, and if so, pass the job on to it.

[thorwhalen]

Update from the #86/Option G design study (doc, PR #88): the "include hooks" idea here got a precise shape and a precise limit.

The interface-spec mechanism prototyped there (per-method declarations of where keys/values appear, compiled at wrap time) is a natural registry for fast-op hooks: a spec is exactly the place where "this backend has a native _filter_ / batch / query fast path" could be declared once and discovered by tooling like filt_iter.

The limit, established while attacking the design: pushdown is wrapper knowledge flowing down, which is not expressible as argument/return mapping — and filt_iter itself is a key-set change, not a codec, so it can't be a layer in the flat codec stack either. The doc's P3 proposes extending the layer vocabulary with a first-class filter layer kind; that's the point where filt_iter-consults-_filter_ would become implementable without the wrap seam swallowing it (the s3dol measurement of the seam cost: 22 LIST requests where 1 would do). So: same wish as this discussion, now with a concrete plug point and a named blocker.