Default name-mapping policy puts a CLI decision in the Python signature: reconsider the default, or let the policy be set once per process

[thorwhalen]

The problem

Since v0.30 the default name-mapping policy BY_NAME_IF_KWONLY maps a plain def f(foo, bar=None) so that bar becomes a CLI positional. The only documented way to keep bar an option (--bar) is to reshape the Python signature to def f(foo, *, bar=None). The changelog states it directly:

The following function does not map to func foo [--bar] anymore: def func(foo, bar=None): .... Since this release it maps to func foo [bar] instead. Please update the function this way to keep bar an "option": def func(foo, *, bar=None).

The trouble is that this edits the function for a reason that has nothing to do with the function. The same f(foo, bar=None) is usually also called from notebooks, the REPL, tests, and other modules, and sometimes it is a third-party function I am only wrapping and cannot edit. In all of those Python contexts an argument with a default may be passed positionally or by name, at the caller's discretion. Writing *, (or, when keyword-only and defaulted-positional args mix, handling a hard ArgumentNameMappingError) forces a CLI-only constraint onto callers who never touch the CLI. A dispatcher is middleware between a Python function and an argv vector, and reconciling the two calling conventions is its job. Right now that job leaks into the Python signature. I expand on this principle in a comment below.

What I am proposing

1. Reconsider the default so an ordinary def f(foo, bar=None) maps bar back to --bar. Ideally not via plain BY_NAME_IF_HAS_DEFAULT (it has a real flaw with keyword-only args that lack a default, which I cover in a comment below) but via a refined mapping: an argument becomes an option if it is keyword-only or has a default. That honors * fully and keeps ordinary defaulted args as options.
2. If the default cannot change (I know the transition plan aims to make BY_NAME_IF_KWONLY the silent default around v0.33/v1.0): let the policy be set once, process-wide (a module-level setter, an environment variable, or a config), defaulting to today's behaviour so that nobody who does not opt in is affected. Today the only knob is name_mapping_policy= on every add_commands / set_default_command / dispatch_command(s) call, which is the per-call repetition I would like to retire.
3. Improve the error or warning so it fires only on the genuinely ambiguous case and its message names the one-line global setter from (2).

Options at a glance

Option	What it entails	Pros	Cons
Flip the default back to BY_NAME_IF_HAS_DEFAULT	restore the pre-0.30 default	zero signature edits for ordinary functions	another breaking change; contradicts the roadmap; mistreats def f(*, x) with no default
New "option if keyword-only or has-default" policy as default	one extra enum member plus a default change	honors * fully and keeps defaulted args as options	still a default change, needs a transition window
Process-wide policy setter, default unchanged	one setter plus a precedence rule (per-call > env > config > built-in)	zero churn for non-opters; removes per-call boilerplate	global state is action-at-a-distance; published packages should still set it per call
Status quo plus better docs and error message	wording change only	trivial, no behaviour change	the boilerplate and the signature pressure remain

My order of preference: the process-wide setter is the no-regret step (opt-in, back-compat-safe, removes the repetition); the "option if keyword-only or has-default" policy is the principled long-term default if a breaking change is ever on the table again; the docs and error-message fix is worth doing regardless.

Further research

I dug into this fairly deeply before and while writing the issue: the original rationale, the prior threads where it surfaced, the exact edge cases, and the full trade-off analysis. To keep the issue itself readable I have put that material in the comments below rather than inline.

• Why this is a separation-of-concerns problem (dispatch is middleware): the principle in full. A dispatcher bridges Python and argv, so reconciling their two calling conventions is its job, not the function author's.
• What has already been discussed upstream, and how this differs: the rationale in Keyword-Only Arguments as Options #191 / Keyword-only arguments as options (closes #191) #199, prior art in Python 3: arguments lost in edge cases #50 / 0.30.X is breaking console scripts by moving options to positional arguments #206 / override function signature with decorator #142 / Setting the argument name with dest is ignored #224, and the constraints (the roadmap, the "no more forced edits" objection) that the asks are shaped around.
• A caveat: BY_NAME_IF_HAS_DEFAULT does not fully honor *: why I am asking for a refined "option if keyword-only or has-default" mapping rather than plain BY_NAME_IF_HAS_DEFAULT, with a side-by-side table.
• The options in more detail: each option from the table above expanded against those constraints, with a recommended sequence.

[thorwhalen]

Why this is a separation-of-concerns problem (dispatch is middleware)

A function exposed to a CLI usually has more than one life. The same f(foo, bar=None) is also called from notebooks, the REPL, tests, and other modules. In all of those, Python's calling convention is unambiguous and ergonomic: an argument with a default may be passed positionally or by name, at the caller's discretion. The CLI convention is different: an argument is either positional xor named (--bar). These conventions genuinely conflict, because a CLI cannot offer "positional or named" for one token the way Python can.

The crucial question is who resolves the conflict. argh sits as middleware between a Python function and an argv vector, and the defining job of middleware is to reconcile the conventions of the two sides it bridges. When the Python side (an optional argument may be positional or named) collides with the CLI side (pick one), it is the middleware's responsibility to choose a deterministic, sensible mapping, not to reject the function and demand that its author rewrite it to suit one side.

BY_NAME_IF_KWONLY does the opposite: it pushes the conflict back onto the function author, forcing a Python-level signature change (*,) whose only motivation is CLI mapping. The cost lands on the wrong party, the notebook or REPL user, who must now pass bar= by name for a reason that has nothing to do with their use case, in order to satisfy a constraint that originates entirely in an interface they may never touch. That is a leak from the interface layer into the domain layer.

A natural mapping already exists. There is a total, unsurprising mapping from a normal signature to a CLI: no default implies a required positional, a default implies an optional --flag. That is essentially BY_NAME_IF_HAS_DEFAULT: what argh did before 0.30, what click does by convention, and what a Python programmer already reads a signature as. Zero signature changes required.

There is a legitimate counter-case, and I do not want to dismiss it: a function written solely as a CLI command, where the author wants the signature to be the CLI spec and uses * to make "this is an option" explicit. That is fine, but as an opt-in, not as the default. A default should optimize the common, low-surprise path. The asymmetry is the whole argument: a mapping that treats an argument as an option when it is keyword-only or has a default is strictly more accommodating as a default, because it still honors * (keyword-only args remain options) while also not penalizing ordinary signatures. BY_NAME_IF_KWONLY penalizes the ordinary-signature author, who has no way to say "treat my defaulted args as options" short of mutating the signature or threading a policy kwarg through every call site, in order to serve an author who could simply have written *, to opt in.

Guiding rule: a function's Python signature should be writable for its Python life; the CLI mapping is the dispatcher's job, and a sane default mapping already exists.

[thorwhalen]

What has already been discussed upstream, and how this differs

I went back through the tracker before filing this, both to avoid duplicating settled discussion and to make sure I am engaging your actual reasoning rather than a strawman. Here is what I found.

Where the policy and its rationale come from. The machinery was introduced in #191 (the RFC) and #199 (the implementation). Your case there is that mapping by default value leaves the author with "no control over how the argument is mapped (positional vs named)" and that "doing it through default value is not always desirable," whereas * lets the developer "define the default value and the positional/named way of calling independently." #199 actually shipped BY_NAME_IF_HAS_DEFAULT as the default ("closely follows the pre-v.0.30 behaviour") while marking BY_NAME_IF_KWONLY as recommended, and the #191 transition plan is explicit that the intent is to make the new policy the default later. So I want to be clear that I am not arguing against explicit author control. * should keep meaning exactly what it means today. I am arguing about which behaviour should be the default for a signature that carries no such marker.

The separation-of-concerns complaint is not new, so I am not claiming a fresh insight. In #50, @tanriol pushed back on a "reshape your signature" workaround precisely because "the functions are called not only by argh, but from Python code too," and you replied that this "really does make sense." In #206, @valentin-feron called the choice to map non-kwonly args to positionals "arbitrary" and proposed letting the user decide via a policy parameter. In #142 you concluded yourself that a related mapping limitation "introduces unnecessary limitations and should be changed." #224 is a recent, concrete instance of the same coupling: a user could not expose a --json flag without naming the Python parameter json. The grievance has recurred for years across several reporters. What has not been proposed, as far as I can tell, is the specific pair of asks in this issue: a refined default mapping, or a one-time process-wide setter.

Constraints I want to acknowledge up front, because they are the real reasons not to do the naive thing:

• You explicitly did not want a fix that makes users "edit the code again" (0.30.X is breaking console scripts by moving options to positional arguments #206). The process-wide setter is designed around exactly that objection: it is opt-in and defaults to current behaviour, so it imposes no migration on anyone.
• There is a committed roadmap (the Keyword-Only Arguments as Options #191 transition plan, the 0.30.4 changelog) to make BY_NAME_IF_KWONLY the silent default around v0.33/v1.0. Ask 1 cuts against that, which is precisely why ask 2 is the pragmatic path that does not.
• In 0.30.X is breaking console scripts by moving options to positional arguments #206 you weighed and rejected fully dual-mapping each positional-or-keyword arg (one positional plus one option) as infeasible with argparse. Nothing in this issue needs that capability.
• You did not want to warn on def f(foo=1, *, bar=2) because it is valid under the current policy and a warning would "pollute the output for a totally valid use case." Ask 3 is meant to respect that: narrow the signal to the genuinely ambiguous case only.

In other words, the argument is old, but the concrete, back-compat-safe asks are new, and I have tried to shape them around the constraints you have already stated on the record.

[thorwhalen]

A caveat: BY_NAME_IF_HAS_DEFAULT does not fully honor *, and the refined mapping that fixes it

It would be easy to phrase ask 1 as "just bring back BY_NAME_IF_HAS_DEFAULT," but that policy has a real flaw I do not want to gloss over, because it is the same kind of leak in reverse. Under BY_NAME_IF_HAS_DEFAULT, a keyword-only argument without a default, def f(*, x), is mapped to a positional rather than to a required option. The changelog and the NameMappingPolicy docstring both flag this as a deviation from pre-0.30 behaviour. So BY_NAME_IF_HAS_DEFAULT does not actually honor * in all cases; it honors it only for keyword-only args that happen to have defaults.

The mapping I would actually argue for as a default closes that gap. State it as a union over two independent properties:

• keyword-only (after *) implies option, always, regardless of default. This is the explicit author control from Keyword-Only Arguments as Options #191, fully preserved.
• otherwise (positional-or-keyword): has a default implies optional --flag, no default implies required positional.

Concretely, for def f(a, b=1, *, c, d=2):

Param	Kind	BY_NAME_IF_KWONLY (current default)	plain BY_NAME_IF_HAS_DEFAULT	proposed union
a	pos-or-kw, no default	positional a	positional a	positional a
b	pos-or-kw, default	positional b	option --b	option --b
c	kwonly, no default	required option --c	positional c	required option --c
d	kwonly, default	option --d	option --d	option --d

The proposed union is the only column that never surprises: * always produces an option (your control, preserved), and an ordinary defaulted argument also produces an option (the Python reader's expectation). It is strictly more accommodating than either policy shipped today, which is why I think it, and not plain BY_NAME_IF_HAS_DEFAULT, is the right candidate if the default is ever revisited.

[thorwhalen]

The options in more detail

This expands the "Options at a glance" table in the issue body, with the trade-offs spelled out against the constraints from the discussion above.

(a) Flip the default back to BY_NAME_IF_HAS_DEFAULT. Entails changing the default and nothing else. Pro: zero signature edits for ordinary functions, familiar to pre-0.30 users. Con: it is another breaking change, it contradicts the committed roadmap, and it still mistreats def f(*, x) (see the caveat comment). I list it for completeness; I do not actually recommend the plain version.

(b) Introduce an "option if keyword-only or has-default" policy and make it the default. Entails one new enum member plus a default change. Pro: honors * fully and keeps defaulted args as options, so it answers the #191 control concern instead of fighting it. Con: it is still a default change and so needs its own deprecation and transition window. This is the principled long-term answer if a breaking change is ever acceptable again.

(c) Add a process-wide policy setter, default unchanged. Entails a single setter (a module-level function, an environment variable, or a config) plus a clear precedence rule, for instance per-call name_mapping_policy= overrides an environment variable overrides a config overrides the built-in default. Pro: zero churn for anyone who does not opt in, it removes the per-call boilerplate immediately, and it directly answers your #206 objection about not forcing more code edits. Con: it introduces global state, which is action-at-a-distance, so the guidance for published packages should remain to set the policy explicitly per call; a personal CLI is where a global setter actually shines. This is my recommended near-term step, because it is opt-in and back-compat-safe.

(d) Keep the status quo and improve only the docs and the error message. Entails wording changes. Pro: trivial and risk-free. Con: the boilerplate and the signature pressure remain, and each new caller rediscovers the gotcha. Worth doing regardless of which of the above you pick.

Recommended sequence: (c) now, because it costs existing users nothing and retires the per-call repetition; (b) as the eventual default if and when a breaking change is on the table; (d) alongside either.