Ignore %E and %O instead of failing to format - #1816
Conversation
POSIX gives `%E` and `%O` as modifiers selecting an *alternative
representation* (era-based years, alternative digits), and says an
implementation that has no alternative representation must behave as if
the modifier were absent. chrono has none, but the parser has no arm for
either letter, so they fall through to the error case and formatting
returns `Err`.
That is not only a problem for a caller who types `%Ey`. `%x`, `%X`,
`%c` and `%r` are expanded from the locale's own `d_fmt`, `t_fmt`,
`d_t_fmt` and `t_fmt_ampm` while formatting, and nine of the locales in
`pure-rust-locales` carry a modifier in those strings:
az_IR d_fmt = %Oy/%Om/%Od
fa_IR d_fmt = %Oy/%Om/%Od
lo_LA d_fmt = %d/%m/%Ey
lzh_TW d_fmt = %OC%Oy年%B%Od日
mnw_MM d_fmt = %OC%Oy %b %Od %A
my_MM d_fmt = %OC%Oy %b %Od %A
or_IN d_fmt = %Od-%Om-%Oy
shn_MM d_fmt = %OC%Oy %b %Od %A
th_TH d_fmt = %d/%m/%Ey
So `dt.format_localized("%x", Locale::th_TH)` panics out of `to_string`
with "a Display implementation returned an error unexpectedly", for a
format string the caller never wrote. `fa_IR` is the worst of them: its
`t_fmt_ampm` is empty, so `%r` falls back to `t_fmt`, which is
`%OH:%OM:%OS`, and fails too.
Skipping the modifier and formatting the base specifier loses only the
era representation, which chrono cannot produce anyway, and keeps the
locale's own field order.
Reported downstream as nushell/nushell#15266.
…carries `%O` (#18924) Follow-up to #18918 (which fixed #15266). Two of the affected locales still fail on `main`, and the set of affected locales turned out to be larger than that PR said. Both found while working up the upstream chrono fix @fdncred asked for. # Description ## The bug #18918 left behind `az_IR` and `fa_IR` have an **empty** `t_fmt_ampm`, so #18918 deliberately leaves `%r` alone for chrono to resolve — that carve-out is what keeps `de_DE`/`fr_FR`/`nl_NL` from rendering blank. But chrono's fallback for an empty am/pm form is the locale's `t_fmt`, and for these two that is `%OH:%OM:%OS`. The modifier we just removed comes straight back in: ``` az_IR T_FMT_AMPM="" T_FMT="%OH:%OM:%OS" chrono-today=Err merged=Err proposed=Ok("20:00:12") fa_IR T_FMT_AMPM="" T_FMT="%OH:%OM:%OS" chrono-today=Err merged=Err proposed=Ok("20:00:12") de_DE T_FMT_AMPM="" T_FMT="%T" chrono-today=Ok("20:00:12") merged=Ok("20:00:12") proposed=Ok("20:00:12") fr_FR T_FMT_AMPM="" T_FMT="%T" chrono-today=Ok("20:00:12") merged=Ok("20:00:12") proposed=Ok("20:00:12") nl_NL T_FMT_AMPM="" T_FMT="%T" chrono-today=Ok("20:00:12") merged=Ok("20:00:12") proposed=Ok("20:00:12") ``` So `LC_TIME=fa_IR.UTF-8` still gets `nu::shell::type_mismatch, invalid format` from `date now | format date %r`. The fix is to make chrono's own fallback explicit: when `t_fmt_ampm` is empty, substitute `t_fmt` ourselves, so it goes through the stripping path like everything else. `de_DE`, `fr_FR` and `nl_NL` are byte-identical, since their `t_fmt` is `%T`. ## The affected locale list was wrong #18918 named `th_TH` and `lo_LA` and admitted the set was not enumerated exhaustively. It is nine, from scanning every `LC_TIME` block in `pure-rust-locales` 0.8.2: ``` az_IR: D_FMT=%Oy/%Om/%Od | D_T_FMT=%A %Oe %B %Oy، %OH:%OM:%OS | T_FMT=%OH:%OM:%OS fa_IR: D_FMT=%Oy/%Om/%Od | D_T_FMT=%A %Oe %B %Oy، %OH:%OM:%OS | T_FMT=%OH:%OM:%OS lo_LA: D_FMT=%d/%m/%Ey | D_T_FMT=%a %e %b %Ey, %H:%M:%S lzh_TW: D_FMT=%OC%Oy年%B%Od日 | D_T_FMT=%OC%Oy年%B%Od日 (%A) %OH時%OM分%OS秒 | T_FMT=%OH時%OM分%OS秒 | T_FMT_AMPM=%p %OI時%OM分%OS秒 mnw_MM: D_FMT=%OC%Oy %b %Od %A | D_T_FMT=%OC%Oy %b %Od %A %OI:%OM:%OS %Op %Z | T_FMT=%OI:%OM:%OS %p my_MM: D_FMT=%OC%Oy %b %Od %A | D_T_FMT=%OC%Oy %b %Od %A %OI:%OM:%OS %Op %Z | T_FMT=%OI:%OM:%OS %p or_IN: D_FMT=%Od-%Om-%Oy | D_T_FMT=%Oe %B %Oy %OI:%OM:%OS %p %Z | T_FMT=%OI:%OM:%OS %p shn_MM: D_FMT=%OC%Oy %b %Od %A | D_T_FMT=%OC%Oy %b %Od %A %OI:%OM:%OS %Op %Z | T_FMT=%OH:%OM:%OS %p th_TH: D_FMT=%d/%m/%Ey | D_T_FMT=%a %e %b %Ey, %H:%M:%S ``` The good news is that the existing approach already covers the seven that were never named — chrono has `%C`, `%e`, `%I`, `%p` and the rest, so stripping is enough. `lzh_TW` `%c` renders `2021年十月22日 (週五) 20時00分12秒`, `my_MM` `%X` renders `08:00:12 ညနေ`. Only the two `%r` cases above needed code. A test now pins all nine so the next locale table bump cannot quietly break one. # User-Facing Changes `format date` with `%r` under `LC_TIME=az_IR` or `fa_IR` renders instead of erroring. Nothing else changes: every other locale, including the empty-am/pm ones, produces the same bytes as before. # Tests + Formatting - `resolves_the_am_pm_fallback_when_it_also_carries_a_modifier` — the new case, `az_IR`/`fa_IR`. - `falls_back_to_the_plain_time_format_when_the_locale_has_no_am_pm` — the old `de_DE`/`fr_FR` test, updated for the new resolution and still asserting the rendered output is unchanged. - `renders_every_locale_that_carries_an_alternative_modifier` — `%x`, `%X`, `%c`, `%r` under all nine. Mutation-checked rather than trusted green. Dropping the fallback turns three tests red with the reported error: ``` test strings::format::date::test::resolves_the_am_pm_fallback_when_it_also_carries_a_modifier ... FAILED test strings::format::date::test::falls_back_to_the_plain_time_format_when_the_locale_has_no_am_pm ... FAILED test strings::format::date::test::renders_every_locale_that_carries_an_alternative_modifier ... FAILED az_IR %r did not render: Error(TypeMismatch { err_message: "invalid format", span: Span(TEST) }) ``` Green: ``` $ cargo test -p nu-command --lib test result: ok. 814 passed; 0 failed; 2 ignored; 0 filtered out; across 4 groups $ cargo test -p nu-command --test tests test result: ok. 2930 passed; 0 failed; 35 ignored; 0 filtered out; across 29 groups ``` `cargo fmt --all -- --check` and `cargo clippy -p nu-command --all-targets` are clean. Linux, Rust stable, `nu-command` only. I did not run the full workspace suite or the Windows/macOS paths. # After Submitting The upstream fix @fdncred asked for is chronotope/chrono#1816 — one line in chrono's `parse_next_item` that skips `%E`/`%O` per POSIX, with the same nine locales tested. If that lands and nushell picks up the release, this whole `resolve_locale_specifiers` layer and the `pure-rust-locales` direct dependency can be deleted. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: aron-intframe <intedu2024@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
nushell should migrate to a different crate: |
|
Understood on the wind-down, and that is your call to make. I am not asking you to reconsider #1768. One correction to the framing, though: this is not a nushell problem, and it is not an POSIX says an implementation that has no alternative representation must format as if the modifier were absent. With
I had read #1768 as winding down feature work while correctness fixes still land, since If that reading is wrong and the door is shut on everything now, say so plainly and I will stop here. If it is right, this is a one-line panic fix in the same category as those three. The branch already sits directly on top of 6adaa52, so no rebase is needed, and it is green: Both new tests are mutation-checked rather than trusted green: deleting the one changed line turns them red with the original panic. One limitation I should state rather than let you find it, because it is a genuine argument against merging: dropping the modifier loses the era value, so |
POSIX gives
%Eand%Oas modifiers that select an alternative representation (era-based years, alternative digits), and says an implementation that has no alternative representation must behave as if the modifier were absent. chrono has none, butparse_next_itemhas no arm for either letter, so both fall through to the error case and formatting returnsErr.That is not only a problem for a caller who types
%Ey.%x,%X,%cand%rare expanded from the locale's ownd_fmt,t_fmt,d_t_fmtandt_fmt_ampmwhile formatting, and nine of the locales inpure-rust-localescarry a modifier in those strings. So a caller who never wrote a modifier still gets an error:On
maintoday, rendering2021-10-22 20:00:12+00:00:%x%X%c%raz_IRfa_IRlo_LAlzh_TWmnw_MMmy_MMor_INshn_MMth_THThe strings behind that:
fa_IRandaz_IRare the awkward pair:t_fmt_ampmis empty, so%ralready falls back tot_fmt, and that carries%Otoo.The change
One line in
parse_next_item, after the padding/alternate handling:The modifier is skipped and the base specifier is formatted. Only the era representation is lost, which chrono cannot produce anyway; the locale's own field order is kept.
%E/%Oat the end of a string is still an error, and%OQis still an error, becausenext!()feeds an unknown specifier through the same path.After the change every cell in the table above is
Ok, e.g.th_TH%xrenders09/02/24andfa_IR%rrenders18:54:32.Tests
Two added:
test_strftime_alternative_modifiers—%Ey==%y,%EY==%Y,%Od==%d,%OH:%OM:%OSrenders,%%Eystays the literal%Ey, and%E/%OQstill fail to parse.test_strftime_localized_alternative_modifiers—%x,%X,%c,%rparse under all nine locales, with theth_TH,lo_LAandfa_IRoutputs pinned.Both were mutation-checked rather than trusted green: removing the one-line change turns both red with the original panic.
cargo fmt --all -- --checkis clean.Context and limits
Found from the downstream side: nushell/nushell#15266, where a
LC_TIME=th_TH.UTF-8user could not start the shell because its default config formats%x %X. nushell has since worked around it locally (nushell/nushell#18918) by expanding the locale strings and stripping the modifiers before handing them to chrono, which is the same rule applied one layer out. This is the fix in the right place.Verified on Linux, Rust stable. I did not run the MSRV, wasm, or no-std CI jobs.
🤖 Generated with Claude Code