Skip to content

Ignore %E and %O instead of failing to format - #1816

Closed
aron-intframe wants to merge 1 commit into
chronotope:mainfrom
aron-intframe:fix/strftime-alternative-modifiers
Closed

Ignore %E and %O instead of failing to format#1816
aron-intframe wants to merge 1 commit into
chronotope:mainfrom
aron-intframe:fix/strftime-alternative-modifiers

Conversation

@aron-intframe

Copy link
Copy Markdown

POSIX gives %E and %O as modifiers that select an alternative representation (era-based years, alternative digits), and says an implementation that has no alternative representation must behave as if the modifier were absent. chrono has none, but parse_next_item has no arm for either letter, so both fall through to the error case and formatting returns Err.

That is not only a problem for a caller who types %Ey. %x, %X, %c and %r are expanded from the locale's own d_fmt, t_fmt, d_t_fmt and t_fmt_ampm while formatting, and nine of the locales in pure-rust-locales carry a modifier in those strings. So a caller who never wrote a modifier still gets an error:

thread 'repro' panicked at library/alloc/src/string.rs:2929:
a Display implementation returned an error unexpectedly: Error

On main today, rendering 2021-10-22 20:00:12+00:00:

locale %x %X %c %r
az_IR Err Err Err Err
fa_IR Err Err Err Err
lo_LA Err Ok Err Ok
lzh_TW Err Err Err Err
mnw_MM Err Err Err Err
my_MM Err Err Err Err
or_IN Err Err Err Err
shn_MM Err Err Err Err
th_TH Err Ok Err Ok

The strings behind that:

az_IR   d_fmt = %Oy/%Om/%Od       t_fmt = %OH:%OM:%OS      t_fmt_ampm = ""
fa_IR   d_fmt = %Oy/%Om/%Od       t_fmt = %OH:%OM:%OS      t_fmt_ampm = ""
lo_LA   d_fmt = %d/%m/%Ey         d_t_fmt = %a %e %b %Ey, %H:%M:%S
lzh_TW  d_fmt = %OC%Oy年%B%Od日    t_fmt_ampm = %p %OI時%OM分%OS秒
mnw_MM  d_fmt = %OC%Oy %b %Od %A  t_fmt = %OI:%OM:%OS %p
my_MM   d_fmt = %OC%Oy %b %Od %A  t_fmt = %OI:%OM:%OS %p
or_IN   d_fmt = %Od-%Om-%Oy       t_fmt = %OI:%OM:%OS %p
shn_MM  d_fmt = %OC%Oy %b %Od %A  t_fmt = %OH:%OM:%OS %p
th_TH   d_fmt = %d/%m/%Ey         d_t_fmt = %a %e %b %Ey, %H:%M:%S

fa_IR and az_IR are the awkward pair: t_fmt_ampm is empty, so %r already falls back to t_fmt, and that carries %O too.

The change

One line in parse_next_item, after the padding/alternate handling:

let spec = if spec == 'E' || spec == 'O' { next!() } else { spec };

The modifier is skipped and the base specifier is formatted. Only the era representation is lost, which chrono cannot produce anyway; the locale's own field order is kept. %E/%O at the end of a string is still an error, and %OQ is still an error, because next!() feeds an unknown specifier through the same path.

After the change every cell in the table above is Ok, e.g. th_TH %x renders 09/02/24 and fa_IR %r renders 18:54:32.

Tests

Two added:

  • test_strftime_alternative_modifiers%Ey == %y, %EY == %Y, %Od == %d, %OH:%OM:%OS renders, %%Ey stays the literal %Ey, and %E / %OQ still fail to parse.
  • test_strftime_localized_alternative_modifiers%x, %X, %c, %r parse under all nine locales, with the th_TH, lo_LA and fa_IR outputs pinned.

Both were mutation-checked rather than trusted green: removing the one-line change turns both red with the original panic.

$ cargo test --features unstable-locales
test result: ok. 301 passed; 0 failed
test result: ok. 2 passed; 0 failed
test result: ok. 1 passed; 0 failed
test result: ok. 281 passed; 0 failed

$ cargo test          # default features
test result: ok. 293 passed; 0 failed

cargo fmt --all -- --check is clean.

Context and limits

Found from the downstream side: nushell/nushell#15266, where a LC_TIME=th_TH.UTF-8 user could not start the shell because its default config formats %x %X. nushell has since worked around it locally (nushell/nushell#18918) by expanding the locale strings and stripping the modifiers before handing them to chrono, which is the same rule applied one layer out. This is the fix in the right place.

Verified on Linux, Rust stable. I did not run the MSRV, wasm, or no-std CI jobs.

🤖 Generated with Claude Code

POSIX gives `%E` and `%O` as modifiers selecting an *alternative
representation* (era-based years, alternative digits), and says an
implementation that has no alternative representation must behave as if
the modifier were absent. chrono has none, but the parser has no arm for
either letter, so they fall through to the error case and formatting
returns `Err`.

That is not only a problem for a caller who types `%Ey`. `%x`, `%X`,
`%c` and `%r` are expanded from the locale's own `d_fmt`, `t_fmt`,
`d_t_fmt` and `t_fmt_ampm` while formatting, and nine of the locales in
`pure-rust-locales` carry a modifier in those strings:

    az_IR   d_fmt = %Oy/%Om/%Od
    fa_IR   d_fmt = %Oy/%Om/%Od
    lo_LA   d_fmt = %d/%m/%Ey
    lzh_TW  d_fmt = %OC%Oy年%B%Od日
    mnw_MM  d_fmt = %OC%Oy %b %Od %A
    my_MM   d_fmt = %OC%Oy %b %Od %A
    or_IN   d_fmt = %Od-%Om-%Oy
    shn_MM  d_fmt = %OC%Oy %b %Od %A
    th_TH   d_fmt = %d/%m/%Ey

So `dt.format_localized("%x", Locale::th_TH)` panics out of `to_string`
with "a Display implementation returned an error unexpectedly", for a
format string the caller never wrote. `fa_IR` is the worst of them: its
`t_fmt_ampm` is empty, so `%r` falls back to `t_fmt`, which is
`%OH:%OM:%OS`, and fails too.

Skipping the modifier and formatting the base specifier loses only the
era representation, which chrono cannot produce anyway, and keeps the
locale's own field order.

Reported downstream as nushell/nushell#15266.
fdncred pushed a commit to nushell/nushell that referenced this pull request Aug 28, 2026
…carries `%O` (#18924)

Follow-up to #18918 (which fixed #15266). Two of the affected locales
still fail on `main`, and the set of affected locales turned out to be
larger than that PR said. Both found while working up the upstream
chrono fix @fdncred asked for.

# Description

## The bug #18918 left behind

`az_IR` and `fa_IR` have an **empty** `t_fmt_ampm`, so #18918
deliberately leaves `%r` alone for chrono to resolve — that carve-out is
what keeps `de_DE`/`fr_FR`/`nl_NL` from rendering blank. But chrono's
fallback for an empty am/pm form is the locale's `t_fmt`, and for these
two that is `%OH:%OM:%OS`. The modifier we just removed comes straight
back in:

```
az_IR  T_FMT_AMPM="" T_FMT="%OH:%OM:%OS"
       chrono-today=Err merged=Err proposed=Ok("20:00:12")
fa_IR  T_FMT_AMPM="" T_FMT="%OH:%OM:%OS"
       chrono-today=Err merged=Err proposed=Ok("20:00:12")
de_DE  T_FMT_AMPM="" T_FMT="%T"
       chrono-today=Ok("20:00:12") merged=Ok("20:00:12") proposed=Ok("20:00:12")
fr_FR  T_FMT_AMPM="" T_FMT="%T"
       chrono-today=Ok("20:00:12") merged=Ok("20:00:12") proposed=Ok("20:00:12")
nl_NL  T_FMT_AMPM="" T_FMT="%T"
       chrono-today=Ok("20:00:12") merged=Ok("20:00:12") proposed=Ok("20:00:12")
```

So `LC_TIME=fa_IR.UTF-8` still gets `nu::shell::type_mismatch, invalid
format` from `date now | format date %r`.

The fix is to make chrono's own fallback explicit: when `t_fmt_ampm` is
empty, substitute `t_fmt` ourselves, so it goes through the stripping
path like everything else. `de_DE`, `fr_FR` and `nl_NL` are
byte-identical, since their `t_fmt` is `%T`.

## The affected locale list was wrong

#18918 named `th_TH` and `lo_LA` and admitted the set was not enumerated
exhaustively. It is nine, from scanning every `LC_TIME` block in
`pure-rust-locales` 0.8.2:

```
az_IR: D_FMT=%Oy/%Om/%Od | D_T_FMT=%A %Oe %B %Oy، %OH:%OM:%OS | T_FMT=%OH:%OM:%OS
fa_IR: D_FMT=%Oy/%Om/%Od | D_T_FMT=%A %Oe %B %Oy، %OH:%OM:%OS | T_FMT=%OH:%OM:%OS
lo_LA: D_FMT=%d/%m/%Ey | D_T_FMT=%a %e %b %Ey, %H:%M:%S
lzh_TW: D_FMT=%OC%Oy年%B%Od日 | D_T_FMT=%OC%Oy年%B%Od日 (%A) %OH時%OM分%OS秒 | T_FMT=%OH時%OM分%OS秒 | T_FMT_AMPM=%p %OI時%OM分%OS秒
mnw_MM: D_FMT=%OC%Oy %b %Od %A | D_T_FMT=%OC%Oy %b %Od %A %OI:%OM:%OS %Op %Z | T_FMT=%OI:%OM:%OS %p
my_MM: D_FMT=%OC%Oy %b %Od %A | D_T_FMT=%OC%Oy %b %Od %A %OI:%OM:%OS %Op %Z | T_FMT=%OI:%OM:%OS %p
or_IN: D_FMT=%Od-%Om-%Oy | D_T_FMT=%Oe %B %Oy %OI:%OM:%OS %p %Z | T_FMT=%OI:%OM:%OS %p
shn_MM: D_FMT=%OC%Oy %b %Od %A | D_T_FMT=%OC%Oy %b %Od %A %OI:%OM:%OS %Op %Z | T_FMT=%OH:%OM:%OS %p
th_TH: D_FMT=%d/%m/%Ey | D_T_FMT=%a %e %b %Ey, %H:%M:%S
```

The good news is that the existing approach already covers the seven
that were never named — chrono has `%C`, `%e`, `%I`, `%p` and the rest,
so stripping is enough. `lzh_TW` `%c` renders `2021年十月22日 (週五)
20時00分12秒`, `my_MM` `%X` renders `08:00:12 ညနေ`. Only the two `%r` cases
above needed code. A test now pins all nine so the next locale table
bump cannot quietly break one.

# User-Facing Changes

`format date` with `%r` under `LC_TIME=az_IR` or `fa_IR` renders instead
of erroring. Nothing else changes: every other locale, including the
empty-am/pm ones, produces the same bytes as before.

# Tests + Formatting

- `resolves_the_am_pm_fallback_when_it_also_carries_a_modifier` — the
new case, `az_IR`/`fa_IR`.
- `falls_back_to_the_plain_time_format_when_the_locale_has_no_am_pm` —
the old `de_DE`/`fr_FR` test, updated for the new resolution and still
asserting the rendered output is unchanged.
- `renders_every_locale_that_carries_an_alternative_modifier` — `%x`,
`%X`, `%c`, `%r` under all nine.

Mutation-checked rather than trusted green. Dropping the fallback turns
three tests red with the reported error:

```
test strings::format::date::test::resolves_the_am_pm_fallback_when_it_also_carries_a_modifier ... FAILED
test strings::format::date::test::falls_back_to_the_plain_time_format_when_the_locale_has_no_am_pm ... FAILED
test strings::format::date::test::renders_every_locale_that_carries_an_alternative_modifier ... FAILED
az_IR %r did not render: Error(TypeMismatch { err_message: "invalid format", span: Span(TEST) })
```

Green:

```
$ cargo test -p nu-command --lib
test result: ok. 814 passed; 0 failed; 2 ignored; 0 filtered out; across 4 groups

$ cargo test -p nu-command --test tests
test result: ok. 2930 passed; 0 failed; 35 ignored; 0 filtered out; across 29 groups
```

`cargo fmt --all -- --check` and `cargo clippy -p nu-command
--all-targets` are clean.

Linux, Rust stable, `nu-command` only. I did not run the full workspace
suite or the Windows/macOS paths.

# After Submitting

The upstream fix @fdncred asked for is chronotope/chrono#1816 — one line
in chrono's `parse_next_item` that skips `%E`/`%O` per POSIX, with the
same nine locales tested. If that lands and nushell picks up the
release, this whole `resolve_locale_specifiers` layer and the
`pure-rust-locales` direct dependency can be deleted.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: aron-intframe <intedu2024@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
@djc

djc commented Aug 28, 2026

Copy link
Copy Markdown
Member

nushell should migrate to a different crate:

@djc djc closed this Aug 28, 2026
@aron-intframe

Copy link
Copy Markdown
Author

Understood on the wind-down, and that is your call to make. I am not asking you to reconsider #1768.

One correction to the framing, though: this is not a nushell problem, and it is not an unstable-locales problem either. On today's main (6adaa52), with default features and no locale feature at all, format("%Ey").to_string() panics:

thread 'repro_plain_default_features' (1465372) panicked at /rustc/8bab26f4f68e0e26f0bb7960be334d5b520ea452/library/alloc/src/string.rs:2929:14:
a Display implementation returned an error unexpectedly: Error

POSIX says an implementation that has no alternative representation must format as if the modifier were absent. parse_next_item has no arm for E or O, so they fall through to the error case, Display returns Err, and every to_string() / format!() on it panics. nushell is only where I happened to notice it.

With unstable-locales, the same gap makes chrono's own locale data unformattable for callers who never typed a modifier, because %x, %X, %c and %r are expanded from d_fmt/t_fmt/d_t_fmt while formatting. Re-run against main today:

locale   %x       %X       %c       %r
az_IR    Err      Err      Err      Err
fa_IR    Err      Err      Err      Err
lo_LA    Err      Ok       Err      Ok
th_TH    Err      Ok       Err      Ok

lzh_TW, mnw_MM, my_MM, or_IN and shn_MM are Err in all four columns too.

I had read #1768 as winding down feature work while correctness fixes still land, since git log --since=2026-01-22 shows:

2026-08-02 6adaa52 Return None from from_isoywd_opt for out-of-range years
2026-07-29 7b6436f Fix reversed `NaiveDate` day and week iterators
2026-06-18 3ffcd1b fix: clamp offset minute rounding to avoid invalid +24:00

If that reading is wrong and the door is shut on everything now, say so plainly and I will stop here. If it is right, this is a one-line panic fix in the same category as those three. The branch already sits directly on top of 6adaa52, so no rebase is needed, and it is green:

test result: ok. 301 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.14s

Both new tests are mutation-checked rather than trusted green: deleting the one changed line turns them red with the original panic.

test format::strftime::tests::test_strftime_alternative_modifiers ... FAILED
test format::strftime::tests::test_strftime_localized_alternative_modifiers ... FAILED
test result: FAILED. 299 passed; 2 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.16s

One limitation I should state rather than let you find it, because it is a genuine argument against merging: dropping the modifier loses the era value, so th_TH %x renders 09/02/24 where glibc renders the Buddhist-era year. I could not measure the glibc side to confirm the exact expected string, because th_TH.UTF-8 is not generated on this machine and date silently fell back to the C locale. So the fix trades a panic for a Gregorian year under the two era locales. That is what POSIX prescribes, but it is a real loss, and if you would rather keep the hard error than print a quietly wrong year, that is a defensible reason to leave this closed. I also did not run the MSRV, wasm or no-std CI jobs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants