You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Add fbuild build --shrink[=MODE] (default auto) and --no-shrink to reduce flash size on a per-platform basis. Auto-resolution is conservative — it picks off on platforms with no measured win and safe on platforms with a documented shrinker. Default behavior on supported platforms (Arduino-ESP32 IDF 5.x newlib, STM32, NRF52, RP2040/RP2350, ESP8266, AVR, ESP32-C3) saves 6–24 KB of flash per sketch with zero behavior change.
Linking is via a GCC spec file (printf-thin.specs) that inserts a precompiled shadow archive (libprintf_thin.a) before -lc. This is naturally LTO-safe and composes with nano.specs. --wrap is the fallback used only for --shrink=printf single-knob debugging.
-1.0 to -1.7 KB (printf_min downgrade after static scan)
STM32 (f103/f411/f429/h747)
ARM newlib
safe
-8 KB
NRF52 (nrf52840/833, Adafruit cores)
ARM newlib
safe
-10 KB
RP2040 / RP2350
newlib + pico-sdk
safe
-6 to -12 KB (pico-sdk printf(compiler) + picolibc)
Teensy 2.0 (AVR)
avr-libc
safe
per AVR rule
Teensy 3+/4+ (Cortex-M)
ARM newlib
off
n/a (Print.cpp bypasses vfprintf)
ESP8266
xtensa-lx106 newlib
safe
-10 to -12 KB
Unknown / probe fails
n/a
off
fail-closed
Linking strategy
Primary: spec-file + shadow archive
fbuild emits ~/.fbuild/cache/printf-thin/<toolchain-hash>/printf-thin.specs at first build and adds --specs=<path>/printf-thin.specs to the link line. The spec rewrites *libc: to insert -lprintf_thin before -lc:
libprintf_thin.a is a complete shadow of newlib's vfprintf.o + svfprintf.o co-exports — every symbol newlib pulls those objects in for: vfprintf, _vfprintf_r, _printf_r, printf, __sbprintf, _svfprintf_r, vsnprintf, _vsnprintf_r, svfprintf, sprintf, etc. Linker resolves all to our archive during the --start-group scan; newlib's vfprintf.o/svfprintf.o never get pulled.
The archive contains vendored picolibc tinystdio (~8 files from newlib/libc/tinystdio/: vfprintf.c, vfscanf.c, dtoa_engine.c, dtoa.h, stdio_private.h, formatter-spec headers) compiled with -fno-lto, _NEED_IO_LONG_LONG=1, _NEED_IO_DOUBLE=1, io-float-exact=false, no C99/positional/wchar. Plus thin shim functions that wrap each entry point with a stack-allocated cookie FILE whose lock is pre-bound to __lock___libc_recursive_mutex (ESPHome's lazy-mutex-leak fix).
Why this is the right primary path:
Naturally LTO-safe. Native archive; sketch bitcode references vfprintf as external; LTO has no bitcode to inline.
Clean stack traces. Frames show vfprintf; addr2line resolves to vendored picolibc source.
Composes with --specs=nano.specs cumulatively.
Mirrors how Espressif (nano.specs), ARM (picolibc.specs), and Newlib itself ship printf variants.
Cost: a per-newlib-version symbol manifest (~50 lines listing each printf-related symbol newlib co-exports). Newlib's stdio layout has been stable for ~15 years; the manifest changes once every few years and is covered by a CI nm-diff test.
Fallback: -Wl,--wrap= (used only by --shrink=printf)
For single-knob debugging via --shrink=printf, use:
Scanf-thin and stdio FILE strip are bundled into the shadow archive at no additional implementation cost — same vendored sources, same --specs integration, additional ~5 KB savings when sketches use sscanf/atof/String::toInt.
ESP32 sdkconfig stack knobs under --shrink=aggressive (CONFIG_BT_CLASSIC_ENABLED=n, CONFIG_BT_BLE_50_FEATURES_SUPPORTED=n, CONFIG_LWIP_PPP_SUPPORT=n) live in a per-platform aggressive registry; trigger a sdkconfig regeneration and ESP-IDF cache rebuild. On detection of explicit CONFIG_ESP_COREDUMP_ENABLE_TO_FLASH=y in user sdkconfig, aggressive mode errors out with "explicit coredump enable detected; use --shrink=safe or remove the config".
Baseline flags already applied across all fbuild platform configs (universal, not new appliers — but surface in reporting): -fno-rtti, -fno-exceptions, -ffunction-sections, -fdata-sections, -Wl,--gc-sections, -fno-unwind-tables, -fno-asynchronous-unwind-tables (via eh_frame_policy.rs).
Reporting
--shrink=auto (default, no flag)
Single green one-liner listing the symbol families being shadowed, sourced from a per-platform registry. Omitted entirely when the platform has no auto-shrink to apply:
0.12 auto shrinking: vfprintf, vsnprintf, snprintf, printf, vprintf, fprintf, vsprintf, sprintf <- green
When auto resolves to off: no line.
When --no-shrink: Shrink: --no-shrink (user override; auto would have applied: printf-thin).
--shrink=safe|aggressive|printf (explicit)
Verbose multi-line plan with checkboxes — explicit intent earns explicit reporting:
Use owo-colors = "4" (auto-disables on non-tty + respects NO_COLOR). The auto shrinking: prefix is green().bold(); the symbol list is plain green(). Verbose plan uses default color with a single bold first line.
Per-platform shrinker registry
pubstructAutoShrinkEntry{pubcategory:&'staticstr,// "printf-thin", "esp_err_msg", etc.pubsymbols:&'static[&'staticstr],// printed in the green one-liner}
Empty registry entry → no green line, silent. Adding a new shrinker on a platform appends to its entry and the symbols automatically appear in the green one-liner.
--platformio compat mode: shrink auto-disabled (byte-identical PIO output expected). Logged at debug level.
CONFIG_LIBC_NEWLIB_NANO_FORMAT=y detected in resolved sdkconfig: shrink auto-downgrades to off. Picolibc shadow (~6 KB) is net-negative against nano vfprintf (~2 KB). Logged: "shrink: disabled (newlib-nano already in use)".
Libc probe fails: defaults to off and logs warning. Probe is a 5-line TU with #ifdef __PICOLIBC__\n#error PICOLIBC\n#endif inspected via preprocessor exit code. Never guesses.
Implementation layout
crates/fbuild-build/src/shrink/
mod.rs ShrinkMode, ShrinkPlan, auto-resolver
probe.rs fail-closed libc probe (__PICOLIBC__ test TU)
registry.rs per-platform AutoShrinkEntry table
printf_thin/ vendored picolibc tinystdio sources (-fno-lto always)
vfprintf.c from picolibc/newlib/libc/tinystdio/
vfscanf.c
dtoa_engine.c
...
COPYING.picolibc
COPYING.NEWLIB
NOTICE
newlib_stdio_manifest.rs per-newlib-version map of vfprintf.o / svfprintf.o co-exports
shadow_archive.rs builds libprintf_thin.a (complete shadow + shims with __lock___libc_recursive_mutex pre-binding)
spec_emitter.rs emits printf-thin.specs per toolchain hash
wrap_fallback.rs --wrap flag set + noinline,externally_visible stub TU (for --shrink=printf)
applier.rs translates ShrinkMode → compile/link flags, sdkconfig overrides
crates/fbuild-build/src/build_fingerprint/mod.rs — include ShrinkMode + resolved applier set in fingerprint so toggling shrink invalidates the link cache.
crates/fbuild-build/src/compile_many.rs:121 — extend seed_stage2_core_from_stage1 to include libprintf_thin.a + spec file so multi-sketch CI builds amortize printf-thin in stage 1.
crates/fbuild-packages/src/library/esp32_framework/sdk_paths.rs — IDF version detection via tools/esp32-arduino-libs/{mcu}/include/esp_common/include/esp_idf_version.h for ESP_IDF_VERSION_MAJOR; cache in Esp32Framework.
Cargo.toml — add owo-colors = "4" for the green reporting.
Acceptance
fbuild build --shrink and fbuild build --no-shrink both work; --shrink=auto|off|safe|aggressive|printf parses.
Default mode (no flag) on tests/platform/esp32s3 -e esp32s3 resolves to safe, prints the green one-liner, and drops flash by ≥ 18 KB (target; measured ceiling ~24 KB).
Default mode on tests/platform/uno -e uno resolves to safe (printf_min downgrade) when the sketch has no %f/%lld; resolves to off (the user keeps full printf) when it does.
Default mode on Arduino-ESP32 IDF 6.x resolves to off (no green line printed).
A clean build with --shrink=safe produces a firmware where xtensa-esp-elf-nm shows no _vfprintf_r / _svfprintf_r; _dtoa_r absent unless the sketch calls strtod / dtostrf.
build_info.json records the resolved mode, applied appliers, and link strategy.
--no-shrink produces a byte-for-byte identical (modulo timestamp) ELF to the pre---shrink build.
CI test diffs newlib's nm output against newlib_stdio_manifest.rs; fails on drift so the shadow archive stays complete across toolchain bumps.
CI test on a String-heavy sketch (FastLED Apa102 example uses String += float) confirms _dtoa_r is still present and float formatting is intact after --shrink=safe.
CI test confirms snprintf(NULL, 0, "...") does not clobber errno after the shadow archive is in place.
ESP32 panic-path test (trigger esp_panic) confirms the shadow is not reached from IRAM/cache-disabled context.
--platformio flag auto-disables shrink, produces byte-identical PlatformIO-compat output.
Spec-file path is naturally LTO-safe — a sketch with -flto in build_flags builds correctly with --shrink=safe and the shadow is honored.
Toggling --shrink between builds correctly invalidates the link cache; second build re-links.
Documented caveats (for BREAKING_CHANGES.md)
Banker's vs half-up rounding: picolibc dtoa-engine uses round-half-up; newlib _dtoa_r uses banker's rounding. Sketches with golden-output tests will diverge. Recommend --no-shrink for byte-exact reproduction.
Float quality: picolibc tinystdio with io-float-exact=false is slightly less accurate than newlib's _dtoa_r on edge cases (off-by-1-ulp on a tiny fraction of values). Fine for embedded logging; not safe for code that depends on exact IEEE round-trip from snprintf → strtod.
Newlib symbol-list drift: if newlib reshuffles vfprintf.o/svfprintf.o co-exports, our shadow misses a symbol and the linker pulls newlib's full object back in. Mitigation is the CI nm-diff test; the manifest is per-newlib-version.
Eight phases, each independently mergeable, ordered by dependency. Phases 1–5 ship the ESP32-S3 path end-to-end behind explicit opt-in; phase 6 turns on auto-default and rolls out platforms; phases 7–8 add aggressive mode and polish. Sub-phases inside a phase can land as separate PRs.
Goal: user-facing surface lands. Every invocation is a no-op at the link level; the shrink machinery decides and reports but doesn't change link flags yet. Safe to default-on because the resolver returns off for every platform in this phase.
Scope:
crates/fbuild-cli/src/cli/args.rs: ShrinkMode enum + --shrink[=MODE] / --no-shrink on Build, Deploy, TestEmu, Ci. default_missing_value = "auto", conflicts_with = "shrink" on --no-shrink.
crates/fbuild-build/src/shrink/probe.rs: fail-closed libc probe — compile a 5-line TU with #ifdef __PICOLIBC__\n#error PICOLIBC\n#endif, inspect preprocessor exit. Cache result per toolchain hash. On any failure → return Probe::Unknown.
crates/fbuild-build/src/shrink/registry.rs: AutoShrinkEntry { category, symbols } table. All entries empty for v1.
crates/fbuild-build/src/shrink/mod.rs: ShrinkPlan { applied: Vec<Applier>, skipped: Vec<(Applier, Reason)>, strategy: Strategy }. Auto-resolver function that returns ShrinkMode::Off for every platform (real resolution lands in phase 4+).
crates/fbuild-build/src/shrink/reporting.rs: green one-liner (empty registry → no line); verbose plan for explicit modes; --no-shrink / --platformio overrides logged.
crates/fbuild-build/src/build_info.rs: record shrink: { mode, resolved, applied: [], strategy: null } in build_info.json. Empty in phase 1.
Auto-disable conditions wired but not yet exercising appliers: --platformio → force off; CONFIG_LIBC_NEWLIB_NANO_FORMAT=y in resolved sdkconfig → force off with log.
Out of scope: any actual flag / file emission; LTO interaction; shadow archive; wrap fallback.
Tests:
Unit: CLI parse for all five modes + --no-shrink.
Unit: probe TU compiles and returns expected Probe::Newlib / Probe::Picolibc / Probe::Unknown.
Integration: fbuild build tests/platform/esp32s3 -e esp32s3 --shrink=safe prints "Shrink: safe (explicit)" + empty plan, builds identically to today.
Pass: every existing test still passes byte-for-byte. --no-shrink and --shrink=off print exactly one line each; --shrink=auto (default) prints nothing because all registries are empty.
Size: ~600 LOC, 1 PR.
Phase 2 — Shadow archive build (no link integration)
Goal: libprintf_thin.a produced as a build artifact; can be inspected with nm but isn't yet linked into any firmware.
Scope:
crates/fbuild-build/src/shrink/printf_thin/ — vendor picolibc tinystdio sources from picolibc/newlib/libc/tinystdio/: vfprintf.c, vfscanf.c, dtoa_engine.c, dtoa.h, stdio_private.h, formatter-spec headers. Include COPYING.picolibc, COPYING.NEWLIB, NOTICE.
crates/fbuild-build/src/shrink/shadow_archive.rs — shim TUs: each entry point (vfprintf, _vfprintf_r, _printf_r, printf, __sbprintf, _svfprintf_r, vsnprintf, _vsnprintf_r, svfprintf, sprintf, vsprintf, _sprintf_r, _vsprintf_r, fprintf, vprintf, fiprintf, vfiprintf) sets up a stack-allocated cookie FILE whose lock is pre-bound to __lock___libc_recursive_mutex (ESPHome's lazy-mutex-leak fix), then forwards to picolibc's __d_vfprintf. Guarded by #ifdef __PICOLIBC__ #error #endif.
build.rs invocation via cc::Build that produces libprintf_thin.a per target triple. Compiled with -fno-lto -Os -ffunction-sections -fdata-sections -D_NEED_IO_LONG_LONG=1 -D_NEED_IO_DOUBLE=1 -DPICOLIBC_FLOAT_PRINTF_SCANF=double -D_NEED_IO_C99_FORMATS=0 -D_NEED_IO_POS_ARGS=0 -D_NEED_IO_WCHAR=0 -D_NEED_IO_LONG_DOUBLE=0 -D_PICOLIBC_PRINTF=__d_vfprintf.
Out of scope: spec-file emission; integration into orchestrator; turning shrink on.
Tests:
Build: archive compiles for xtensa-esp-elf toolchain.
Symbol: xtensa-esp-elf-nm libprintf_thin.a lists every symbol in the manifest (phase 3 defines the manifest; in phase 2 we hand-check a known list).
Size: archive is ≤ 8 KB on Xtensa.
Pass: archive build is reproducible across runs (same byte hash from same sources); nm confirms expected entry points present and no undefined symbols outside libc.
Size: ~1500 LOC vendored + ~200 LOC shim, 1 PR.
Phase 3 — Newlib stdio symbol manifest + CI drift detection
Goal: catch drift between our shadow archive and newlib's actual vfprintf.o / svfprintf.o co-exports before it bites us in production.
CI test (crates/fbuild-build/tests/newlib_manifest_drift.rs) that walks each installed toolchain's libc.a, runs nm, and diffs against the manifest. Fails on any drift (missing symbol or new symbol). Provides a clear "update the manifest at X" hint.
Documentation: crates/fbuild-build/src/shrink/MANIFEST.md — how to regenerate a manifest entry when a toolchain bumps.
Out of scope: actually using the manifest to build the archive (still hand-coded in phase 2; phase 4 wires the manifest into archive generation).
Tests: CI runs against all toolchains in the test matrix; passes when manifest is in sync.
Pass: manifest is the authoritative source of truth; if a developer adds a new toolchain support, the CI test forces them to add a manifest entry.
Size: ~300 LOC manifest data + ~150 LOC test, 1 PR.
Phase 4 — ESP32-S3 end-to-end (first platform)
Goal: first measurable shrink. fbuild build tests/platform/esp32s3 -e esp32s3 (no flag) saves ≥ 18 KB by default.
Scope:
crates/fbuild-build/src/shrink/spec_emitter.rs — emit ~/.fbuild/cache/printf-thin/<toolchain-hash>/printf-thin.specs on first build per toolchain. Cache key includes toolchain hash + manifest version + picolibc source hash.
Wire --specs=<path>/printf-thin.specs -L<path> into the ESP32 link orchestrator (crates/fbuild-build/src/esp32/orchestrator/build.rs link step).
Turn on registry entry for arduino-esp32 IDF 5.x newlib Xtensa (AutoShrinkEntry listing the 8 wrap targets + scanf symbols).
Auto-resolver returns safe for arduino-esp32 IDF 5.x newlib Xtensa; still off everywhere else.
crates/fbuild-build/src/build_fingerprint/mod.rs — include ShrinkMode + resolved applier set in fingerprint. Toggling shrink invalidates the link cache (object cache stays valid).
crates/fbuild-build/src/compile_many.rs:121 — extend seed_stage2_core_from_stage1 to include libprintf_thin.a + spec file.
IDF version detection via tools/esp32-arduino-libs/{mcu}/include/esp_common/include/esp_idf_version.h for ESP_IDF_VERSION_MAJOR; cache in Esp32Framework.
build_info.json records strategy: "spec-file" + the applier list.
A/B: --no-shrink build is byte-for-byte identical (modulo timestamp) to pre-shrink baseline.
Cache: toggling --shrink between two builds correctly re-links (verified by mtime + symbol diff).
Errno: snprintf(NULL, 0, "...") does not clobber errno (small unit-test sketch).
Panic path: trigger esp_panic; shadow not reached (verified by addr2line on the panic frame).
String sketch: FastLED Apa102 example with String += float shows _dtoa_r still linked, float formatting intact.
Pass: green one-liner appears on default build; flash savings ≥ 18 KB; all symbol / errno / panic / String tests pass.
Size: ~800 LOC + acceptance tests, 1 PR.
Phase 5 — --wrap fallback for --shrink=printf
Goal: single-knob A/B debugging path lands. Implementation is a parallel mechanism; spec-file remains primary.
Scope:
crates/fbuild-build/src/shrink/wrap_fallback.rs — -Wl,--wrap= flag set (vfprintf, vprintf, printf, fprintf, vsnprintf, snprintf, vsprintf, sprintf, fiprintf, vfiprintf). Stub TU with __attribute__((noinline, externally_visible)) on each __wrap_*. Picolibc TUs already -fno-lto from phase 2.
Wire into orchestrator: --shrink=printf → use wrap path instead of spec-file path. build_info.json records strategy: "wrap".
Auto-resolver does NOT pick wrap path for any platform — wrap is debug-only.
Tests: --shrink=printf on ESP32-S3 produces equivalent ELF (same symbol delta, within 200 bytes flash) to --shrink=safe. Differences attributable to wrap overhead are documented.
Pass: wrap path is exercised in CI but not the default for any platform.
Size: ~200 LOC + test, 1 PR.
Phase 6 — Per-platform rollout
Goal: turn on auto = safe for the remaining ~7 platforms with measured wins, one sub-PR per platform.
Scope (one PR each):
Sub-PR
Platform
Mechanism
Target savings
6a
STM32
ARM picolibc shadow + spec-file (same path as ESP32-S3)
-8 KB
6b
NRF52
ARM picolibc shadow + spec-file
-10 KB
6c
RP2040/RP2350
(a) pico_set_printf_implementation(compiler), (b) ARM picolibc shadow + spec-file
-6 to -12 KB
6d
ESP8266
xtensa-lx106 picolibc shadow + spec-file
-10 KB
6e
ESP32-C3 (IDF 5.x RISC-V)
RISC-V picolibc shadow + spec-file
-6 KB
6f
AVR (special case)
Static format-string scan + swap -lprintf_min for -lprintf when no %f/%lld
-1.0 to -1.7 KB
Per sub-PR:
Add manifest entry (uses phase 3's CI test to catch drift)
Add platform-specific archive build target
Add registry entry with the symbols printed in the green one-liner
Acceptance test on a representative sketch (tests/platform/<platform>)
Update auto-resolver to return safe for this platform
AVR sub-PR is structurally different — no shadow archive, just a linker flag swap based on a static scan of all printf-family format-string literals in the sketch TUs (reuses fbuild-header-scan).
Out of scope: Teensy 3+/4+, ESP32-C6/H2, IDF 6.x — these stay at off (documented in matrix).
Tests: per platform, acceptance test runs in CI matrix.
Pass: each platform's default build drops flash by its target; green one-liner appears; no behavior regressions.
Size: ~200–400 LOC per sub-PR; 6 PRs total in this phase.
Phase 7 — Aggressive mode appliers
Goal: --shrink=aggressive enables behavior-altering wins behind explicit opt-in. Not part of auto.
Scope (each its own sub-PR):
7a — -Oz (GCC ≥ 12): gate on toolchain GCC version probe; replace -Os with -Oz in compile flags. Toolchain probe at fbuild orchestrator startup.
7b — ESP32 sdkconfig stack knobs: per-MCU registry of CONFIG_BT_CLASSIC_ENABLED=n, CONFIG_BT_BLE_50_FEATURES_SUPPORTED=n, CONFIG_LWIP_PPP_SUPPORT=n, etc. Triggers ESP-IDF cache rebuild. Detect explicit user enables and skip them (don't override).
7c — esp_err_msg_table strip: linker-script EXCLUDE_FILE or --wrap=esp_err_to_name to return "ESP_ERR(0x%x)" formatted from the code. Document the human-readable-name loss.
7d — Coredump strip: sdkconfig override CONFIG_ESP_COREDUMP_ENABLE=n. Detect explicit CONFIG_ESP_COREDUMP_ENABLE_TO_FLASH=y in user sdkconfig.defaults; on detection, error with "explicit coredump enable detected; use --shrink=safe or remove the config". Silent strip of debug feature is wrong default.
7e — Drop unused C++ exception personality: __gxx_personality_v0 already mostly dead via -fno-exceptions; aggressive mode adds -Wl,--gc-sections enforcement on the eh-frame and personality symbols.
Each sub-PR: registry entry + applier + acceptance test that measures the delta.
Tests: per applier, integration test on tests/platform/esp32s3 -e esp32s3 --shrink=aggressive measures expected delta; aggregate test confirms total aggressive delta is sum of individual deltas (no double-counting).
Pass: --shrink=aggressive saves ≥ 27 KB total on tests/platform/esp32s3; explicit-coredump test errors cleanly; loud failure when sdkconfig regeneration fails.
Size: ~150–300 LOC per sub-PR; 5 PRs in this phase.
Phase 5 (wrap fallback) and Phase 6 (per-platform) can run in parallel after phase 4. Phase 6 sub-PRs (6a–6f) can run in parallel with each other. Phase 7 sub-PRs (7a–7e) can run in parallel after phase 6 has at least the ESP32 platforms landed.
Summary
Add
fbuild build --shrink[=MODE](defaultauto) and--no-shrinkto reduce flash size on a per-platform basis. Auto-resolution is conservative — it picksoffon platforms with no measured win andsafeon platforms with a documented shrinker. Default behavior on supported platforms (Arduino-ESP32 IDF 5.x newlib, STM32, NRF52, RP2040/RP2350, ESP8266, AVR, ESP32-C3) saves 6–24 KB of flash per sketch with zero behavior change.Linking is via a GCC spec file (
printf-thin.specs) that inserts a precompiled shadow archive (libprintf_thin.a) before-lc. This is naturally LTO-safe and composes withnano.specs.--wrapis the fallback used only for--shrink=printfsingle-knob debugging.Supersedes #492.
CLI surface
On
Build,Deploy,TestEmu,Cisubcommands:--shrinkwith no value resolves toauto.--no-shrinkoverrides any auto-enable from project config or framework detection.Auto-resolver matrix
autoresolves toprintf(compiler)+ picolibc)Linking strategy
Primary: spec-file + shadow archive
fbuild emits
~/.fbuild/cache/printf-thin/<toolchain-hash>/printf-thin.specsat first build and adds--specs=<path>/printf-thin.specsto the link line. The spec rewrites*libc:to insert-lprintf_thinbefore-lc:libprintf_thin.ais a complete shadow of newlib'svfprintf.o+svfprintf.oco-exports — every symbol newlib pulls those objects in for:vfprintf,_vfprintf_r,_printf_r,printf,__sbprintf,_svfprintf_r,vsnprintf,_vsnprintf_r,svfprintf,sprintf, etc. Linker resolves all to our archive during the--start-groupscan; newlib'svfprintf.o/svfprintf.onever get pulled.The archive contains vendored picolibc tinystdio (~8 files from
newlib/libc/tinystdio/:vfprintf.c,vfscanf.c,dtoa_engine.c,dtoa.h,stdio_private.h, formatter-spec headers) compiled with-fno-lto,_NEED_IO_LONG_LONG=1,_NEED_IO_DOUBLE=1,io-float-exact=false, no C99/positional/wchar. Plus thin shim functions that wrap each entry point with a stack-allocated cookieFILEwhoselockis pre-bound to__lock___libc_recursive_mutex(ESPHome's lazy-mutex-leak fix).Why this is the right primary path:
vfprintfas external; LTO has no bitcode to inline.vfprintf;addr2lineresolves to vendored picolibc source.--specs=nano.specscumulatively.nano.specs), ARM (picolibc.specs), and Newlib itself ship printf variants.Cost: a per-newlib-version symbol manifest (~50 lines listing each printf-related symbol newlib co-exports). Newlib's stdio layout has been stable for ~15 years; the manifest changes once every few years and is covered by a CI nm-diff test.
Fallback:
-Wl,--wrap=(used only by--shrink=printf)For single-knob debugging via
--shrink=printf, use:The
fiprintf/vfiprintfentries cover__assert_func(newlib's integer-only assert path). LTO safety requires (1) vendored picolibc compiled-fno-lto, (2) stub functions marked__attribute__((noinline, externally_visible)).Linker scripts cannot fix LTO+
--wrapinteractions because they run after LTO inlining. The spec-file approach sidesteps this entirely.Bundled appliers per mode
printf--wrapfallbacksafeaggressive-Oz(GCC ≥ 12)off/--no-shrinkautoScanf-thin and stdio FILE strip are bundled into the shadow archive at no additional implementation cost — same vendored sources, same
--specsintegration, additional ~5 KB savings when sketches usesscanf/atof/String::toInt.ESP32 sdkconfig stack knobs under
--shrink=aggressive(CONFIG_BT_CLASSIC_ENABLED=n,CONFIG_BT_BLE_50_FEATURES_SUPPORTED=n,CONFIG_LWIP_PPP_SUPPORT=n) live in a per-platform aggressive registry; trigger a sdkconfig regeneration and ESP-IDF cache rebuild. On detection of explicitCONFIG_ESP_COREDUMP_ENABLE_TO_FLASH=yin user sdkconfig, aggressive mode errors out with "explicit coredump enable detected; use--shrink=safeor remove the config".Baseline flags already applied across all fbuild platform configs (universal, not new appliers — but surface in reporting):
-fno-rtti,-fno-exceptions,-ffunction-sections,-fdata-sections,-Wl,--gc-sections,-fno-unwind-tables,-fno-asynchronous-unwind-tables(viaeh_frame_policy.rs).Reporting
--shrink=auto(default, no flag)Single green one-liner listing the symbol families being shadowed, sourced from a per-platform registry. Omitted entirely when the platform has no auto-shrink to apply:
When auto resolves to
off: no line.When
--no-shrink:Shrink: --no-shrink (user override; auto would have applied: printf-thin).--shrink=safe|aggressive|printf(explicit)Verbose multi-line plan with checkboxes — explicit intent earns explicit reporting:
Use
owo-colors = "4"(auto-disables on non-tty + respectsNO_COLOR). Theauto shrinking:prefix isgreen().bold(); the symbol list is plaingreen(). Verbose plan uses default color with a single bold first line.Per-platform shrinker registry
Empty registry entry → no green line, silent. Adding a new shrinker on a platform appends to its entry and the symbols automatically appear in the green one-liner.
Configuration
platformio.iniPrecedence
lowest → highest: platform default →
platformio.ini→ CLI flag.--no-shrinkalways wins.Auto-disables
--platformiocompat mode: shrink auto-disabled (byte-identical PIO output expected). Logged at debug level.CONFIG_LIBC_NEWLIB_NANO_FORMAT=ydetected in resolved sdkconfig: shrink auto-downgrades tooff. Picolibc shadow (~6 KB) is net-negative against nano vfprintf (~2 KB). Logged: "shrink: disabled (newlib-nano already in use)".offand logs warning. Probe is a 5-line TU with#ifdef __PICOLIBC__\n#error PICOLIBC\n#endifinspected via preprocessor exit code. Never guesses.Implementation layout
Touch points:
crates/fbuild-cli/src/cli/args.rs—ShrinkModeenum,--shrink/--no-shrinkflags on Build/Deploy/TestEmu/Ci. Per-subcommand (mirrors--clean/--environment/--verboseprecedent).crates/fbuild-build/src/build_fingerprint/mod.rs— includeShrinkMode+ resolved applier set in fingerprint so toggling shrink invalidates the link cache.crates/fbuild-build/src/build_info.rs— recordshrink: { mode, resolved, applied: [...], strategy: "spec-file" | "wrap" }.crates/fbuild-build/src/compile_many.rs:121— extendseed_stage2_core_from_stage1to includelibprintf_thin.a+ spec file so multi-sketch CI builds amortize printf-thin in stage 1.crates/fbuild-packages/src/library/esp32_framework/sdk_paths.rs— IDF version detection viatools/esp32-arduino-libs/{mcu}/include/esp_common/include/esp_idf_version.hforESP_IDF_VERSION_MAJOR; cache inEsp32Framework.Cargo.toml— addowo-colors = "4"for the green reporting.Acceptance
fbuild build --shrinkandfbuild build --no-shrinkboth work;--shrink=auto|off|safe|aggressive|printfparses.tests/platform/esp32s3 -e esp32s3resolves tosafe, prints the green one-liner, and drops flash by ≥ 18 KB (target; measured ceiling ~24 KB).tests/platform/uno -e unoresolves tosafe(printf_min downgrade) when the sketch has no%f/%lld; resolves tooff(the user keeps full printf) when it does.off(no green line printed).--shrink=safeproduces a firmware wherextensa-esp-elf-nmshows no_vfprintf_r/_svfprintf_r;_dtoa_rabsent unless the sketch callsstrtod/dtostrf.build_info.jsonrecords the resolved mode, applied appliers, and link strategy.--no-shrinkproduces a byte-for-byte identical (modulo timestamp) ELF to the pre---shrinkbuild.nmoutput againstnewlib_stdio_manifest.rs; fails on drift so the shadow archive stays complete across toolchain bumps.String += float) confirms_dtoa_ris still present and float formatting is intact after--shrink=safe.snprintf(NULL, 0, "...")does not clobbererrnoafter the shadow archive is in place.esp_panic) confirms the shadow is not reached from IRAM/cache-disabled context.--platformioflag auto-disables shrink, produces byte-identical PlatformIO-compat output.-fltoinbuild_flagsbuilds correctly with--shrink=safeand the shadow is honored.--shrinkbetween builds correctly invalidates the link cache; second build re-links.Documented caveats (for
BREAKING_CHANGES.md)_dtoa_ruses banker's rounding. Sketches with golden-output tests will diverge. Recommend--no-shrinkfor byte-exact reproduction.io-float-exact=falseis slightly less accurate than newlib's_dtoa_ron edge cases (off-by-1-ulp on a tiny fraction of values). Fine for embedded logging; not safe for code that depends on exact IEEE round-trip fromsnprintf→strtod.vfprintf.o/svfprintf.oco-exports, our shadow misses a symbol and the linker pulls newlib's full object back in. Mitigation is the CI nm-diff test; the manifest is per-newlib-version.Related
fbuild bloat: emit and preservefirmware.mapautomatically. Required for accurate "before / after" shrink reporting with archive attribution.Implementation phases
Eight phases, each independently mergeable, ordered by dependency. Phases 1–5 ship the ESP32-S3 path end-to-end behind explicit opt-in; phase 6 turns on auto-default and rolls out platforms; phases 7–8 add aggressive mode and polish. Sub-phases inside a phase can land as separate PRs.
Phase 0 — Prerequisites
Goal: clear the runway. No user-facing behavior.
Scope:
owo-colors = "4"tocrates/fbuild-cli/Cargo.toml(auto-disables on non-tty +NO_COLOR).crates/fbuild-build/src/shrink/directory with amod.rsstub and aREADME.md. Wire intolib.rs.Pass:
cargo build --workspace --all-targetsclean;cargo clippy --workspace --all-targets -- -D warningsclean.Size: 1 PR, <50 LOC of fbuild code (most of the PR is #491).
Phase 1 — CLI + auto-resolver + reporting (no-op linking)
Goal: user-facing surface lands. Every invocation is a no-op at the link level; the shrink machinery decides and reports but doesn't change link flags yet. Safe to default-on because the resolver returns
offfor every platform in this phase.Scope:
crates/fbuild-cli/src/cli/args.rs:ShrinkModeenum +--shrink[=MODE]/--no-shrinkonBuild,Deploy,TestEmu,Ci.default_missing_value = "auto",conflicts_with = "shrink"on--no-shrink.crates/fbuild-build/src/shrink/probe.rs: fail-closed libc probe — compile a 5-line TU with#ifdef __PICOLIBC__\n#error PICOLIBC\n#endif, inspect preprocessor exit. Cache result per toolchain hash. On any failure → returnProbe::Unknown.crates/fbuild-build/src/shrink/registry.rs:AutoShrinkEntry { category, symbols }table. All entries empty for v1.crates/fbuild-build/src/shrink/mod.rs:ShrinkPlan { applied: Vec<Applier>, skipped: Vec<(Applier, Reason)>, strategy: Strategy }. Auto-resolver function that returnsShrinkMode::Offfor every platform (real resolution lands in phase 4+).crates/fbuild-build/src/shrink/reporting.rs: green one-liner (empty registry → no line); verbose plan for explicit modes;--no-shrink/--platformiooverrides logged.crates/fbuild-build/src/build_info.rs: recordshrink: { mode, resolved, applied: [], strategy: null }inbuild_info.json. Empty in phase 1.--platformio→ forceoff;CONFIG_LIBC_NEWLIB_NANO_FORMAT=yin resolved sdkconfig → forceoffwith log.Out of scope: any actual flag / file emission; LTO interaction; shadow archive; wrap fallback.
Tests:
--no-shrink.Probe::Newlib/Probe::Picolibc/Probe::Unknown.fbuild build tests/platform/esp32s3 -e esp32s3 --shrink=safeprints "Shrink: safe (explicit)" + empty plan, builds identically to today.Pass: every existing test still passes byte-for-byte.
--no-shrinkand--shrink=offprint exactly one line each;--shrink=auto(default) prints nothing because all registries are empty.Size: ~600 LOC, 1 PR.
Phase 2 — Shadow archive build (no link integration)
Goal:
libprintf_thin.aproduced as a build artifact; can be inspected withnmbut isn't yet linked into any firmware.Scope:
crates/fbuild-build/src/shrink/printf_thin/— vendor picolibc tinystdio sources frompicolibc/newlib/libc/tinystdio/:vfprintf.c,vfscanf.c,dtoa_engine.c,dtoa.h,stdio_private.h, formatter-spec headers. IncludeCOPYING.picolibc,COPYING.NEWLIB,NOTICE.crates/fbuild-build/src/shrink/shadow_archive.rs— shim TUs: each entry point (vfprintf,_vfprintf_r,_printf_r,printf,__sbprintf,_svfprintf_r,vsnprintf,_vsnprintf_r,svfprintf,sprintf,vsprintf,_sprintf_r,_vsprintf_r,fprintf,vprintf,fiprintf,vfiprintf) sets up a stack-allocated cookieFILEwhoselockis pre-bound to__lock___libc_recursive_mutex(ESPHome's lazy-mutex-leak fix), then forwards to picolibc's__d_vfprintf. Guarded by#ifdef __PICOLIBC__ #error #endif.build.rsinvocation viacc::Buildthat produceslibprintf_thin.aper target triple. Compiled with-fno-lto -Os -ffunction-sections -fdata-sections -D_NEED_IO_LONG_LONG=1 -D_NEED_IO_DOUBLE=1 -DPICOLIBC_FLOAT_PRINTF_SCANF=double -D_NEED_IO_C99_FORMATS=0 -D_NEED_IO_POS_ARGS=0 -D_NEED_IO_WCHAR=0 -D_NEED_IO_LONG_DOUBLE=0 -D_PICOLIBC_PRINTF=__d_vfprintf.Out of scope: spec-file emission; integration into orchestrator; turning shrink on.
Tests:
xtensa-esp-elftoolchain.xtensa-esp-elf-nm libprintf_thin.alists every symbol in the manifest (phase 3 defines the manifest; in phase 2 we hand-check a known list).Pass: archive build is reproducible across runs (same byte hash from same sources);
nmconfirms expected entry points present and no undefined symbols outside libc.Size: ~1500 LOC vendored + ~200 LOC shim, 1 PR.
Phase 3 — Newlib stdio symbol manifest + CI drift detection
Goal: catch drift between our shadow archive and newlib's actual
vfprintf.o/svfprintf.oco-exports before it bites us in production.Scope:
crates/fbuild-build/src/shrink/newlib_stdio_manifest.rs— per-toolchain map:{ toolchain_id: { vfprintf_o: [symbols], svfprintf_o: [symbols] } }. Initial entries:xtensa-esp-elf-14.2.0_*(Arduino-ESP32 IDF 5.x newlib),arm-none-eabi-13.x(STM32/NRF52/RP2040),xtensa-lx106-elf-3.x(ESP8266).crates/fbuild-build/tests/newlib_manifest_drift.rs) that walks each installed toolchain'slibc.a, runsnm, and diffs against the manifest. Fails on any drift (missing symbol or new symbol). Provides a clear "update the manifest at X" hint.crates/fbuild-build/src/shrink/MANIFEST.md— how to regenerate a manifest entry when a toolchain bumps.Out of scope: actually using the manifest to build the archive (still hand-coded in phase 2; phase 4 wires the manifest into archive generation).
Tests: CI runs against all toolchains in the test matrix; passes when manifest is in sync.
Pass: manifest is the authoritative source of truth; if a developer adds a new toolchain support, the CI test forces them to add a manifest entry.
Size: ~300 LOC manifest data + ~150 LOC test, 1 PR.
Phase 4 — ESP32-S3 end-to-end (first platform)
Goal: first measurable shrink.
fbuild build tests/platform/esp32s3 -e esp32s3(no flag) saves ≥ 18 KB by default.Scope:
crates/fbuild-build/src/shrink/spec_emitter.rs— emit~/.fbuild/cache/printf-thin/<toolchain-hash>/printf-thin.specson first build per toolchain. Cache key includes toolchain hash + manifest version + picolibc source hash.--specs=<path>/printf-thin.specs -L<path>into the ESP32 link orchestrator (crates/fbuild-build/src/esp32/orchestrator/build.rslink step).arduino-esp32IDF 5.x newlib Xtensa (AutoShrinkEntrylisting the 8 wrap targets + scanf symbols).safefor arduino-esp32 IDF 5.x newlib Xtensa; stilloffeverywhere else.crates/fbuild-build/src/build_fingerprint/mod.rs— includeShrinkMode+ resolved applier set in fingerprint. Toggling shrink invalidates the link cache (object cache stays valid).crates/fbuild-build/src/compile_many.rs:121— extendseed_stage2_core_from_stage1to includelibprintf_thin.a+ spec file.tools/esp32-arduino-libs/{mcu}/include/esp_common/include/esp_idf_version.hforESP_IDF_VERSION_MAJOR; cache inEsp32Framework.build_info.jsonrecordsstrategy: "spec-file"+ the applier list.Out of scope: STM32 / NRF52 / RP2040 / ESP8266 / AVR / C3 (phase 6); aggressive mode (phase 7); --wrap fallback (phase 5).
Tests:
fbuild build tests/platform/esp32s3 -e esp32s3drops flash by ≥ 18 KB (target ~24 KB).nmshows no_vfprintf_r/_svfprintf_r;_dtoa_rabsent (sketch doesn't callstrtod).--no-shrinkbuild is byte-for-byte identical (modulo timestamp) to pre-shrink baseline.--shrinkbetween two builds correctly re-links (verified by mtime + symbol diff).snprintf(NULL, 0, "...")does not clobbererrno(small unit-test sketch).esp_panic; shadow not reached (verified by addr2line on the panic frame).String += floatshows_dtoa_rstill linked, float formatting intact.Pass: green one-liner appears on default build; flash savings ≥ 18 KB; all symbol / errno / panic / String tests pass.
Size: ~800 LOC + acceptance tests, 1 PR.
Phase 5 —
--wrapfallback for--shrink=printfGoal: single-knob A/B debugging path lands. Implementation is a parallel mechanism; spec-file remains primary.
Scope:
crates/fbuild-build/src/shrink/wrap_fallback.rs—-Wl,--wrap=flag set (vfprintf, vprintf, printf, fprintf, vsnprintf, snprintf, vsprintf, sprintf, fiprintf, vfiprintf). Stub TU with__attribute__((noinline, externally_visible))on each__wrap_*. Picolibc TUs already-fno-ltofrom phase 2.--shrink=printf→ use wrap path instead of spec-file path.build_info.jsonrecordsstrategy: "wrap".Tests:
--shrink=printfon ESP32-S3 produces equivalent ELF (same symbol delta, within 200 bytes flash) to--shrink=safe. Differences attributable to wrap overhead are documented.Pass: wrap path is exercised in CI but not the default for any platform.
Size: ~200 LOC + test, 1 PR.
Phase 6 — Per-platform rollout
Goal: turn on
auto = safefor the remaining ~7 platforms with measured wins, one sub-PR per platform.Scope (one PR each):
pico_set_printf_implementation(compiler), (b) ARM picolibc shadow + spec-file-lprintf_minfor-lprintfwhen no%f/%lldPer sub-PR:
tests/platform/<platform>)safefor this platformAVR sub-PR is structurally different — no shadow archive, just a linker flag swap based on a static scan of all printf-family format-string literals in the sketch TUs (reuses
fbuild-header-scan).Out of scope: Teensy 3+/4+, ESP32-C6/H2, IDF 6.x — these stay at
off(documented in matrix).Tests: per platform, acceptance test runs in CI matrix.
Pass: each platform's default build drops flash by its target; green one-liner appears; no behavior regressions.
Size: ~200–400 LOC per sub-PR; 6 PRs total in this phase.
Phase 7 — Aggressive mode appliers
Goal:
--shrink=aggressiveenables behavior-altering wins behind explicit opt-in. Not part of auto.Scope (each its own sub-PR):
-Oz(GCC ≥ 12): gate on toolchain GCC version probe; replace-Oswith-Ozin compile flags. Toolchain probe at fbuild orchestrator startup.CONFIG_BT_CLASSIC_ENABLED=n,CONFIG_BT_BLE_50_FEATURES_SUPPORTED=n,CONFIG_LWIP_PPP_SUPPORT=n, etc. Triggers ESP-IDF cache rebuild. Detect explicit user enables and skip them (don't override).esp_err_msg_tablestrip: linker-scriptEXCLUDE_FILEor--wrap=esp_err_to_nameto return"ESP_ERR(0x%x)"formatted from the code. Document the human-readable-name loss.CONFIG_ESP_COREDUMP_ENABLE=n. Detect explicitCONFIG_ESP_COREDUMP_ENABLE_TO_FLASH=yin user sdkconfig.defaults; on detection, error with "explicit coredump enable detected; use--shrink=safeor remove the config". Silent strip of debug feature is wrong default.__gxx_personality_v0already mostly dead via-fno-exceptions; aggressive mode adds-Wl,--gc-sectionsenforcement on the eh-frame and personality symbols.Each sub-PR: registry entry + applier + acceptance test that measures the delta.
Tests: per applier, integration test on
tests/platform/esp32s3 -e esp32s3 --shrink=aggressivemeasures expected delta; aggregate test confirms total aggressive delta is sum of individual deltas (no double-counting).Pass:
--shrink=aggressivesaves ≥ 27 KB total ontests/platform/esp32s3; explicit-coredump test errors cleanly; loud failure when sdkconfig regeneration fails.Size: ~150–300 LOC per sub-PR; 5 PRs in this phase.
Phase 8 — Documentation, breaking-change notice, release prep
Goal: ship-ready polish.
Scope:
BREAKING_CHANGES.mdentry: default-on--shrink=autochanges Arduino-ESP32 IDF 5.x firmware behavior (smaller, picolibc float formatting). Direct users to--no-shrinkfor byte-exact reproduction.docs/SHRINK.md— user-facing doc: what the modes do, when to use each, troubleshooting (toolchain probe failures, manifest drift, wrap-vs-spec).README.mdandCLAUDE.md(project root) to mention--shrinkin the commands section.crates/fbuild-build/src/shrink/README.md— developer doc on the architecture, where to add a new shrinker, manifest update workflow.Out of scope: blog post, external announcements.
Tests: docs review only.
Pass: docs accurately reflect shipped behavior; release notes ready.
Size: docs only, 1 PR.
Dependency graph
Phase 5 (wrap fallback) and Phase 6 (per-platform) can run in parallel after phase 4. Phase 6 sub-PRs (6a–6f) can run in parallel with each other. Phase 7 sub-PRs (7a–7e) can run in parallel after phase 6 has at least the ESP32 platforms landed.
Total scope estimate