Category
Technical Debt (cleanup, refactor)
Component
Other — logging subsystem (host HostLogger + AICPU device_log + --log-level surface), spanning both a2a3 and a5.
Description
Two coupled problems with the current logging design:
1. simpler self-seeds CANN's driver INFO stream (device performance tax).
When ASCEND_GLOBAL_LOG_LEVEL is unset, simpler_init calls dlog_setlevel(-1, HostLogger::level(), 0). HostLogger::level() defaults to INFO and -1 targets every CANN module, so we open the CANN driver's own INFO logs as a side effect of wanting our own host verbosity. Those device-side writes land inside the measured device wall (~4–6% on some workloads; see #1409). It is not a CANN default — it is self-inflicted.
2. The V0..V9 INFO sub-tier scheme is a two-axis (severity, info_v) design that is more complex than what the code uses. In practice device logging is binary (V0/V9 only — see #1428), and the v-tiers force a --log-level surface (v0..v9) wired through conftest.py, _log.py::_split_threshold, log_config.py, and the docs.
Target: a single-axis ladder debug / info / timing / warn / error, default timing, replacing V0..V9. STRACE moves from LOG_INFO_V9 to the timing level; simpler_init maps the device (CANN) level so timing → WARN. Result:
- default
timing → host emits only STRACE (+ warn/error); device level = WARN → no driver-log tax on any run (not just rounds > 1);
info / debug → device INFO back on for debugging, without the DEBUG flood;
- WARN/ERROR always visible; one
--log-level still drives both host and device.
Benchmark measurement is unaffected either way — [STRACE] rides host stderr + the rtMemcpy device-phase readback, not the CANN log.
This supersedes the per-run ASCEND_GLOBAL_LOG_LEVEL env-var mitigation in #1409, and depends on the footprint reduction in #1428 landing first.
Location
- Device seeding:
src/common/platform/onboard/host/c_api_shared.cpp (simpler_init → dlog_setlevel(-1, HostLogger::level(), 0))
- STRACE emission:
src/common/log/include/common/strace.h (LOG_INFO_V9 → timing)
- Two-axis machinery:
python/simpler/_log.py (_split_threshold), simpler_setup/log_config.py (LOG_LEVEL_CHOICES), conftest.py (V0..V9 registration), src/common/log/host_log.cpp (info_v), src/common/platform/onboard/aicpu/device_log.cpp (g_log_info_v), src/common/log/include/common/unified_log.h (LOG_INFO_V0..V9)
- Call-site migration, both arches:
src/a2a3/** and src/a5/** LOG_INFO_V* sites
- Docs:
docs/logging.md, docs/testing.md, docs/troubleshooting/device-error-codes.md, .claude/rules/running-onboard.md
Proposed Fix
Phased (after #1428):
- Introduce the ladder
debug / info / timing / warn / error (single severity axis, timing between info and warn); make timing the default. Emit STRACE at timing.
- Wire the device mapping in
simpler_init: debug→DEBUG, info→INFO, timing→WARN, warn→WARN, error→ERROR — so default timing no longer seeds CANN INFO.
- Remove
V0..V9 (or keep as deprecated aliases: v9→timing, v5–v8→info, v0–v4→debug) and migrate remaining call sites + the --log-level surface + docs.
Open decisions to settle in the issue: hard-remove vs deprecated aliases for v0..v9; whether the 109 must-see device diagnostics map to info or debug.
Priority
Medium (real device-side overhead, but #1409 provides a stopgap; depends on #1428)
Context / root-cause discussion: #1409. Depends on: #1428. Related: #409 (hot-path logging overhead).
Category
Technical Debt (cleanup, refactor)
Component
Other — logging subsystem (host
HostLogger+ AICPUdevice_log+--log-levelsurface), spanning both a2a3 and a5.Description
Two coupled problems with the current logging design:
1. simpler self-seeds CANN's driver INFO stream (device performance tax).
When
ASCEND_GLOBAL_LOG_LEVELis unset,simpler_initcallsdlog_setlevel(-1, HostLogger::level(), 0).HostLogger::level()defaults to INFO and-1targets every CANN module, so we open the CANN driver's own INFO logs as a side effect of wanting our own host verbosity. Those device-side writes land inside the measured device wall (~4–6% on some workloads; see #1409). It is not a CANN default — it is self-inflicted.2. The
V0..V9INFO sub-tier scheme is a two-axis (severity, info_v) design that is more complex than what the code uses. In practice device logging is binary (V0/V9 only — see #1428), and the v-tiers force a--log-levelsurface (v0..v9) wired throughconftest.py,_log.py::_split_threshold,log_config.py, and the docs.Target: a single-axis ladder
debug / info / timing / warn / error, defaulttiming, replacingV0..V9. STRACE moves fromLOG_INFO_V9to thetiminglevel;simpler_initmaps the device (CANN) level sotiming → WARN. Result:timing→ host emits only STRACE (+ warn/error); device level = WARN → no driver-log tax on any run (not justrounds > 1);info/debug→ device INFO back on for debugging, without the DEBUG flood;--log-levelstill drives both host and device.Benchmark measurement is unaffected either way —
[STRACE]rides host stderr + thertMemcpydevice-phase readback, not the CANN log.This supersedes the per-run
ASCEND_GLOBAL_LOG_LEVELenv-var mitigation in #1409, and depends on the footprint reduction in #1428 landing first.Location
src/common/platform/onboard/host/c_api_shared.cpp(simpler_init→dlog_setlevel(-1, HostLogger::level(), 0))src/common/log/include/common/strace.h(LOG_INFO_V9→timing)python/simpler/_log.py(_split_threshold),simpler_setup/log_config.py(LOG_LEVEL_CHOICES),conftest.py(V0..V9 registration),src/common/log/host_log.cpp(info_v),src/common/platform/onboard/aicpu/device_log.cpp(g_log_info_v),src/common/log/include/common/unified_log.h(LOG_INFO_V0..V9)src/a2a3/**andsrc/a5/**LOG_INFO_V*sitesdocs/logging.md,docs/testing.md,docs/troubleshooting/device-error-codes.md,.claude/rules/running-onboard.mdProposed Fix
Phased (after #1428):
debug / info / timing / warn / error(single severity axis,timingbetweeninfoandwarn); maketimingthe default. Emit STRACE attiming.simpler_init:debug→DEBUG, info→INFO, timing→WARN, warn→WARN, error→ERROR— so defaulttimingno longer seeds CANN INFO.V0..V9(or keep as deprecated aliases:v9→timing,v5–v8→info,v0–v4→debug) and migrate remaining call sites + the--log-levelsurface + docs.Open decisions to settle in the issue: hard-remove vs deprecated aliases for
v0..v9; whether the 109 must-see device diagnostics map toinfoordebug.Priority
Medium (real device-side overhead, but #1409 provides a stopgap; depends on #1428)
Context / root-cause discussion: #1409. Depends on: #1428. Related: #409 (hot-path logging overhead).