ACPI thermal zone permanent deadlock: Sleep opcode in _TMP method holds mutex, all thermal kworkers blocked

[btdwv]

Orange Pi 6 Plus — ACPI Thermal Zone Permanent Deadlock

System Info

• Board: Orange Pi 6 Plus (CIX Phecda / CD8180)
• BIOS: 1.4 (01/30/2026)
• Kernel: 7.0.0-37-cix patches-7.0: replace cdnsp-sky1 shutdown SError mitigations with host teardown #37-Ubuntu SMP PREEMPT_DYNAMIC (built May 19 2026)
• Tainted at crash: G S W — [S]=CPU_OUT_OF_SPEC, [W]=WARN
• Hardware: Cix Technology Group Co., Ltd. CIX Phecda Board

Summary

ACPI thermal zone temperature read method (_TMP) executes a Sleep opcode in AML bytecode
while holding the ACPI mutex. The sleeping thread never releases the lock, causing all other
thermal worker threads (9 threads) to deadlock permanently. The system never recovers —
RCU stalls continue for 17+ hours until forced power-off.

Timeline (2026-07-20 ~ 2026-07-21)

Time	Event
07-20 21:41:50	System boot (kernel 7.0.0-37-cix)
07-21 02:30:00	First RCU stall detected on CPU 0 — thermal deadlock begins
07-21 02:30:26	9 thermal kworker threads blocked >122 seconds on ACPI semaphore
07-21 02:34:32	systemd-oomd watchdog timeout #1 (SIGABRT)
07-21 02:47:23	systemd-oomd watchdog timeout #2
07-21 03:17:13	systemd-oomd watchdog timeout #3
07-21 04:05:33	systemd-oomd watchdog timeout #4
07-21 05:28:33	systemd-oomd watchdog timeout #5
07-21 10:01:32	systemd-oomd watchdog timeout #6
07-21 02:30 ~ 20:03	~200 RCU stall events — system permanently degraded
07-21 22:09	Forced reboot by long-pressing power button (~20 hours after deadlock)

Root Cause

The ACPI firmware's thermal zone _TMP method contains AML bytecode that calls Sleep()
while holding the ACPI namespace evaluation mutex. Once the sleeping thread is scheduled out,
it never gets CPU time to release the lock, permanently blocking all subsequent thermal reads.

Deadlock call chain (from all 9 blocked kworker threads):

thermal_zone_device_check
  → __thermal_zone_get_temp → thermal_get_temp → acpi_thermal_get_temperature
    → acpi_evaluate_integer → acpi_evaluate_object → acpi_ns_evaluate
      → acpi_ps_execute_method → acpi_ps_parse_aml
        → acpi_ds_exec_end_op → acpi_ex_opcode_1A_0T_0R
          → acpi_ex_system_do_sleep → acpi_os_sleep
            ↑ holds ACPI mutex, sleeps indefinitely

Root holder: kworker/u48:4 (PID 49624) — all other thermal workers blocked on:

INFO: task kworker/u48:X blocked on a semaphore likely last held by task kworker/u48:4:49624

Blocked Tasks (all at 02:30:26, all in thermal_events workqueue)

Task	PID	Status
kworker/u48:0	49494	D (blocked >122s)
kworker/u48:1	46730	D (blocked >122s)
kworker/u48:2	36847	D (blocked >122s)
kworker/u48:3	48107	D (blocked >122s)
kworker/u48:4	49624	D — root holder
kworker/u48:5	43983	D (blocked >122s)
kworker/u48:6	51836	D (blocked >122s)
kworker/u48:7	51837	D (blocked >122s)
kworker/u48:8	51838	D (blocked >122s)

RCU Stall Pattern

The RCU stalls continued continuously from 02:30 to 20:03 (17.5 hours), indicating the
system was permanently degraded but not fully dead — userspace processes could still
run (slowly) on non-thermal CPUs, but the kernel scheduling was severely impaired.

Key Observations

1. Permanent deadlock: The ACPI mutex is never released. No self-recovery is possible.
2. Not load-dependent: System had been running idle for ~4.75 hours before the crash.
3. Not reproducible on demand: The system ran fine for those hours before the deadlock
   triggered, suggesting a rare timing condition or ACPI runtime variable state.
4. ACPI firmware bug: The AML bytecode should not call Sleep() while holding the thermal
   zone evaluation lock. This is a CIX BIOS/firmware defect.

Related Issues

• #8 — ACPI LPI / CPPC; deep CPU idle states may not be entered
• #24 — Kernel Error During BOOT on Orange PI 6 (BIOS 1.4)

Attachments

• syslog — Full system log (18MB, covers July 19–21)
• kern.log — Full kernel log (5.1MB)
• systemd-oomd.crash — systemd-oomd crash dump (219KB)
• udevadm.crash — udevadm crash dump (2.9MB)

[btdwv]

opi6p-crash-logs.tar.gz

[srcshelton]

Orange Pi 6 Plus — ACPI Thermal Zone Permanent Deadlock

Looks like torvalds/linux@4096fd0: upgrade to kernel 7.0.1 or later.

(This issue was specific to 7.0.0 and didn't affect 6.18.x or any prior release)

[btdwv]

Root Cause Confirmed

This is not a CIX firmware bug — it's a Linux kernel 7.0.0 regression in the clockevents subsystem.

The Bug

Commit d6e152d ("clockevents: Prevent timer interrupt starvation") introduced a prevention mechanism, but missed resetting the next_event_forced flag in several code paths. This caused:

1. CPU enters idle state
2. next_event_forced flag is stale → timer interrupts are suppressed
3. CPU cannot wake from idle → RCU scheduling stalls
4. Any in-flight ACPI Sleep() operation (like thermal zone temperature reads) depends on timer interrupts to wake up — it never does
5. ACPI mutex is held forever → all thermal kworker threads deadlock
6. System is permanently unresponsive

The Fix

Commit 4096fd0 by Thomas Gleixner (2026-04-16) adds the missing resets. It was merged in v7.0.1 and is also in v7.1-rc1+.

The affected version is only 7.0.0. Kernels 6.18.x and earlier are not affected.

Verification

• My system was running 7.0.0-37-cix #37-Ubuntu — exactly the affected version
• The RCU stall pattern, cpuidle_enter_state hang, and thermal worker deadlock all match the timer interrupt starvation symptom
• aiwintermute's report on Discourse (Radxa Orion O6, same CIX P1 SoC) shows identical RCU stalls

Resolution

Upgrading to kernel 7.0.1 or later resolves this issue. No BIOS/firmware update needed.

Closing this issue as it's a known upstream kernel regression with an available fix.