Topic/drm amdgpu gtt lock timeout ms

[chun-wan]

Add a new module parameter gtt_lock_timeout_ms (default 0 = unbounded,
byte-identical to the current mutex_lock() behaviour) that caps the
wait time for adev->mman.gtt_window_lock in the three amdgpu_ttm.c
paths that take it. At the deadline the helper returns -ETIME so the
caller fails fast instead of parking indefinitely on a wedged SDMA ring.

Motivation

In multi-tenant HIP/AMDGPU serving workloads, a single wedged SDMA
ring can park every caller of amdgpu_ttm_copy_mem_to_mem(),
amdgpu_ttm_clear_buffer(), and amdgpu_fill_buffer() on the global
mman.gtt_window_lock mutex for minutes-plus while a single in-flight
copy waits for the ring. The hold time is bounded only by the SDMA
recovery path (which itself may need tens of seconds). The legacy
unbounded mutex_lock() then converts a localised SDMA hang into a
system-wide GTT-window stall affecting every VRAM-touching ioctl from
every tenant on the device.

gtt_lock_timeout_ms lets operators opt the GTT window contention
path into a bounded-wait failure mode (return -ETIME) so the higher-
level survival policy can choose whether to retry, re-queue, or
surface the error to the application.

Technical Details

• drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
  Declare int amdgpu_gtt_lock_timeout_ms; + module_param_named()
  • MODULE_PARM_DESC() + DOC comment. Default 0 = unbounded.
• drivers/gpu/drm/amd/amdgpu/amdgpu.h
  Add extern int amdgpu_gtt_lock_timeout_ms;.
• drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
  Add static int amdgpu_ttm_lock_gtt_window(struct amdgpu_device *adev)
  helper. When amdgpu_gtt_lock_timeout_ms == 0, the helper
  degrades to a plain mutex_lock(&adev->mman.gtt_window_lock)
  and returns 0 (byte-identical to current behaviour). Otherwise
  it does a mutex_trylock() loop with schedule_timeout_interruptible(1ms)
  bounded by the configured deadline, returning -ETIME on expiry
  or -ERESTARTSYS on pending signal. Values below 100 ms are
  clamped up to 100 ms to avoid a fail-only path.
  Three callsites switch from mutex_lock(&...gtt_window_lock) to
  r = amdgpu_ttm_lock_gtt_window(adev); if (r) return r;:
  • amdgpu_ttm_copy_mem_to_mem() (SDMA buffer<->buffer copy)
  • amdgpu_ttm_clear_buffer() (SDMA buffer clear/wipe)
  • amdgpu_fill_buffer() (SDMA buffer fill)
    mutex_unlock() at each function's error label is unchanged.

JIRA ID

ROCM-21571

Test Plan

Reproducer: multi-tenant SDMA-bound workload with one wedged ring
plus concurrent VRAM-touching ioctls from co-tenants. Open-source
reproducer to be added in a follow-up test patch.

Expected behaviour:

• gtt_lock_timeout_ms unset / 0: stock behaviour; all existing
  tests pass byte-identically (helper degrades to mutex_lock()).
• gtt_lock_timeout_ms=4000: under multi-tenant SDMA load, the
  worst-case wait on the lock is ~4 seconds (matching the
  configured deadline), with -ETIME propagated back to the caller
  cleanly.

Test Result

• All existing drm-tip self-tests pass at gtt_lock_timeout_ms=0
  (default). No behavioural change for stock callers.
• Multi-tenant SDMA reproducer confirms bounded ~4 s tail latency on
  the GTT window-lock contention path with gtt_lock_timeout_ms=4000
  vs unbounded minutes pre-patch.

Reviewers may suggest using mutex_lock_killable_timeout() if
preferred over the explicit trylock-and-sleep loop; the current form
keeps the dependency surface minimal (no kernel API additions).

Submission Checklist

• Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.

Signed-off-by: chun-wan chun-wan@users.noreply.github.com