Skip to content

[Feature] Track reader accesses in TensorMap and derive WAR dependencies automatically #1388

Description

@ChaoWao

Summary

Extend TensorMap from a producer-only map into an outstanding-access tracker so automatic dependency generation derives write-after-read (WAR) edges for overlapping tensor regions. Pure INPUT readers must remain concurrent with each other, while a later INOUT or OUTPUT_EXISTING writer automatically waits for every overlapping live reader.

Motivation / Use Case

Issue #1306 demonstrated a deterministic correctness failure for a loop-carried buffer:

W0: add_inout(buf)
R0: add_input(buf)
W1: add_inout(buf)

The current TensorMap records only writers, so it derives W0 -> R0 (RAW) and W0 -> W1 (WAW), but misses R0 -> W1 (WAR). W1 may overwrite buf while R0 is reading it, causing silent wrong results.

Promoting readers to INOUT serializes independent readers. Requiring explicit dependencies makes every frontend/user responsible for detecting aliasing WAR patterns. Automatic scopes should preserve correctness without losing reader concurrency.

Related: #1306 and hw-native-sys/pypto#2058.

Proposed API / Behavior

Keep the public orchestration API unchanged and enhance TensorMap internally.

  1. Add an access kind (READER/WRITER) to PTO2TensorMapEntry; generalize producer_task_id into the task owning the access.
  2. After fanin calculation, register automatic-scope, non-manual_dep INPUT arguments as reader entries. Continue registering INOUT and OUTPUT_EXISTING as writers.
  3. Apply access-specific lookup rules:
INPUT:
    depend on overlapping writers (RAW)
    never depend on readers
    register reader

INOUT:
    depend on overlapping writers (RAW/WAW)
    depend on overlapping readers (WAR)
    retire fully covered old accesses
    register writer

OUTPUT_EXISTING:
    preserve existing write-only semantics for old writers
    depend on overlapping readers (WAR)
    retire fully covered old accesses
    register writer
  1. Reuse existing overlap and per-task cleanup semantics. A fully covering writer may remove an old access after wiring its dependency; partially overlapping entries must remain. Reader entries become stale when their task retires.
  2. Host reads should wait only for writers. Host writes, including set_tensor_data(), should also wait for overlapping readers.
  3. Replace output-only entry counting with access-entry counting. Re-evaluate entry-pool/bucket sizing, arena memory, pool back-pressure, bucket-chain length, and fanin spill capacity.
  4. Keep runtime and replay behavior equivalent across a2a3 tensormap_and_ringbuffer, a2a3 host_build_graph, a5 tensormap_and_ringbuffer, and dep_gen_replay/DFX paths.

A correctness-first implementation may use one mixed chain with access-kind filtering. If reader-heavy profiling shows a significant RAW lookup regression, split reader/writer bucket indexes while sharing the entry pool.

Acceptance criteria

  • W0(inout X) -> R0(input X) -> W1(inout X) contains R0 -> W1.
  • Multiple pure readers remain mutually independent.
  • A writer depends on every overlapping live reader, with fanin deduplication.
  • Disjoint regions create no WAR edge.
  • Partial overlap preserves the reader entry; full coverage may retire it after wiring.
  • Retired readers create no stale dependency.
  • Cross-ring WAR works.
  • set_tensor_data() waits for a pure INPUT reader of the target region.
  • The [Bug] Task dep-gen misses WAR edge: reader task not ordered before a later aliasing inout writer #1306 loop-carried reproducer passes on supported variants.
  • Chain length, pool high-water mark, orchestration latency, and fanin spill usage are measured before default enablement.

Alternatives Considered

  • Promote readers to INOUT: creates the edge but serializes readers as WAW.
  • Require add_dep: precise, but omitted detection silently corrupts results.
  • Detect only in PyPTO: useful for compatibility, but does not protect other orchestration producers.
  • Start with separate maps: cleaner reader-heavy performance but a larger initial layout change than a mixed access-kind chain.

Additional Context

Relevant implementation areas:

  • src/{arch}/runtime/{variant}/runtime/pto_dep_compute.h
  • src/{arch}/runtime/{variant}/runtime/pto_tensormap.h
  • src/{arch}/runtime/{variant}/runtime/pto_orchestrator.cpp
  • src/{arch}/runtime/{variant}/runtime/pto_runtime2.cpp
  • src/{arch}/runtime/{variant}/host/dep_gen_replay.cpp

Current commit reviewed: cddf2b6cb38e2f9bcc4739b5f888feaf33129675.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions