Skip to content

sccache: Regenerate local depfiles on direct hits - #2

Open
geir-t wants to merge 2 commits into
mainfrom
geir/d_files
Open

sccache: Regenerate local depfiles on direct hits#2
geir-t wants to merge 2 commits into
mainfrom
geir/d_files

Conversation

@geir-t

@geir-t geir-t commented Aug 13, 2026

Copy link
Copy Markdown

Problem

With a multilevel disk,webdav cache and SCCACHE_BASEDIRS, dependency-file
behavior differs between an L1 WebDAV hit and the following L0 disk hit.

The shared entry was produced under /home/jenkins, while the developer build
runs under /home/geir. On the WebDAV hit, local preprocessing creates a valid
local .d, so sccache ignores the cached .d. The raw WebDAV archive is then
backfilled to disk, including its Jenkins .d.

On the next clean build, the direct/preprocessor manifest hits. No local
preprocessing occurs, and the backfilled .d is extracted from disk. Ninja
then records /home/jenkins/.conan2/... dependencies. If /home/jenkins is
not traversable, a later Ninja invocation fails with stat(...): Permission denied.

Reproduction

rm -rf ~/.cache/sccache
./build.py -m --icefish --no-image   # WebDAV hits; backfill disk
rm -rf build
./build.py -m --icefish --no-image   # disk hits
strings build/armv8/Release/.ninja_deps | grep -c /home/jenkins/

Before this change, the second build recorded 4,943 Jenkins paths.

Solution

The preprocessor manifest already contains and validates the exact local
source/header paths associated with the direct cache key. For the supported
Unix GCC dependency form (-MD, one -MT or -MQ, no -MP), this change:

  1. creates a local Make-format .d from those validated manifest paths;
  2. restores the cached object but does not overwrite the local .d; and
  3. keeps the existing direct/object cache keys.

Unsupported dependency modes, path syntax, and platforms fall back to normal
local preprocessing. Behavior without SCCACHE_BASEDIRS is unchanged.

This avoids both unsafe text substitution of producer paths and the cost of
rerunning the compiler preprocessor on every direct hit.

The Huddly package version is bumped from 0.17.1 to 0.17.2.

Verification

Exact empty-L0 to L1-backfill to warm-L0 sequence:

  • First build: 653/655 WebDAV hits, 653 disk backfills, zero Jenkins paths.
  • Second build: 655/655 disk hits, zero compilations, zero Jenkins paths.
  • Incremental Ninja build succeeds while /home/jenkins is non-traversable.

For one real RoomMap.cpp cache entry, the original Jenkins .d and generated
local .d each contained 530 unique dependencies. After normalizing the two
producer base paths, the sorted dependency sets were byte-identical and had
the same SHA-256 digest.

Timing

Measured on the same Falcon build:

Case Wall time
Existing direct-mode warm L0 34.46 s
First correctness fix (rerun preprocessing) 91.21 s
This change, warm L0 27.35 s
SCCACHE_DIRECT=false, warm L0 average 97.62 s

The exact non-Icefish command reported by the developer improved from 93.70 s
to 25.99 s with this change (518/518 cache hits).

Tests

  • cargo fmt --check
  • cargo test compiler::c::test (17 passed)
  • cargo test compiler::preprocessor_cache::test (11 passed)
  • cargo +nightly clippy --workspace --all-targets --all-features -- via
    pre-commit
  • git diff --cached --check

The pre-commit rustfmt hook itself currently invokes cargo +nightly fmt --
with staged filenames, which loses the crate's Rust 2024 edition context and
rejects existing let-chain syntax. The edition-aware cargo fmt --check
passes.

SCCACHE_BASEDIRS can give identical cache keys to builds rooted in different
locations. A dependency file backfilled unchanged from a slower cache level
can therefore expose the producer's paths on a later direct L0 hit.

Use the already validated preprocessor manifest to create a local depfile for
the supported GCC -MD form. Fall back to preprocessing for formats that cannot
be reproduced safely. This retains direct-mode performance while preventing
foreign paths from reaching Ninja.

Bump the Huddly package version to 0.17.2.
@geir-t

geir-t commented Aug 21, 2026

Copy link
Copy Markdown
Author

Review finding

  • P1 — Add an automated cross-root direct-cache regression test before merging. The patch's essential behavior is untested in CI: write_dependency_file_from_manifest() is only unit-tested through
    dependency_target(), while the existing test_gcc_clang_depfile neither sets SCCACHE_BASEDIRS nor exercises an L1-backfilled, warm-L0 direct hit. This exact path is where the bug lives. A future refactor
    could restore foreign .d files without failing the current suite.

    The test should:

    1. populate the cache from source root A;
    2. build identical source root B with SCCACHE_BASEDIRS;
    3. ensure a warm direct-cache hit restores the object but leaves a B-root depfile; and
    4. assert no producer-root path appears in the .d file.

I found no functional defect in the implementation itself. The PR correctly handles Falcon's -MD -MT … -MF … form: it creates a local depfile from the validated direct-mode manifest and filters the cached d
artifact before extraction.

The PR is exactly the needed fix, and it is cleanly based on current main; after adding the regression test, I would approve it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant