Skip to content

Run on central AnaTuples/AnaCaches via version flags + fix remote log location reporting#269

Merged
kandrosov merged 2 commits into
cms-flaf:mainfrom
kandrosov:fix-test-hello-world-deterministic
Jun 15, 2026
Merged

Run on central AnaTuples/AnaCaches via version flags + fix remote log location reporting#269
kandrosov merged 2 commits into
cms-flaf:mainfrom
kandrosov:fix-test-hello-world-deterministic

Conversation

@kandrosov

Copy link
Copy Markdown
Contributor

Summary

Two related changes for working with centrally produced AnaTuples/AnaCaches and for
correct failed-job log reporting with the bundle system.

1. Run on existing central AnaTuples/AnaCaches (consume side)

Adds convenience version parameters so a run can read centrally produced upstream
outputs while producing its own downstream outputs at --version:

  • --anaTuple-version — forces the version of all upstream AnaTuple/AnaProd tasks
    (InputFileTask, AnaTuple*List*, AnaTupleMergeTask).
  • --anaCache-version — forces the version of AnalysisCacheTask /
    AnalysisCacheAggregationTask (central BtagShape etc.).
  • --ana-version — single flag combining both.

Upstream production dependencies (and their bundles) are pruned via self.complete()
early-returns when the central outputs already exist, so InputFileTask and friends no
longer appear in the graph. bundle_flavours carries a (flavour, version) tuple so the
AnaTupleFileList bundle is pulled at the central version.

Verified by constructing the task graph: with --anaTuple-version v2605a,
AnaTupleMergeTask/AnaTupleFileListTask resolve to v2605a while
AnalysisCacheTask.version stays at the dev --version; --anaCache-version resolves the
cache tasks to the central version.

2. fix #265: failed-task log locations now point at EOS

With bundles, job logs are staged to EOS by stageout_logs.sh, but failure reports still
advertised the old AFS path. PR #267 added _BundleAwareHTCondorWorkflowProxy to remap the
log entry, but it never took effect:

This PR completes the fix:

  • sets HTCondorWorkflow._defined_workflow_proxy = True so the proxy is actually used;
  • computes the remote log base in _submit_group from self.task
    (remote_dir_target(version, "logs", <TaskClass>, period)), matching what stageout_logs.sh
    uploads;
  • removes Fixed reported log file locations for failed tasks with new remote log location #267's now-dead render-variable fallback (provably unreachable — same WLCG condition
    and computation as the primary path);
  • stageout_logs.sh pre-creates the remote parent dir (gfal-mkdir -p) so the log lands at
    the exact single path instead of a nested stdall_0To1.txt/stdall_0To1.txt.

Also adds a worker-side dev overlay (bootstrap.sh sources flaf_dev.sh; bundle/stageout
scripts resolved via FLAF_PATH) so jobs run edited FLAF on the worker, and a deterministic
test/test_hello_world.py + test/test_hello_world_all.py harness.

Testing

test/test_hello_world_all.py runs 6 variants (local/htcondor × success/force-fail, ± bundle).
The force-fail htcondor variants assert the reported log: path equals the remote EOS URL and
that no AFS /data/.../stdall path is mentioned. All 6 pass. A dedicated re-run after the #267
cleanup confirmed the failure report shows
log: davs://eoshome-k.cern.ch:8444/.../logs/HelloWorldTask/Run3_2022EE/stdall_0To1.txt.

Notes

…ion/--anaCache-version/--ana-version and prune upstream production deps
…or/bundle jobs; FLAF_PATH/CORRECTIONS_PATH-based dev overlay (no AFS access in bundle mode) and deterministic hello-world test
@kandrosov kandrosov force-pushed the fix-test-hello-world-deterministic branch from 378adc1 to 6da2bf0 Compare June 15, 2026 12:44
@cms-flaf-bot

Copy link
Copy Markdown
Collaborator

pipeline#15033697 started

@cms-flaf-bot

Copy link
Copy Markdown
Collaborator

pipeline#15033697 passed

@kandrosov kandrosov merged commit 738d423 into cms-flaf:main Jun 15, 2026
5 checks passed
@kandrosov kandrosov deleted the fix-test-hello-world-deterministic branch June 15, 2026 14:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Task failed log file locations are wrong with new bundles

2 participants