Override TMPDIR etc. to /tmp for skynet bdsf run

[tikk3r]

This addresses an issue where PyBDSF will crash with OSError: AF_UNIX path too long due to a temporary directory path becoming too long. This prevented me from running delay calibration with a model_image specified on cosma.

[jurjen93]

I ran into the same issue this morning, so I am very happy with this fix :-)

[gmloose]

This can be solved by adding the following lines to the CWL CommandLineTool:

hints:
  - class: EnvVarRequirement
    envDef:
      TMPDIR: /tmp

No need to tweak the environment in your Python script.

HOWEVER -- roadblock ahead! -- the latest cwltool introduced a bug that breaks the solution above. I would say that adapting your script is the "work around it" method, and it is easily forgotten to remove those lines again. The "proper" solution, IMHO, is to temporarily downgrade CWLTool (and consequently Toil), until this bug has been fixed. Mattia Mancini already make a pull request for this..

[tikk3r]

This can be solved by adding the following lines to the CWL CommandLineTool:

hints:
  - class: EnvVarRequirement
    envDef:
      TMPDIR: /tmp

No need to tweak the environment in your Python script.

HOWEVER -- roadblock ahead! -- the latest cwltool introduced a bug that breaks the solution above. I would say that adapting your script is the "work around it" method, and it is easily forgotten to remove those lines again. The "proper" solution, IMHO, is to temporarily downgrade CWLTool (and consequently Toil), until this bug has been fixed. Mattia Mancini already make a pull request for this..

I tried adding

hints:
  - class: EnvVarRequirement
    envDef:
      TMPDIR: /tmp
      APPTAINERENV_TMPDIR: /tmp
      SINGULARITYENV_TMPDIR: /tmp

at first, but that didn't seem to take hold and still use my locally defined APPTAINER_TMPDIR as bind path to /tmp. That is why I opted for this now.

There seem to be a couple of breaking changes in recent cwltool indeed, as I cannot run or validate LINC with many of them beyond the versions mentioned in the JSON error ticket at LINC.

[lonbar]

I think it would help if we discuss why that error appears. It may be a problem with bdsf.

At a glance I would be inclined to agree with @gmloose. I can confirm that the bug appears in both cwltool and toil (though --preserve-entire-environment doesn't seem to be required). If the path is already under review I see no strong reason not to wait until it is merged.

The latest version of cwltool which doesn't have this problem (and I can install via pip) is 3.1.20240708091337, which is quite old. One flocs version that has a working version of cwltool is v.5.4.0.

[lonbar]

Flocs versions as recent as v5.7.0 don't seem to have the bug that @gmloose described.

[tikk3r]

I guess one question is now then: how long should we wait for the fix to be merged? In this small scenario I think function should take priority over form. We can't seem to run this aspect of the pipeline without one of these fixes and downgrading Toil and cwltool would reintroduce other bugs that have been fixed since.