Skip to content

Upgrade miniwdl to 1.15.0 and refresh WDL unit test skip lists (#5377) - #5567

Open
annagiroti wants to merge 2 commits into
masterfrom
issues/5377-wdl12-allow-nested-inputs
Open

Upgrade miniwdl to 1.15.0 and refresh WDL unit test skip lists (#5377)#5567
annagiroti wants to merge 2 commits into
masterfrom
issues/5377-wdl12-allow-nested-inputs

Conversation

@annagiroti

@annagiroti annagiroti commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator

miniwdl 1.15.0 adds parsing support for WDL 1.2 hints sections, unblocking several previously-skipped conformance unit tests and letting most nested-input tests run. Removes 30 entries across the WDL 1.1 and 1.2 unsupported-test lists that now pass, and updates the remaining entries' comments to reflect their current (rather than pre-upgrade) failure reasons.

miniwdl 1.15.0 also changed CallCache.put()'s signature, adding two new required keyword arguments, inputs and add_paths. Threads the actual input bindings through to all three call sites (task cache, workflow cache, and the write_* file cache) so cache entries record real inputs; add_paths is passed empty since Toil doesn't track that concept yet.

Addresses #5377

Changelog Entry

To be copied to the draft changelog by merger:

  • PR submitter writes their recommendation for a changelog entry here

Reviewer Checklist

  • Make sure it is coming from issues/XXXX-fix-the-thing in the Toil repo, or from an external repo.
    • If it is coming from an external repo, make sure to pull it in for CI with:
      contrib/admin/test-pr otheruser theirbranchname issues/XXXX-fix-the-thing
      
    • If there is no associated issue, create one.
  • Read through the code changes. Make sure that it doesn't have:
    • Addition of trailing whitespace.
    • New variable or member names in camelCase that want to be in snake_case.
    • New functions without type hints.
    • New functions or classes without informative docstrings.
    • Changes to semantics not reflected in the relevant docstrings.
    • New or changed command line options for Toil workflows that are not reflected in docs/running/{cliOptions,cwl,wdl}.rst
    • New features without tests.
  • Comment on the lines of code where problems exist with a review comment. You can shift-click the line numbers in the diff to select multiple lines.
  • Finish the review with an overall description of your opinion.

Merger Checklist

  • Make sure the PR passed tests, including the Gitlab tests, for the most recent commit in its branch.
  • Make sure the PR has been reviewed. If not, review it. If it has been reviewed and any requested changes seem to have been addressed, proceed.
  • Merge with the Github "Squash and merge" feature.
    • If there are multiple authors' commits, add Co-authored-by to give credit to all contributing authors.
  • Copy its recommended changelog entry to the Draft Changelog.
  • Append the issue number in parentheses to the changelog entry.

miniwdl 1.15.0 adds parsing support for WDL 1.2 `hints` sections,
unblocking several previously-skipped conformance unit tests and
letting most nested-input tests run. Removes 30 entries across the
WDL 1.1 and 1.2 unsupported-test lists that now pass, and updates the
remaining entries' comments to reflect their current (rather than
pre-upgrade) failure reasons.
@annagiroti
annagiroti requested a review from adamnovak August 18, 2026 07:10

@adamnovak adamnovak left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks OK, but I think for the return codes ones we might be removing information that's still maybe accurate.

I think the CI failed due to the code not type checking against the new MiniWDL, so you might need to fix that.

Comment thread src/toil/test/wdl/wdltoil_test.py Outdated
Comment on lines +65 to +66
"single_return_code_task", # Workflow did not return the correct return code! Got: 1. Expected: 0.
"all_return_codes_task", # Workflow did not return the correct return code! Got: 42. Expected: 0.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Has MiniWDL 1.15 learned about return_codes? I think the Got: codes are probably right for these and it's the expected codes that are wrong.

@adamnovak adamnovak left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks OK, but we probably want to redesign our own caching stuff a bit if the MiniWDL caching now needs the inputs and not just the key.

Comment thread src/toil/wdl/wdltoil.py
Comment on lines 857 to +861
cache_key: str,
output_bindings: WDLBindings,
file_store: AbstractFileStore,
wdl_options: WDLContext,
input_bindings: WDLBindings,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we now need to pass the whole input bindings to the cache, we probably should move responsibility for computing the key inside this function.

@adamnovak

Copy link
Copy Markdown
Member

I looked a little at the failing tests.

I think basic_directory is failing because now the WDL code is seeing absolute paths, and its regex in out wdl-conformance-tests repo doesn't have a .* before the first item:

https://github.com/DataBiosphere/wdl-conformance-tests/blob/d3e05db85696f5c5c5a5f5e1785c64fb2229b097/conformance.yaml#L1233

@annagiroti I invited you to that repo so you should be able to fix it there. You can run that test in that repo with:

 python3 run.py --versions 1.2 --id basic_directory --runner toil-wdl-runner

For the spec tests, the test_map test has started failing with a bunc of complaints about:

	WDL.Error.EvalError: source-relative File/Directory path resolution requires runtime context

I'm not sure why, but you can run it with:

python3 setup_unit_tests.py --version 1.2 --repo https://github.com/adamnovak/wdl.git --branch patch-3 --force-pull
python3 run_unit.py --versions 1.2 --id test_map

(This grabs a patched version of the spec tests that has openwdl/wdl#799 so it actually contains only parse-able JSON. You could also set up the pinned version of the spec tests that Toil is using.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants