feature/SOF-8004 feat: add Custom Python Script workflow - #150
Open
VsevolodX wants to merge 7 commits into
Open
feature/SOF-8004 feat: add Custom Python Script workflow #150VsevolodX wants to merge 7 commits into
VsevolodX wants to merge 7 commits into
Conversation
VsevolodX
force-pushed
the
feature/SOF-8004
branch
from
August 20, 2026 19:56
3cfeae5 to
b380451
Compare
Runs a user-supplied Python script against a material. The unit chain is the point: an object_storage IO unit fetches the uploaded script and any files it reads - the script's own parameters travel as one of those files - an api IO unit fetches the material, and an assignment puts MATERIAL into the workflow scope for the runner to write out as material.json. Uses the existing hello_world flavor rather than a new one. The flavor name in a saved workflow is resolved against every consumer's ApplicationRegistry - Python standata in JupyterLite, npm standata in the web-app server and browser - so a new flavor makes any client without the release fail with "Flavor ... not found". hello_world already provides the two input slots needed (script.py, requirements.txt), whose contents the caller overrides per job. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
VsevolodX
force-pushed
the
feature/SOF-8004
branch
from
August 21, 2026 02:54
b380451 to
9f9f29b
Compare
The wip-88b8881 release tarball the pin referenced was removed by the stale-WIP cleanup, so npm install 404s and every CI run on this repo fails at the install step - main included. The package is published to the registry; 2026.8.18-0 is the same content the tarball carried. [release] so the standata WIP tarball for this branch republishes on top of the fixed install. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
VsevolodX
changed the title
io-user-files carried an explicit flowchartId that nothing read - the builder generates one, as it already does for assign-material and custom_script. io-material's must be pinned because assign-material reads its scope by id, but the invented hex suffix said nothing; it is now custom-script-io-material, namespaced so it cannot clash with another subworkflow fetching a material in the same workflow. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…cript [release] Same four-unit chain - io(object_storage) for user files, io(api) for the material, assignment of MATERIAL, and an execution unit built from the shell app's default hello_world flavor. One pinned flowchartId, custom-shell-io-material, because assign-material reads that unit's scope; everything else takes builder UUIDs. Enables notebooks where the user's shell script runs any node-side application (e.g. Quantum ESPRESSO) against uploaded files.
…d version [release] CI on every branch, main included, fails with a 404 on wip-8b2c388/utils.tgz - the WIP release has been cleaned up. Same failure and same fix as the @mat3ra/ide pin earlier on this branch: pin the latest published version (2026.8.18-1).
…d version [release] Second and last dead WIP pin (swept the whole package.json this time): the wip-1d8901b prode tarball is gone the same way the utils one was. Pinned the latest published 2026.8.18-0.
Main brought the nwchem relaxation assets, the espresso formation-energy fix and the published-version repins; this branch adds the Custom Python Script and Custom Shell Script workflows. The two generated data modules conflicted and were regenerated by the build rather than merged by hand; both custom workflows and main's additions are present in the built output.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of SOF-8004 — "Create NB for Custom Python Calculation". Merge this one first; the api-examples notebook resolves this workflow by name, and web-app seeds it into the workflow bank for UI use.
What
A
Custom Python Scriptworkflow (+ subworkflow) that runs a user-supplied Python script against a material. Minimal four-unit chain:io-user-files— io,object_storagesettings.jsonparameters — onto the compute node.inputis filled per job, since the objects only exist once uploadedio-material— io,apiassign-materialMATERIAL = DATA[0], into workflow scope for the runnercustom_script— executionBuilderScript parameters deliberately travel as an uploaded
settings.json, not a dedicated assignment unit: re-running a saved workflow with new parameters means overwriting one small object, with no edit to the workflow itself.Why the existing
hello_worldflavor, and not a new oneA first pass added a
custom_scriptflavor with the runner as a standata template. That fails hard: the flavor name in a saved workflow is resolved against every consumer'sApplicationRegistry— Python standata inside JupyterLite, npm standata in both the web-app server and the browser — so any client without the release dies withFlavor custom_script not found. Observed in a browser during development.hello_worldalready provides the two input slots needed (script.py,requirements.txt), whose contents the caller sets per job.Also here
@mat3ra/iderepointed at its published npm version2026.8.18-0— thewip-88b8881tarball it referenced was removed by the stale-WIP cleanup, which brokenpm install(and therefore all CI on this repo,mainincluded).Verification
Exercised end to end against a local platform and the real compute cluster, via the api-examples notebook and its Cypress feature: three jobs, all four units
finished, results fixed by crystal structure (Si → 4, graphene C → 3 coordination; diamond C → 4 atcutoff_scale1.3 via a re-uploadedsettings.jsonon the saved workflow).npm run buildclean; unrelated build drift indata/workflows/workflows/vasp/neb.jsondeliberately left out.🤖 Generated with Claude Code