Skip to content

fix(backend): handle async functions in AST parsing utilities - #892

Open
harshhh817 wants to merge 1 commit into
kubeflow:mainfrom
harshhh817:fix-async-function-parsing
Open

fix(backend): handle async functions in AST parsing utilities#892
harshhh817 wants to merge 1 commit into
kubeflow:mainfrom
harshhh817:fix-async-function-parsing

Conversation

@harshhh817

Copy link
Copy Markdown
Contributor

What this does

Treats ast.AsyncFunctionDef the same as ast.FunctionDef in kale/common/astutils.py, so async def helpers in a functions/step cell are handled correctly by:

  • parse_functions — now extracts async functions (previously dropped them).
  • get_function_and_class_names — now registers async function names.
  • get_marshal_candidates — now stops at async function bodies, so it no longer leaks their local variables and correctly adds the function name.

Why

astutils only checked ast.FunctionDef, so an async def was mishandled in three ways. Reproducer on main:

from kale.common import astutils

astutils.parse_functions("async def foo():\n    return 1")
# {}                      -> expected {"foo": ...}

astutils.get_function_and_class_names("async def foo():\n    pass")
# set()                   -> expected {"foo"}

astutils.get_marshal_candidates("async def foo():\n    local_x = 1\n    return local_x")
# ['local_x']             -> expected ['foo']

The get_marshal_candidates case is the most harmful: it walked into the async function body and returned the function's local variable as a marshal candidate while missing the function name. In a pipeline this means an async def helper would not be injected into steps, would be treated as a missing dependency, and its internals could be marshalled between steps.

Testing

Extended kale/tests/unit_tests/test_ast.py with an async snippet and a dedicated parse_functions async test. All three new assertions fail on main and pass with this change; the full test_ast.py suite (47 tests) passes.

I came across this while working on the output-artifact/AST code in #874. Happy to open a tracking issue if maintainers prefer.

@google-oss-prow
google-oss-prow Bot requested a review from jesuino July 21, 2026 19:14
@google-oss-prow

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign jesuino for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@harshhh817

Copy link
Copy Markdown
Contributor Author

Friendly ping — small backend fix: astutils only checked ast.FunctionDef, so an async def helper was dropped by parse_functions, its name was missing from get_function_and_class_names, and get_marshal_candidates walked into its body and returned the function's local variables as marshal candidates.

Regression tests included; still green. Happy to rebase onto current main if that helps.

astutils only checked ast.FunctionDef, so async def helpers were
mishandled: parse_functions dropped them, get_function_and_class_names
did not register their names, and get_marshal_candidates walked into
their bodies -- leaking the function's local variables as marshal
candidates while missing the function name itself.

Treat ast.AsyncFunctionDef the same as ast.FunctionDef in all three, so
an async def in a functions/step cell is parsed, its name recognised,
and its locals no longer leak between steps. Add regression tests.

Signed-off-by: Harsh Gupta <harshgupta93198@gmail.com>
@harshhh817
harshhh817 force-pushed the fix-async-function-parsing branch from a80ce93 to 4193cc4 Compare September 8, 2026 05:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant