[cncf-kubernetes] Defer kubernetes_asyncio import in callbacks to TYPE_CHECKING#3
Open
dor-bernstein wants to merge 1 commit into
Open
[cncf-kubernetes] Defer kubernetes_asyncio import in callbacks to TYPE_CHECKING#3dor-bernstein wants to merge 1 commit into
dor-bernstein wants to merge 1 commit into
Conversation
…E_CHECKING Importing airflow.providers.cncf.kubernetes.operators.pod (the base KubernetesPodOperator) transitively imports callbacks.py, which eagerly did `import kubernetes_asyncio.client` (and `import kubernetes.client`) at module scope. The async client is heavy (~0.2s; pulls aiohttp + ~625 generated model modules) and is only needed at runtime in the triggerer for deferrable pods — yet every DAG that builds a KubernetesPodOperator paid it at parse time. The two client imports are used only for the module-level `client_type: TypeAlias = k8s.CoreV1Api | async_k8s.CoreV1Api` and for parameter annotations. Because this module uses `from __future__ import annotations` (PEP 563), all those annotations are strings and never evaluated at runtime, so the imports and the alias only need to exist for static type checkers. Move both client imports and the alias under TYPE_CHECKING. Behavior is unchanged (ExecutionMode and KubernetesPodOperatorCallback still import normally); kubernetes/kubernetes_asyncio are no longer loaded at parse. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Move the
kubernetes.clientandkubernetes_asyncio.clientimports — and theclient_typetype alias — inproviders/cncf/kubernetes/.../callbacks.pyunderTYPE_CHECKING.Why
Importing
airflow.providers.cncf.kubernetes.operators.pod(the baseKubernetesPodOperator) transitively importscallbacks.py. That module eagerly ranimport kubernetes_asyncio.client(andimport kubernetes.client) at module scope, so every DAG that builds aKubernetesPodOperatorat parse time paid the cost of loading the async client — which pulls inaiohttp+ ~625 generated model modules (~0.2s) and is only actually needed at runtime in the triggerer for deferrable pods.The two client imports were used only for:
client_type: TypeAlias = k8s.CoreV1Api | async_k8s.CoreV1Api, andSince the module uses
from __future__ import annotations(PEP 563), all those annotations are strings and are never evaluated at runtime — so the imports and the alias only need to exist for static type checkers.Change
Both client imports and the
client_typealias are moved into the existingif TYPE_CHECKING:block. No public behavior changes:ExecutionModeandKubernetesPodOperatorCallbackstill import normally;client_typehas no runtime importers (onlyExecutionMode/KubernetesPodOperatorCallbackare imported from this module, byoperators/pod.pyandutils/pod_manager.py).Verification
Loading the module standalone before/after:
kubernetes_asyncioinsys.modulesafter importkubernetes: False)ExecutionMode.ASYNCandKubernetesPodOperatorCallbackstill resolve correctly after the change.