Skip to content

[cncf-kubernetes] Defer kubernetes_asyncio import in callbacks to TYPE_CHECKING#3

Open
dor-bernstein wants to merge 1 commit into
mainfrom
clut-defer-kubernetes-asyncio-import
Open

[cncf-kubernetes] Defer kubernetes_asyncio import in callbacks to TYPE_CHECKING#3
dor-bernstein wants to merge 1 commit into
mainfrom
clut-defer-kubernetes-asyncio-import

Conversation

@dor-bernstein

Copy link
Copy Markdown
Collaborator

What

Move the kubernetes.client and kubernetes_asyncio.client imports — and the client_type type alias — in providers/cncf/kubernetes/.../callbacks.py under TYPE_CHECKING.

Why

Importing airflow.providers.cncf.kubernetes.operators.pod (the base KubernetesPodOperator) transitively imports callbacks.py. That module eagerly ran import kubernetes_asyncio.client (and import kubernetes.client) at module scope, so every DAG that builds a KubernetesPodOperator at parse time paid the cost of loading the async client — which pulls in aiohttp + ~625 generated model modules (~0.2s) and is only actually needed at runtime in the triggerer for deferrable pods.

The two client imports were used only for:

  1. the module-level client_type: TypeAlias = k8s.CoreV1Api | async_k8s.CoreV1Api, and
  2. parameter annotations on the callback methods.

Since the module uses from __future__ import annotations (PEP 563), all those annotations are strings and are never evaluated at runtime — so the imports and the alias only need to exist for static type checkers.

Change

Both client imports and the client_type alias are moved into the existing if TYPE_CHECKING: block. No public behavior changes: ExecutionMode and KubernetesPodOperatorCallback still import normally; client_type has no runtime importers (only ExecutionMode / KubernetesPodOperatorCallback are imported from this module, by operators/pod.py and utils/pod_manager.py).

Verification

Loading the module standalone before/after:

kubernetes_asyncio in sys.modules after import
before True
after False (and kubernetes: False)

ExecutionMode.ASYNC and KubernetesPodOperatorCallback still resolve correctly after the change.

Note: opened against the clutch fork. Provider CI / breeze static checks (ruff, mypy) should run before merge.

…E_CHECKING

Importing airflow.providers.cncf.kubernetes.operators.pod (the base
KubernetesPodOperator) transitively imports callbacks.py, which eagerly did
`import kubernetes_asyncio.client` (and `import kubernetes.client`) at module
scope. The async client is heavy (~0.2s; pulls aiohttp + ~625 generated model
modules) and is only needed at runtime in the triggerer for deferrable pods —
yet every DAG that builds a KubernetesPodOperator paid it at parse time.

The two client imports are used only for the module-level
`client_type: TypeAlias = k8s.CoreV1Api | async_k8s.CoreV1Api` and for
parameter annotations. Because this module uses `from __future__ import
annotations` (PEP 563), all those annotations are strings and never evaluated
at runtime, so the imports and the alias only need to exist for static type
checkers. Move both client imports and the alias under TYPE_CHECKING.

Behavior is unchanged (ExecutionMode and KubernetesPodOperatorCallback still
import normally); kubernetes/kubernetes_asyncio are no longer loaded at parse.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant