Add PY_FORCE_LAZY_IMPORTS env var for deferred module loading#1
Open
logansnow17 wants to merge 7 commits into
Open
Add PY_FORCE_LAZY_IMPORTS env var for deferred module loading#1logansnow17 wants to merge 7 commits into
logansnow17 wants to merge 7 commits into
Conversation
Implements lazy imports inspired by PEP 690 and HRT's CPython fork. When PY_FORCE_LAZY_IMPORTS=true is set, top-level `import foo` statements are deferred until the module's attributes are first accessed. This can significantly reduce startup time for applications with heavy imports. - Lib/_lazy_imports.py: LazyModule proxy type and __import__ hook - Lib/site.py: activate lazy imports based on env var at startup - Lib/test/test_lazy_imports.py: 15 unit and integration tests Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Switch from __new__ to __init__ so the full_name kwarg is consumed by LazyModule rather than leaking to ModuleType.__init__. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Dotted imports (import a.b.c) must be eager because they need to actually load submodules as a side effect. Previously the hook would skip loading submodules when the top-level package was already in sys.modules, breaking code like `import email.mime.text` where internal submodule imports rely on sibling submodules being loaded. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Use __getattribute__ instead of __getattr__ so that attributes which exist on the LazyModule itself (like __dict__, __name__, __spec__) are properly delegated to the real module after resolution. This fixes ssl, enum._convert_, and anything else that inspects module.__dict__ on a lazily-imported module. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Use importlib.util.find_spec() to check that a module actually exists before creating a lazy stub. Without this, conditional imports like `try: import brotli except ImportError: ...` would silently succeed, returning a lazy stub, and only fail later when an attribute is accessed -- far from the try/except guard. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When `from sqlalchemy.engine import Engine` was called and `sqlalchemy` was a LazyModule, the bootstrap's _find_and_load_unlocked would access parent.__path__, triggering _resolve() mid-flight. The resolution would transitively load sqlalchemy.engine as a side effect, then the original _find_and_load_unlocked would resume and load it AGAIN from scratch, creating a new module object that lacked the 'base' attribute. Fix: resolve all lazy ancestors in the parent chain *before* calling _original_import, so the bootstrap never encounters a LazyModule when traversing __path__. Also use __getattribute__ instead of __getattr__ to properly proxy __dict__ and other built-in module attributes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Extends lazy imports to also defer `from X import Y` statements. When the env var is set, `from foo import bar` returns a transparent proxy that resolves on first use (call, attribute access, operators, class inheritance via __mro_entries__, etc). Key design decisions: - Only top-level from-imports are deferred; transitive imports during module loading (relative imports, resolve chains) stay eager via _import_depth tracking - __mro_entries__ delegates to the real object's implementation (e.g. Generic) so metaclass machinery works correctly - Comprehensive dunder methods on the proxy for transparent operation - Core stdlib modules excluded to avoid bootstrap issues Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements lazy imports inspired by PEP 690 and HRT's CPython fork. When PY_FORCE_LAZY_IMPORTS=true is set, top-level
import foostatements are deferred until the module's attributes are first accessed. This can significantly reduce startup time for applications with heavy imports.