Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions .claude/rules/project-layout.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,17 @@ How this repo organizes Python packages, the build system, and example / test di

| Package | Source | What's in wheel | Use for |
| ------- | ------ | --------------- | ------- |
| `simpler` | `python/simpler/` | `task_interface`, `worker`, `env_manager` only | Stable user API at runtime |
| `simpler_setup` | `simpler_setup/` | All files + `_assets/{src,build/lib}` | Test framework, compilers, path resolution |
| `simpler` | `python/simpler/` | `task_interface`, `worker`, `env_manager` only | Runtime user API: `Worker` plus the task/callable types |
| `simpler_setup` | `simpler_setup/` | All files + `_assets/{src,build/lib}` | **Also user-facing**, not internal-only: `KernelCompiler`, `SceneTestCase` / `scene_test`, `Tensor`, `Scalar`, `TaskArgsBuilder`, `ensure_pto_isa_root`, `make_tensor_arg`, plus the `simpler_setup.tools` CLIs. A kernel cannot be compiled without it |
| `_task_interface` | `python/bindings/` | nanobind `.so` at wheel root | Internal nanobind module |

`simpler` exposes `Worker` and the `task_interface` submodule lazily (PEP 562
`__getattr__`), so `from simpler import Worker` works while `import simpler`
still costs nothing and does not require the `_task_interface` extension. Note
that `simpler.task_interface.Tensor` (a device tensor descriptor) and
`simpler_setup.Tensor` (a scene-test arg spec `NamedTuple`) are **different
types with the same name** — do not flatten them into one namespace.

The 4 files `kernel_compiler.py`, `runtime_compiler.py`, `toolchain.py`, `elf_parser.py` exist in **both** `python/simpler/` and `simpler_setup/` during transition. The `simpler_setup/` copies are authoritative; the `python/simpler/` copies are excluded from wheel via `pyproject.toml::wheel.exclude`. New code must `import` from `simpler_setup.*`, not `simpler.*`, for these four.

## Build System Lookup
Expand Down
12 changes: 5 additions & 7 deletions docs/user/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,13 +36,11 @@ Both `simpler` and `simpler_setup` are part of the surface you write against:
`simpler_setup.tools`.

You cannot compile a kernel without `simpler_setup`, so treat both as yours to
use. Note that
[`.claude/rules/project-layout.md`](../../.claude/rules/project-layout.md)
describes `simpler_setup` as internal test framework — that boundary does not
match how the examples import today, and neither does the claim that `simpler`
is the whole user API: `simpler.__all__` currently exports only the logging
helpers, so `Worker` must be imported as `from simpler.worker import Worker`
rather than `from simpler import Worker`.
use. `from simpler import Worker` works; the task and callable types come from
`simpler.task_interface`. One name to watch:
`simpler.task_interface.Tensor` is a device tensor descriptor while
`simpler_setup.Tensor` is a scene-test argument spec — same name, different
types.

## Where the worked examples live

Expand Down
8 changes: 5 additions & 3 deletions docs/user/reference/python-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,13 @@ The surface you write against, hand-maintained. There is no generated API
reference in this repo, so treat the source as authoritative when the two
disagree and fix this page in the same change.

Import paths matter: `simpler.__all__` currently exports only the logging
helpers, so the runtime types are imported from their submodules.
`Worker` is available from the package root; the remaining task and callable
types live in `simpler.task_interface`. Both resolve on first access, so
`import simpler` alone stays cheap and does not require the `_task_interface`
extension.

```python
from simpler.worker import Worker
from simpler import Worker # or: from simpler.worker import Worker
from simpler.task_interface import (
ArgDirection, CallConfig, ChipCallable, ChipStorageTaskArgs,
CoreCallable, DataType, Tensor,
Expand Down
28 changes: 28 additions & 0 deletions python/simpler/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,16 @@
`simpler_init` reads CANN dlog config off that same HostLogger, onboard only).
The level forwarded is a one-shot snapshot of the `simpler` Python logger.
Nothing log-related needs to happen at import time here.

`Worker` and the `task_interface` submodule resolve on first attribute access
rather than at import time: both pull in the `_task_interface` extension, so
eager imports here would make `import simpler` fail wherever that extension is
missing or stale, including for callers that only want the logging helpers.
"""

import importlib
from typing import Any

# Importing _log auto-configures the simpler logger to V5 if unset.
from ._log import (
DEFAULT_THRESHOLD,
Expand All @@ -37,6 +45,7 @@

__all__ = [
"DEFAULT_THRESHOLD",
"Worker",
"NUL",
"V0",
"V1",
Expand All @@ -50,4 +59,23 @@
"V9",
"get_current_config",
"get_logger",
"task_interface",
]

# name -> (module, attribute). Resolved by __getattr__ on first access.
_LAZY_ATTRS = {"Worker": (f"{__name__}.worker", "Worker")}
_LAZY_SUBMODULES = ("task_interface",)


def __getattr__(name: str) -> Any:
"""Resolve the extension-backed surface on first access (PEP 562)."""
if name in _LAZY_ATTRS:
module_name, attribute = _LAZY_ATTRS[name]
return getattr(importlib.import_module(module_name), attribute)
if name in _LAZY_SUBMODULES:
return importlib.import_module(f"{__name__}.{name}")
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")


def __dir__() -> list[str]:
return sorted(__all__)
77 changes: 77 additions & 0 deletions tests/ut/py/test_package_surface.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
# Copyright (c) PyPTO Contributors.
# This program is free software, you can redistribute it and/or modify it under the terms and conditions of
# CANN Open Software License Agreement Version 2.0 (the "License").
# Please refer to the License for details. You may not use this file except in compliance with the License.
# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED,
# INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.
# See LICENSE in the root of the software repository for the full text of the License.
# -----------------------------------------------------------------------------------------------------------
"""Tests for the `simpler` package's public attribute surface."""

import importlib
import subprocess
import sys

import pytest
import simpler


def test_worker_and_task_interface_are_advertised():
assert "Worker" in simpler.__all__
assert "task_interface" in simpler.__all__
assert "Worker" in dir(simpler)
assert "task_interface" in dir(simpler)


def test_logging_helpers_remain_exported():
for name in ("get_logger", "get_current_config", "DEFAULT_THRESHOLD", "NUL"):
assert name in simpler.__all__
assert hasattr(simpler, name)


def test_unknown_attribute_raises_attribute_error():
with pytest.raises(AttributeError, match="has no attribute"):
getattr(simpler, "definitely_not_an_attribute") # noqa: B009 -- exercises __getattr__


def test_importing_simpler_does_not_load_the_worker_module():
"""`import simpler` must not pull in `simpler.worker`.

`worker` imports `_task_interface` and `cloudpickle` unguarded, so resolving
it eagerly here would make `import simpler` fail outright whenever the
extension is missing or stale — including for callers that only want the
logging helpers, which `_log` deliberately keeps working via a fallback.
A subprocess is used because this test session has already imported it.

Note `_task_interface` itself is a weaker signal: `_log` probes it inside a
try/except, so it legitimately appears in `sys.modules` when it is present.
"""
code = "import simpler, sys; print('simpler.worker' in sys.modules)"
out = subprocess.run( # noqa: S603 -- fixed argv, no shell
[sys.executable, "-c", code], capture_output=True, text=True, check=True
)
assert out.stdout.strip() == "False", f"import simpler eagerly loaded simpler.worker: {out.stdout!r} {out.stderr!r}"


def test_importing_simpler_survives_without_the_extension():
"""The logging surface must import even when `_task_interface` is absent."""
code = "import sys; sys.modules['_task_interface'] = None; import simpler; print(callable(simpler.get_logger))"
out = subprocess.run( # noqa: S603 -- fixed argv, no shell
[sys.executable, "-c", code], capture_output=True, text=True, check=True
)
assert out.stdout.strip() == "True", f"{out.stdout!r} {out.stderr!r}"


def test_worker_resolves_to_the_same_object_as_the_submodule():
# Not importorskip: a stale extension raises plain ImportError rather than
# ModuleNotFoundError, which importorskip is deprecating for pytest 9.1.
# Only an unusable _task_interface is a skip; `worker` also imports
# cloudpickle, and losing that must fail rather than quietly skip.
try:
worker_module = importlib.import_module("simpler.worker")
except ImportError as exc:
if exc.name != "_task_interface":
raise
pytest.skip(f"needs a current nanobind extension: {exc}")
Comment thread
coderabbitai[bot] marked this conversation as resolved.

assert simpler.Worker is worker_module.Worker
Loading