Conversation
|
|
|
Please dont spam like this lol 🙏 |
thesayyn
force-pushed
the
claude/review-pr-942-gZbpc
branch
from
0fa9da8 to
46ea075
Compare
|
thesayyn
force-pushed
the
claude/review-pr-942-gZbpc
branch
from
276dde4 to
5f158ff
Compare
thesayyn
changed the title
Rewrite py_image_layer with analysis-time grouped OCI layers and globally-shared pip tars via two propagated aspects: - _layer_aspect propagates through deps/data/actual; pip per-package tars are declared at the pip target's namespace so they action-share across every rule using that package. First-party PyInfo targets matched by layer_tier.groups are captured for per-group tarring at the binary. - _merge_aspect runs only on the binary, reads pip_packages closure, buckets install_dirs by merge_group, and emits one bsdtar action per group from raw install_dirs (single pass, no intermediates). A new layer_tier rule carries grouping + compression as LayerTierInfo; aspects read it via a //py:layer_tier label_flag so users can switch tiers globally with --//py:layer_tier=//path:custom_tier or pin one per rule via the layer_tier attr. A Python validation action surfaces layer_tier(groups=...) suggestions when the squashed pip layer or individual packages exceed thresholds, or the OCI 127-layer limit is breached. Sharing model: - Solo whole-group + subpath-split pip tars: action-shared across every rule using that package (declared at the pip target's namespace). - Multi-member merged tars: per-binary action, deterministic content (canonical mtree, fixed bsdtar options) so remote CAS / OCI registry dedupes by digest. - First-party grouped tars: per-binary, one tar per group, collected from matched py_library targets in the binary's dep closure. - Ungrouped pip packages: squashed by the rule into one per-rule tar. - Optional dedicated interpreter layer when layer_tier.interpreter_group is set; the tar is declared at the py toolchain target's namespace so it action-shares across every py_image_layer using that toolchain config. Other: - normalize_label tolerates both Bazel 8 (`+`) and Bazel 9 (`~`) module-extension separators by anchoring on the stable `whl_install__` substring instead of the full prefix. - Pull runfiles.repo_mapping_manifest into the binary's source layer and special-case its short_path so it lands at `./app.runfiles/_repo_mapping`. - The validator binary uses rules_py's own py_binary rule (no @rules_python load needed at the BUILD level). - Two container_structure_tests that exec the launcher are gated off on Bazel 9 — Bazel 9 stages bzlmod-canonical repo names in the runfiles tree, so the runfiles.bash bootstrap can't find `bazel_tools/tools/bash/runfiles/runfiles.bash` by apparent name. Same gate the existing listing tests use via `bazel_features.rules.merkle_cache_v2`.
thesayyn
force-pushed
the
claude/review-pr-942-gZbpc
branch
from
8128424 to
2dca728
Compare
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2dca728046
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| return [_LayerInfo( | ||
| source_files = depset(transitive = transitive_source), | ||
| pip_packages = depset( |
There was a problem hiding this comment.
When a target has OutputGroupInfo.install_dir, it is already represented as a pip package layer via pip_packages, but this return still propagates transitive_source from its deps into the binary's default source layer. For pip targets whose internal deps expose the install tree, the same wheel files end up in both the dedicated pip tar and the default tar (the updated my_app_layers_listing.yaml shows the colorama install path in layer 0 and again in the default layer), which defeats the layer split and bloats images. These install-dir targets should stop forwarding source files, or otherwise filter those files before they reach info.source_files.
Useful? React with 👍 / 👎.
3 participants
Optimized py_image_layer creation with maximal action sharing
Changes are visible to end-users: yes
New public api of py_image_layer is quite different than the previous one.
Test plan