Repository navigation
Conversation
…ge tool Before this change the coverage collection was only exercised by running `bazel coverage` on a few targets in CI and the collected data was not checked. This adds a test that runs as part of `bazel test`. A `py_reconfig_test` is built with `--collect_code_coverage` for a Python 3.12 toolchain that has the bundled coverage tool configured, and `coverage_collection_test` runs it with the same environment variables that `bazel coverage` sets and checks the line coverage in the produced lcov file. * Adds the `collect_code_coverage` attribute to `py_reconfig_*` rules. * Configures the bundled coverage tool for the Python 3.12 dev toolchain. Stacked on bazel-contrib#4415. Work towards bazel-contrib#3902
Registering the toolchain with the `3.12` minor version made it take precedence over the `3.12.16` toolchain that //tests/toolchains expects.
…ests Run the instrumented program once and check the line and branch coverage of particular lines in separate tests.
Allow py_runtime.coverage_tool to be a py_library of coverage.py, such as @pypi//coverage, by using its coverage/__main__.py file as the entry point. This allows users to use the coverage version that is locked in their requirements.
A py_library depends on the Python toolchain, so using @pypi//coverage as the coverage_tool of the toolchain creates a dependency cycle. Instead find the coverage/__main__.py entry point in the files of the target, so that @pypi//coverage:extracted_whl_files can be used.
… into aignas.feat.coverage-tool-py-library
It is used to test the coverage collection with the coverage tool from a PyPI hub.
…p hub Configure a 3.13 toolchain with the extracted coverage wheel from the dev_pip hub as the coverage tool and run the same coverage collection checks as for the bundled coverage tool.
Use TEST_TMPDIR directly and rely on the test working directory being the workspace runfiles directory.
Also drop the lib.py data dependency, which is already in the runfiles via the instrumented target.
… into aignas.feat.coverage-tool-py-library
Also test the error for a coverage_tool without coverage/__main__.py and name the news entry after the PR.
…age-tool-py-library
…e tool Using a `py_library`, such as `@pypi//coverage`, as the `coverage_tool` created a dependency cycle: the `py_library` resolves the Python toolchain, whose `py_runtime` depends on the `coverage_tool` again. Similarly to how `rules_go` builds `nogo`, the `coverage_tool` attribute now transitions to a configuration where a private bootstrap flag is set and `collect_code_coverage` is disabled. The `py_runtime` macro drops the `coverage_tool` dependency in that configuration, so the cycle is broken. The hermetic toolchains already select on `collect_code_coverage`.
The bootstrap never executes the coverage tool file: it only adds the grandparent directory of it to `sys.path` and imports `coverage`. Instead of searching for `coverage/__main__.py` within the files of the target, use the import paths of a `coverage_tool` that provides `PyInfo`, e.g. `@pypi//coverage`. * Adds `PyRuntimeInfo.coverage_imports`, which is mutually exclusive with `PyRuntimeInfo.coverage_tool`. * Adds the `%coverage_imports%` substitution to the site init template.
Allow selecting all of the platforms configured with `pip.default` without listing them one by one. This is useful for hubs that need to support cross-building for any of the configured platforms, e.g. a hub that provides a tool for every toolchain.
…coverage-platform-whls
…rage tool Define the `coverage` dependency in a separate `pyproject.toml` with a `uv.lock` file, which locks the latest `coverage` version for each Python version starting from 3.9. The lock file is used by the new `rules_python_coverage` pip hub, which has the wheels for all of the platforms configured with `pip.default`. With bzlmod, `python.toolchain(configure_coverage_tool = True)` now uses `@rules_python_coverage//coverage` as the coverage tool. WORKSPACE still uses the bundled coverage wheels.
Document the `rules_python_coverage` hub and recommend passing a `py_library` as the coverage tool. Using a target without `PyInfo` will be deprecated in the next version.
…s = ["*"] When `target_platforms = ["*"]` is used with a `uv.lock` file, add the platform independent wheel of a package, if there is one, as a fallback for the Python version. The platforms that are not configured with `pip.default` can then still use the package, whilst the configured platforms keep using the more specific wheels.
The default `pip.default` platforms only covered the `x86_64` and `aarch64` Linux, macOS and Windows platforms, so the toolchains for the other platforms could not use the wheels from a pip hub. Add the default platforms for all of the platforms that the hermetic Python toolchains support and a test that ensures that each toolchain platform variant has a matching pip platform. The glibc Linux platforms now also match on `--py_linux_libc`, so that they do not overlap with the new musl platform, which uses the `musllinux` wheels.
…coverage-platform-whls
…erage-platform-whls
…platforms = ["*"]" This reverts commit 5149b91.
…coverage-platform-whls
…coverage-platform-whls
…erage-platform-whls
…coverage-platform-whls
…uv.lock The WORKSPACE macros cannot read the coverage `uv.lock` file, so add a dev-only repository rule, which uses the same wheel selection as the `pip.parse` extension to write the wheels for each Python version and platform to a checked-in `workspace_deps.bzl` file. The file is generated with bzlmod only and a `diff_test` ensures that it is in sync with the `uv.lock` file. Run `bazel run //dev/update_deps:update_coverage_workspace_deps` to update it. This replaces the `update_coverage_deps` script, which maintained the wheel list by hand.
… toolchains With WORKSPACE, `python_register_toolchains(register_coverage_tool = True)` now creates the `@rules_python_coverage` hub from the checked-in `workspace_deps.bzl` file, so the WORKSPACE toolchains use the same coverage wheels as bzlmod without any changes to the `WORKSPACE` file. The WORKSPACE builds do not set the Python version flag by default, so the hub has a coverage tool for each Python version, which only selects the wheel based on the target platform, e.g. `@rules_python_coverage//coverage:3.13`. The `coverage_deps.bzl` file is removed and the coverage collection test now also runs with WORKSPACE.
Collaborator
Author
|
This PR was drafted by an AI coding agent (Claude Code) working for @aignas as part of the plan in #3902. It stays a draft until @aignas has reviewed it. Review comments will be handled by @aignas, possibly with the help of the agent; any agent-written replies will be marked as such. Agent-generated comment, posted automatically on behalf of the user. |
4 of 11 tasks
…olchains The `python/repositories.bzl` file is loaded before `py_repositories` defines the `rules_python_internal` repository, which `whl_library` depends on, so loading it created a cycle in the WORKSPACE file. Use `http_archive` with a `py_library` BUILD file for the coverage wheels instead, which only depends on `bazel_tools`.
WORKSPACE macros cannot read files, so add a `vendor_uv_lock` macro that vendors a `uv.lock` file as a JSON string constant in a `.bzl` file. Only the fields that are needed for selecting the distributions are kept and each package and distribution is on its own line, so the diffs are small. The macro takes the name of the `lock` target that updates the `uv.lock` file and creates `.update` and `.run` targets, which run the same targets of the `lock` first and then vendor the updated `uv.lock`, so a single command updates both files. The `diff_test` only depends on the `uv.lock` file and does not need the network.
…ndored uv.lock Replace the dev-only repository rule, which generated the wheel table, with the `vendor_uv_lock` macro. The WORKSPACE toolchains now decode the vendored `uv.lock` and select the wheels with the same code as `pip.parse` in a pure `select_wheels` function, which has unit tests. The `coverage_repositories` macro gets its params from the vendored JSON constant and only selects the wheels once, as it is called for each toolchain platform. `bazel run //python/private/coverage:uv_lock_bzl.update` now updates both the `uv.lock` and the vendored file, and it is part of `bazel run //dev:all_requirements.update`.
…hon_internal The `patch_whl.bzl` and `whl_extract.bzl` files loaded the `@rules_python_internal` repository only to get the `supports_whl_extraction` and `extract_needs_chmod` values, which only depend on the Bazel version. Compute them from `native.bazel_version` in a new `extract_features` function, which `internal_config_repo` also uses. This allows loading `whl_library` from the files that are loaded before `py_repositories` defines `@rules_python_internal` in WORKSPACE, e.g. `python/repositories.bzl`.
…coverage-hub-workspace
The `pypi_repo_utils.bzl` and `util.bzl` files only used `types.is_string` and `types.is_tuple` from `@bazel_skylib`, which are simple `type()` checks. Without them, `whl_library` can be loaded from the files that are loaded before the `WORKSPACE` defines `@bazel_skylib`, e.g. `python/repositories.bzl`.
Replace the private `extract_features` function with a public `bazel_features(bazel_version)` function in `//python:features.bzl`, which returns the features of a Bazel version, e.g. `bazel_features(native.bazel_version).supports_whl_extraction`. The Bazel version is only available when a repository rule or a module extension runs, so this is a function and not fields of the `features` struct. The values are no longer written to the `@rules_python_internal` config, as nothing reads them from there anymore.
…coverage-hub-workspace
Now that `whl_library` can be loaded from `python/repositories.bzl` (bazel-contrib#4458), use it for the coverage wheels instead of a hand-crafted `http_archive`, so the wheels are extracted the same way as for the pip hubs.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
With WORKSPACE,
python_register_toolchains(register_coverage_tool = True)now uses the same
coveragewheels as bzlmod, which are locked in thepython/private/coverage/uv.lockfile, without any changes to the users'WORKSPACEfiles.The WORKSPACE macros cannot read the
uv.lockfile and apip_parsebasedhub would require users to add
loadandinstall_deps()calls to theirWORKSPACEfile. Instead, this adds avendor_uv_lockmacro in//python/uv:vendor_uv_lock.bzl, which vendors auv.lockfile as a JSONstring constant in a
.bzlfile:and each package and distribution is on its own line, so the diffs are
small. The vendored file is smaller than the
uv.lockitself.locktarget, e.g.lock(name = "requirements"),and creates
.updateand.runtargets, which run the same targets of thelockfirst and then vendor the updateduv.lock. A singlebazel run //python/private/coverage:uv_lock_bzl.updateupdates both files,and it is part of
bazel run //dev:all_requirements.update.diff_testonly depends on theuv.lockfile and does not need thenetwork.
python_register_toolchainscalls thecoverage_repositoriesmacro with thedecoded vendored constant. It selects the wheels with the same
parse_requirementscode aspip.parsein a pureselect_wheelsfunction,which has unit tests, and creates a
whl_libraryfor each wheel and the@rules_python_coveragehub. The WORKSPACE builds do not set thePython version flag by default, so the hub has a coverage tool for each
Python version, which only selects the wheel based on the target platform,
e.g.
@rules_python_coverage//coverage:3.13. The repositories are fetchedlazily, so only the wheels that are used are downloaded.
This replaces the hand-maintained
coverage_deps.bzland its update script.The
WORKSPACEofrules_pythonnow enables the coverage tool, so that thecoverage collection test also runs with WORKSPACE.
Stacked on #4415 and #4458.
Fixes #3902