Skip to content

ci: build and test local CUDA 12 bindings - #3317

Open
rwgk wants to merge 4 commits into
mainfrom
rwgk/cb12-staged-03-bindings-ci
Open

rwgk wants to merge 4 commits into
mainfrom
rwgk/cb12-staged-03-bindings-ci

Conversation

@rwgk

@rwgk rwgk commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

— codex GPT-6 on behalf of Ralf —

Layer 3 of 7 of the staged "CUDA 12 bindings on main" stack. Previous: #3315. Next: #3318. Stack map at the bottom.

Merge sequencing: approve the remaining upper stack before merging this temporary bridge, so the migration can proceed through its replacement.

What this PR does

main starts building the CUDA 12 bindings wheel itself and running its GPU test suite, next to the existing CUDA 13 pipeline. Nothing else moves yet: cuda.core and the metapackage keep consuming the 12.9.x-branch artifacts (that cord is cut in #3318).

  • build-wheel.yml: after the CUDA 13 work, switch to the maintenance mini-CTK (already fetched for Core's cu12 half), build the cuda_bindings_12 wheel with the same cibuildwheel environment (shared through a YAML anchor), twine check it, upload it, and build and upload its Cython test extensions.
  • ci.yml: three new jobs, Maintenance bindings (linux-64 / linux-aarch64 / win-64), call the existing test workflows with maintenance-bindings: true. Only matrix rows with CUDA_VER == 12.9.1 run, and only the bindings suite.
  • test-wheel-linux.yml, test-wheel-windows.yml: new inputs bindings-config and maintenance-bindings; rows gain BINDINGS_ROOT.
  • ci/tools/env-vars, ci/tools/run-tests: CUDA_BINDINGS_ROOT replaces the hard-coded cuda_bindings_13.
  • cuda_bindings_12/pyproject.toml: wheel payload policy copied from CUDA 13 (include-package-data = false: wheels ship .pxd/.pxi and headers, not .pyx sources or .in templates; sdists keep everything) and cibuildwheel settings. Recorded in the upcoming release notes.

Temporary scaffolding: the artifact name maintenance-bindings-python<py>-cuda12.9.1-<platform>-<sha> and the three jobs exist only in this layer. #3318 folds the tests into the regular jobs and #3319 renames the artifacts to the final cuda-bindings-python<py>-cuda<toolkit>-... scheme.

After this merges

  • Every CI run additionally builds one CUDA 12 bindings wheel per Python version and platform (no Windows ARM64; CUDA 12 has no toolkit there) and runs the bindings suite on the matrix rows whose toolkit is 12.9.1 (a few extra GPU jobs per platform).
  • cuda.core, the metapackage, releases, and nightly are unchanged.

Where to look

All changes (8 files, +254 / -19)
  • .github/workflows/build-wheel.yml: maintenance wheel build/check/upload, Cython test build/upload, mini-CTK gating.
  • .github/workflows/ci.yml: ci-vars exports the registry JSON; three test-maintenance-bindings-* jobs; the checks aggregator expects them.
  • .github/workflows/test-wheel-linux.yml, test-wheel-windows.yml: bindings-config / maintenance-bindings inputs, matrix filter, artifact prefix.
  • ci/tools/env-vars: MAINTENANCE_* build variables, CUDA_BINDINGS_ROOT, artifact prefix.
  • ci/tools/run-tests: bindings suite runs from CUDA_BINDINGS_ROOT.
  • cuda_bindings_12/pyproject.toml, docs/source/release/12.X.Y-notes.rst: payload policy and its note.
Validation
  • Full CI passed at the pre-rebase head afe9fc9e (run 41185002774), including the three new maintenance-bindings jobs.
  • Local validation at 9794e83edb7875a53849f562a17043cf79d88797: 316 CI-tool tests and configured workflow lint passed. Full-stack CI on #3322 passed at 126ed74fb5bbfbc91a04e50557cf936efef4da7f (105 successful jobs; one expected skip).
Stack map: what main can do after each layer merges
Layer PR After it merges, main ...
1 #3314 behaves exactly as before; CI has one reader for the two bindings roots; CUDA 13 versions can no longer match a v12.* tag
2 #3315 lets a developer build, test, and document cuda_bindings_12/; the PR-time CPU source-build smoke covers both roots; three CUDA 12 runtime fixes are in
3 #3317 builds the CUDA 12 bindings wheel itself and runs its GPU tests; cuda.core and the metapackage still consume 12.9.x artifacts
4 #3318 uses same-run CUDA 12 wheels everywhere; PR CI no longer reads the 12.9.x branch
5 #3319 can release CUDA 12 from a v12.9.N tag on main; CUDA 13 releases are unchanged
6 #3321 builds and tests only what a change needs, per bindings root, reusing baseline artifacts
7 #3322 carries the schema-2 registry, a manual-only backport workflow, a metadata consistency hook, and the maintenance docs; 12.9.x is frozen

Every layer was designed to be mergeable on its own and had a green full CI run at its pre-rebase head on 2026-10-31. The stack is the "individually mergeable pieces" alternative to #3174-#3182; both stacks reach the same end state except that this one keeps compatibility routes for older artifact sources and retained CUDA 13 tags.

@copy-pr-bot

copy-pr-bot Bot commented Oct 31, 2026

Copy link
Copy Markdown
Contributor

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@rwgk
rwgk added this pull request to stack #3316 October 31, 2026 05:13
@github-actions github-actions Bot added CI/CD CI/CD infrastructure cuda.bindings Everything related to the cuda.bindings module labels Oct 31, 2026
@rwgk rwgk added this to the cuda.bindings next milestone Oct 31, 2026
@rwgk rwgk added the enhancement Any code-related improvements label Oct 31, 2026
@rwgk rwgk self-assigned this Oct 31, 2026
@rwgk
rwgk force-pushed the rwgk/cb12-staged-03-bindings-ci branch 4 times, most recently from 97d969c to afe9fc9 Compare October 31, 2026 08:03

rwgk commented Oct 31, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test afe9fc9

@github-actions

Copy link
Copy Markdown
Contributor

@mdboom mdboom left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is a bit of a special case to call it out. It adds things that we wouldn't want permanently that subsequence steps remove, really to just "bootstrap" the wheels into the right place. It would be better if there were a way to bootstrap in a single step because this will put us on a path to merge each of these individually.

I think a better arrangement would be to have a lower-level wheel-building workflow for specific wheels that could be triggered manually. Then for bootstrapping, we kick it off directly, and otherwise it would just be used in various configurations from the existing build-wheel.yml script. But that's probably not worth it.

🔴 JUST AN IMPORTANT NOTE TO MAKE SURE THE STACK ABOVE THIS IS APPROVED BEFORE MERGING THIS SO WE DON'T GET STUCK IN THIS BAD INTERMEDIATE STATE.

Comment thread .github/workflows/build-wheel.yml Outdated
- name: Prepare local maintenance SCM milestone
if: ${{ !inputs.single-cuda-major && (env.BUILD_BINDINGS == 'true' || env.TEST_BINDINGS == 'true') }}
run: |
python -m ci.tools.bindings_scm_version \

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my previous comments in #3315 and #3316 about bindings_scm_version.

@rwgk rwgk Nov 4, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

— codex GPT-6 on behalf of Ralf —

This caller is now removed as part of the common bootstrap cleanup. The retained v12.9.11a0 milestone is on main and both alternatives are restacked onto it, as recorded in the updated #3315 reply.

Comment thread .github/workflows/ci.yml Outdated
- name: Set up Python for CI tools
uses: ./.github/actions/setup-hardened-python
with:
python-version: "3.12"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, 3.12 is an odd choice. 3.11 would be better.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

— codex GPT-6 on behalf of Ralf —

Changed this tooling job to Python 3.11. implementation.

Comment thread .github/workflows/test-wheel-linux.yml Outdated
Comment on lines +23 to +26
bindings-only:
description: Test this run's maintenance bindings wheel and Cython extensions.
type: boolean
default: false

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bindings-only is a confusing name for this variable (if the comment is correct). Maybe maintenance-bindings?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

— codex GPT-6 on behalf of Ralf —

Renamed the temporary input to maintenance-bindings in both reusable workflows and their callers. implementation; callers.

@rwgk
rwgk force-pushed the rwgk/cb12-staged-03-bindings-ci branch from 24d485d to 9794e83 Compare November 4, 2026 06:08

@rwgk rwgk left a comment •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

— codex GPT-6 on behalf of Ralf —

Thanks for calling out the intermediate-state risk in your review. The PR description now explicitly requires approval of the upper stack before this temporary bridge merges. I kept the existing bridge rather than introducing another wheel-building workflow.

All three inline comments now have direct replies. The Python/input-name changes are applied. The retained v12.9.11a0 milestone is now on main; both alternatives are restacked onto it, and the bootstrap helper, tests, metadata and callers are removed.

rwgk commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 9df7fc6

@rwgk
rwgk force-pushed the rwgk/cb12-staged-03-bindings-ci branch from 9df7fc6 to 02c3828 Compare November 6, 2026 18:02

@mdboom mdboom left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment thread .github/workflows/ci.yml
nruns: ${{ (github.event_name == 'schedule' && 5) || 1 }}
workplan: ${{ needs.detect-changes.outputs.workplan }}

test-maintenance-bindings-windows:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor nit: We don't need both x86 and arm for windows on the maintenance build today, but when we move from 12/13 to 13/14 we will need to add Windows on ARM here, so maybe add a comment?

Base automatically changed from rwgk/cb12-staged-02-developer to main November 6, 2026 19:28
@rwgk
rwgk force-pushed the rwgk/cb12-staged-03-bindings-ci branch from 02c3828 to 3c848c4 Compare November 6, 2026 19:28
rwgk added 4 commits November 7, 2026 09:42
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
@rwgk
rwgk force-pushed the rwgk/cb12-staged-03-bindings-ci branch from 3c848c4 to 8728407 Compare November 7, 2026 17:49
@copy-pr-bot

copy-pr-bot Bot commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@rwgk
rwgk marked this pull request as ready for review November 8, 2026 07:17
@coderabbitai

coderabbitai Bot commented Nov 8, 2026 •

Copy link
Copy Markdown

Explain Change →

Summary

Summary by CodeRabbit

  • New Features

    • Added maintenance-version CUDA bindings wheel builds and tests across supported Linux and Windows platforms.
  • Documentation

    • Clarified which Cython sources, declarations, templates, and headers are included in wheels and source distributions.
  • Chores

    • Updated wheel packaging settings, including Windows wheel repair and Linux build compatibility.

The CI workflows now build prior-major maintenance cuda.bindings wheels and run bindings tests on Linux and Windows. The CUDA 12 package configuration also defines wheel contents and build settings, with release notes describing the packaged files.

Changes: Maintenance cuda.bindings pipeline

Layer / File(s) Summary
CUDA 12 wheel packaging
cuda_bindings_12/pyproject.toml, cuda_bindings_12/docs/source/release/12.X.Y-notes.rst
The package configuration specifies included Cython declarations and headers, wheel platforms, and Windows repair. The release notes describe wheel and source distribution contents.
Maintenance wheel build and Cython artifacts
.github/workflows/build-wheel.yml, ci/tools/env-vars
When prior-major builds are enabled, the workflow builds and checks a maintenance wheel for bindings builds or tests. For bindings tests, it installs that wheel, builds Cython tests, and uploads the artifacts. The environment script resolves maintenance package paths and artifact names.
Maintenance test matrix and environment
.github/workflows/test-wheel-linux.yml, .github/workflows/test-wheel-windows.yml, ci/tools/env-vars, ci/tools/run-tests
Maintenance mode selects package roots by toolkit version and runs bindings tests only. The test environment and runner use the selected package root and maintenance artifact prefix.
CI scheduling and result checks
.github/workflows/ci.yml
The CI workflow obtains the bindings configuration, schedules maintenance test jobs for Linux x86-64, Linux aarch64, and Windows, and checks their expected results.

Suggested reviewers: leofang, juenglin

Change: Other

Merge Risk: Minimal · up to 87284

The CUDA 12 wheel and maintenance-test paths show no established merge-blocking regression. The differing cibuildwheel pin can be considered during normal CI validation.

Security architecture risk: Minimal · up to 87284

The maintenance pipeline reuses existing CI permissions and runners, separates its artifacts from release inputs, and requires successful maintenance tests when selected. No material security risk introduced or worsened by this change was identified.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The added execution surface is repository build code and maintenance test binaries running within existing CI runner and credential contexts. The inspected changes add executions but not a new runner class, token permission scope, external publication destination, or production service boundary.

Trust Boundaries and Controls

  • observed — New maintenance callers retain actions-read and contents-read permissions, use the existing inherited-secret convention, and require the repository-owner and workplan gates. Build containers receive the existing cache runtime credentials through the shared bindings environment; the cache setup condition is extended to bindings-test-only builds.

Resilience and Maintainability Implications

  • observed — Maintenance matrix selection fails on an empty result rather than silently selecting another toolkit. Artifact download retries preserve the requested identity and propagate the final failure instead of falling back to another run or package source.
- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional comments: 1

.github/workflows/build-wheel.yml: 1

563-563: Trivial · Maintainability & Code Quality · Low value

Unverified finding
Verification ran but could not confirm this finding. It is shown for review, not as a verified issue.

suggestion: This step pins pypa/cibuildwheel at v4.3.4 (b9e566a2). The other cibuildwheel steps in this file pin v4.3.5 (f9b966cf). Use the same pin for all steps so the CUDA 12 and CUDA 13 wheels build with one tool version. If the older pin is intentional, add a comment that says why.

Review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between db463bf and 8728407.

Files selected for processing: 8

  • .github/workflows/build-wheel.yml
  • .github/workflows/ci.yml
  • .github/workflows/test-wheel-linux.yml
  • .github/workflows/test-wheel-windows.yml
  • ci/tools/env-vars
  • ci/tools/run-tests
  • cuda_bindings_12/docs/source/release/12.X.Y-notes.rst
  • cuda_bindings_12/pyproject.toml

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 8 remain after this review.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI/CD CI/CD infrastructure cuda.bindings Everything related to the cuda.bindings module enhancement Any code-related improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants