Build and test conda packages and wheels by jharlow-intel · Pull Request #400 · IntelPython/mkl_fft

jharlow-intel · 2026-10-23T20:23:35Z

Adds ci/ and .github/workflows/intel-ci.yml, which build the conda packages for Linux, Windows and Intel macOS and the wheels for Linux and Windows, on every declared Python, and test each in a clean environment.

conda-recipe/ is now the only recipe: the conda-forge one that was conda-recipe-cf/. conda-package.yml and conda-package-cf.yml are removed, as intel-ci.yml builds and tests the recipe.
The conda packages build Python 3.10 to 3.15 plus 3.14t, from the python: list in conda_build_config.yaml, which also sets numpy: '2', as conda-build otherwise falls back to NumPy 1.26. The recipe requires numpy >=1.26.4, as pyproject.toml does, and keeps numpy's run_exports for the upper bound. The wheels follow the pyproject.toml classifiers: 3.10 to 3.15 plus 3.14t and 3.15t.
Wheels build with cibuildwheel, with MSVC on Windows, leave MKL out and require mkl from the mkl-devel version they were built with up to its next major, through [tool.gh-ci] pin-compatible, which ci/build.py adds to each wheel. The Linux wheels build with conda-forge's hardening flags, and ci/build.py refuses a wheel without full RELRO or with a library search path outside $ORIGIN.
meson.build adds Library to CMake's search path on Windows, where mkl-devel puts its CMake config.
ci/test.py runs pytest --pyargs mkl_fft on the installed package. The SciPy interface tests skip without SciPy or mkl-service, as on the 3.15 wheels, which get no mkl-service from PyPI yet. ci/test.py fails when both are installed but the SciPy interface does not import.
Each test job runs for every package that built, even when another build failed. A pull request builds the oldest and newest Python plus the newest free-threaded one; master and tags build them all.
conda-build records the source commit in each package's info/about.json.
ci/benchmark.py runs the asv suite in benchmarks/ against the installed package: one quick sample of each benchmark on a pull request, a full measurement on a tag.
The Agent guidance job, ci/guidance.py, fails a pull request that adds a broken link, path, script or identifier claim to an agent guidance file it changes. dokimi is installed with the hashes in ci/guidance-requirements.txt.

Miniforge is pinned to 26.7.2-0, as 26.9.4-0 makes setup-miniconda's conda init fail on Windows runners.

Overlap with #398. It changes conda-recipe-cf/ and conda-package-cf.yml, which this moves to conda-recipe/ and removes, so whichever lands second needs a rebase.

Validation: run 42669864447 on 837aff0 builds and tests the reduced pull request matrix on Linux, Windows and macOS. Not run yet: the full matrix, which only master and tags build.

Copilot

🔵 Needs a closer look

Full-version builds and the external Jenkins handoff remain unverified and need maintainer validation.

1 open issue

Stop validating Python selectors after the list ends · New

What changed in this PR

Adds GitHub Actions build and test support for mkl_fft’s conda-forge packages, preparing the repository for Intel Python CI’s future Jenkins integration.

Changes:

Derives Linux and Windows build matrices from recipe Python variants, limiting pull requests to the two newest regular versions.
Adds package-building and isolated testing scripts, with pytest JUnit reporting.
Configures artifact naming, pins Miniforge, and documents the CI integration.

File	Description
`conda-recipe-cf/conda_build_config.yaml`	Declares Python variants and NumPy 2.
`conda-recipe-cf/AGENTS.md`	Documents the recipe’s matrix role.
`ci/test.py`	Tests the installed package and writes JUnit results.
`ci/README.md`	Explains CI operation and local usage.
`ci/matrix.py`	Derives platform and Python build matrices.
`ci/hooks.py`	Installs artifacts into isolated test environments.
`ci/config.yml`	Defines workflow, artifact names, and test requirements.
`ci/build.py`	Builds and collects packages for a selected Python version.
`AGENTS.md`	Adds the Intel Python CI directory reference.
`.github/workflows/intel-ci.yml`	Orchestrates matrix generation, builds, and package tests.
`.github/AGENTS.md`	Documents the workflow and its matrix source.

_{Review effort: Balanced}

vchamarthi

Scripts look good and match the other repos, and the 22 jobs are green.

The PR description is out of date though. It says the workflow builds conda-recipe-cf (it's deleted now, and conda-recipe/ is replaced by the cf recipe), Linux and Windows only (macOS and wheels are built too), and that Jenkins isn't on for mkl_fft (it's enabled in config/gh-ci.json, dry run). The test counts in it are old as well, the head run shows 2793 and 2787.

It also doesn't say conda-package.yml is deleted. That workflow was the only one building and testing against the Intel channel (software.repos.intel.com/python/conda), which is what the shipped Intel package links against, and nothing replaces it. Rest is inline.

vchamarthi · 2026-11-04T15:16:28Z


+def require_scipy_fft():
+    """Skips the test unless mkl_fft has its SciPy interface."""
+    if scipy_fft is None:


When import mkl fails, the scipy-interface tests now skip instead of failing. So a conda cell where mkl-service is required in ci/config.yml but imports badly still goes green with about 240 fewer tests. You can already see it in the 3.15 wheel cells: 2551 passed and 54 skipped, against 2793 and 2 on Linux. Could ci/test.py check that the scipy interface loaded when mkl-service is in the requirements? Same pattern in test_basic.py and test_multithreading.py.

Good catch. In 837aff0 ci/test.py fails before pytest runs when SciPy and mkl-service are both installed but mkl_fft.interfaces.scipy_fft doesn't import, so a broken mkl-service import can't turn into skips. The skips stay for cells without mkl-service, like the 3.15 wheels, as PyPI has no mkl-service for 3.15 yet.

vchamarthi · 2026-11-04T15:16:29Z

+try:
+    import mkl_fft.interfaces.scipy_fft as fft
+except ImportError:
+    pytest.skip("This test file needs mkl-service", allow_module_level=True)


Same as in test_interfaces.py: this skip hides a broken mkl-service import.

The same ci/test.py check covers this one (837aff0).

vchamarthi · 2026-11-04T15:16:29Z

    import mkl_fft.interfaces.scipy_fft as fft
 except ImportError:
-    pytest.skip("This test file needs scipy", allow_module_level=True)
+    pytest.skip(


Same again.

Covered by the same check (837aff0).

vchamarthi · 2026-11-04T15:16:29Z

      - python
      - {{ pin_compatible('mkl') }}
-      - numpy >=1.26.4
+      - numpy


Run requirement goes from numpy >=1.26.4 to a bare numpy, with ignore_run_exports: numpy. conda can now pair this with a numpy 1.x older than the numpy>=1.26.4 in pyproject.toml. Could we use numpy's run_exports instead of ignoring them?

Agreed. In 837aff0 the recipe keeps numpy's run_exports, as the conda-forge feedstock does, and requires numpy >=1.26.4, as pyproject.toml does, so the package gets that floor plus numpy's upper bound.

vchamarthi · 2026-11-04T15:16:29Z

+
+  test:
+    name: Test ${{ matrix.name }} Python ${{ matrix.python }}
+    needs: [matrix, build]


If any build leg fails, macOS included, all the test legs are skipped and the run fails. Jenkins needs a green run, so a macOS-only failure would block a tag release even though Jenkins ignores macOS. Intended? Same on test-wheel at line 175.

Yes, a package that fails to build on any platform, macOS included, fails the run. Skipping every test leg wasn't intended, though: in 837aff0 the test jobs use if: ${{ !cancelled() && needs.matrix.result == 'success' }}, as mkl-service does, so every package that built still gets tested and only the failed leg's test fails. Same for test-wheel.

vchamarthi · 2026-11-04T15:16:29Z

+  # macOS runs on Intel runners, as MKL has no Apple silicon build; Jenkins ignores it.
+  build:
+    name: ${{ matrix.name }} Python ${{ matrix.python }}
+    needs: matrix


conda-package-cf.yml and this workflow now both build conda-recipe on every PR, so the runner load is doubled.

Agreed, it was redundant. 837aff0 removes conda-package-cf.yml; intel-ci.yml builds and tests the recipe.

vchamarthi · 2026-11-04T15:16:29Z

+"""
+Intel Python CI (gh-ci) test for mkl_fft.
+
+GitHub Actions, through ci/hooks.py, and Intel's Jenkins both run this with one


This only talks about a conda package, but it tests the wheels too.

Fixed: the docstring says it runs with a conda package or a wheel.

Adds a ci/ directory and .github/workflows/intel-ci.yml. The workflow builds conda-recipe/ with conda-build and the wheels with cibuildwheel, for every platform and Python version the recipe and pyproject.toml declare, and tests each package in a clean environment with ci/test.py. conda-recipe/ is now the only recipe, the conda-forge one that was conda-recipe-cf/; the Intel-channel recipe and conda-package.yml, which copied wheels out through WHEELS_OUTPUT_FOLDER, are removed. conda-package-cf.yml reports the same check names, so the required checks still run. conda_build_config.yaml now lists the Python versions, and numpy: '2', as conda-build otherwise defaults to 1.26. The Agent guidance job, ci/guidance.py, fails a pull request that adds a broken link, path, script or identifier claim to an agent guidance file it changes. Claims that already fail, and untouched files, never fail it. dokimi is installed with the hashes in ci/guidance-requirements.txt. [build-system] requires mkl-devel, and [tool.gh-ci] pin-compatible maps mkl to it: ci/build.py adds Requires-Dist mkl >= the mkl-devel version python -m build installed, < its next major, the way conda's run_exports pin it, and fails a build that logs no version. It takes METADATA's line ending from the headers. The repair commands exclude MKL's libraries by wildcard, and the Linux wheels are tagged manylinux_2_28 only, like the mkl wheel. The Linux wheels build with the hardening flags conda-forge's compilers set: -fstack-protector-strong, -D_FORTIFY_SOURCE=2 and -Wl,-z,relro -Wl,-z,now. ci/build.py refuses a wheel whose ELF files lack full RELRO. It keeps an RPATH or RUNPATH entry only when it is $ORIGIN or a directory under it, as meson-python keeps the isolated build env's absolute lib directory, and refuses a file left with none. It rewrites the wheels in scratch, so a refused wheel never reaches --output, and refuses two built packages with the same file name, which would overwrite each other there. conda-build gets --extra-meta remote_url=... sha=... when GitHub Actions sets GITHUB_SERVER_URL, GITHUB_REPOSITORY and GITHUB_SHA, so each conda package records the commit it was built from in info/about.json. ci/hooks.py tests each wheel in a venv made from conda-forge's Python, as pip users install it. Every python: entry in conda_build_config.yaml names its ABI, which ci/matrix.py requires. ci/matrix.py reads x86 as any x86 CPU and osx64 as undefined, as conda-build does, and refuses a conda-build skip that uses osx64. A pull request also builds and tests the newest free-threaded version, and every push and tag run gets a concurrency group of its own.

…iPy interface The test jobs run for each package that built, even when another build failed. The recipe keeps numpy's run_exports and requires numpy >=1.26.4, as pyproject.toml does. ci/test.py fails when SciPy and mkl-service are installed but mkl_fft.interfaces.scipy_fft does not import, as its tests would otherwise skip. conda-package-cf.yml is removed, as intel-ci.yml builds and tests the same recipe.

jharlow-intel · 2026-11-09T17:59:37Z

Thanks for going through it. The description is updated: it covers macOS, the wheels, the current pins and the test-job change, and links a run on the current head.

jharlow-intel marked this pull request as ready for review October 23, 2026 20:42

jharlow-intel requested review from antonwolfy, ndgrigorian, vlad-perevezentsev and xaleryb as code owners October 23, 2026 20:42

Copilot AI balanced review requested due to automatic review settings October 23, 2026 20:42

Copilot started reviewing on behalf of jharlow-intel October 23, 2026 20:43

View session

Copilot AI reviewed Oct 23, 2026

View reviewed changes

Comment thread ci/matrix.py Resolved

jharlow-intel self-assigned this Oct 28, 2026

jharlow-intel mentioned this pull request Oct 28, 2026

Build and test conda packages and wheels IntelPython/mkl-service#265

Merged

jharlow-intel force-pushed the ci/intel-python-ci branch 13 times, most recently from 36f66c0 to 93e4302 Compare November 3, 2026 19:01

vchamarthi reviewed Nov 4, 2026

View reviewed changes

jharlow-intel changed the title ~~Build and test conda-recipe-cf for Intel Python CI~~ Build and test conda packages and wheels Nov 5, 2026

jharlow-intel force-pushed the ci/intel-python-ci branch 2 times, most recently from 7f0e452 to 2ba9e25 Compare November 9, 2026 13:40

jharlow-intel added 2 commits November 9, 2026 10:52

jharlow-intel force-pushed the ci/intel-python-ci branch from 2ba9e25 to 837aff0 Compare November 9, 2026 17:58

Conversation

jharlow-intel commented Oct 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

Uh oh!

vchamarthi left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jharlow-intel commented Nov 9, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jharlow-intel commented Oct 23, 2026 •

edited

Loading