Skip to content

Add dpnp.minmax with a fused min/max reduction kernel - #3135

Merged
antonwolfy merged 11 commits into
masterfrom
add_minmax_impl
Nov 12, 2026
Merged

antonwolfy merged 11 commits into
masterfrom
add_minmax_impl

Conversation

@antonwolfy

@antonwolfy antonwolfy commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

NumPy 2.6 adds numpy.minmax, which returns the (min, max) tuple of an array in a single pass. This PR adds dpnp.minmax with the same signature. It is backed by a new fused reduction kernel that reads the input once and writes both extrema, instead of running dpnp.min and dpnp.max one after another.

Changes

  • kernels/reductions.hpp: a new tree reduction (no atomics) that keeps the running minimum and maximum per work-item and reduces both over the work-group. It uses native group algorithms for integers, all_of/any_of for bool, and NaN-aware native reductions for real floats: if any work-item holds a NaN, the NaN of the first such work-item is broadcast. Complex types use the custom SLM reduction. There are sequential, one-work-group and multi-pass variants with strided, axis-0 and axis-1 contiguous entry points, and the two outputs have independent strides.
  • source/reductions/minmax.cpp: new binding _tensor_reductions_impl._minmax_over_axis(src, trailing_dims_to_reduce, dst_min, dst_max, sycl_queue, depends).
  • dpnp/tensor/_reduction.py: private _minmax helper, modelled on _comparison_over_axis.
  • dpnp.minmax(a, axis=None, out=None, keepdims=False, initial=None, where=True): out is a tuple of two arrays, and an entry may be None. initial and where are only supported with their default values, as in dpnp.min and dpnp.max.
  • Docs, exports, changelog, and tests in test_statistics.py, test_sycl_queue.py and test_usm_type.py.

Two behaviours follow dpnp.min, dpnp.max and dpnp.modf rather than NumPy, and are not changed here:

  • An out of another dtype receives the result cast after the reduction. NumPy casts the input first.
  • out entries that overlap in memory are rejected, including interleaved views such as (r[:, 0], r[:, 1]).

Performance

Measured on an Intel Core Ultra 7 366H (Panther Lake H) with its integrated GPU on Level Zero and its CPU on OpenCL. The timing is the end-to-end time of dpnp.minmax(a) against dpnp.min(a), dpnp.max(a), each followed by queue.wait(). Runs are interleaved and the median is reported. The speedup is the separate time divided by the fused time; the results of both were checked to be equal in every case.

dtype GPU 1e4 GPU 1e6 GPU 1e8 CPU 1e4 CPU 1e6 CPU 1e8
bool 1.67 1.82 2.17 2.18 3.01 4.00
int8 1.53 1.52 1.81 1.31 1.77 1.83
int32 1.24 1.23 1.55 1.59 1.11 2.03
int64 1.10 1.27 1.35 1.37 1.15 1.88
float16 1.53 1.82 2.41 2.60 3.20 4.89
float32 1.26 1.47 2.22 1.55 2.43 4.60
float64 1.28 1.36 1.41 1.47 2.74 4.69
complex64 1.42 1.47 1.53 1.21 1.12 1.10
  • GPU: the fused kernel is faster in all 84 measured cases, by 1.04x to 2.88x (geomean 1.56x). For example, float32 with 1e8 elements takes 7.2 ms instead of 15.9 ms.
  • CPU: the speedup ranges from 0.93x to 13.9x (geomean 2.29x). The only slower cases are int32 and int64 with 1e5 elements (0.93x and 0.97x), where dpnp.min and dpnp.max use the single-kernel atomic path, and complex128 with 1e3 elements (0.96x).
  • Reductions over each axis of 2-D and 3-D arrays, and over a strided input, gain 1.2x to 2.9x on the GPU and 1.4x to 13.9x on the CPU.

Cost

Build times are for -j6 on 16 CPUs.

  • The full clean build time is unchanged at 15:59. The _tensor_impl link remains the last step to finish.
  • A clean rebuild of the _tensor_reductions_impl target alone goes from 488 s to 511 s (+23 s), measured branch/master/master/branch. The device link takes +20.6 s and minmax.cpp compiles in about 12 s. Compile times of the other reduction sources are unchanged.
  • The _tensor_reductions_impl shared object grows from 35.38 MB to 37.27 MB (+1.89 MB, +5.4%). Of that, 1.0 MB is SPIR-V and 0.45 MB is host code. Compressed with zstd -19 it grows by 166 KB, with gzip -9 by 282 KB, and all dpnp shared objects together grow by 1.0%.
  • Host RSS after import dpnp grows by about 1.6 MB. GPU memory is unchanged, and so is the first-call JIT time of the existing dpnp.min and dpnp.max.

Checklist

  • Have you provided a meaningful PR description?
  • Have you added a test, reproducer or referred to an issue with a reproducer?
  • Have you tested your changes locally for CPU and GPU devices?
  • Have you made sure that new changes do not introduce compiler warnings?
  • Have you checked performance impact of proposed changes?
  • Have you added documentation for your changes, if necessary?
  • Have you added your changes to the changelog?

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The public docstring omits two intentional out incompatibilities: narrowing integer outputs can produce different extrema, and even disjoint interleaved output views are rejected.

1 open issue
What changed in this PR

Adds dpnp.minmax to compute minimum and maximum together using a fused SYCL reduction, extending dpnp’s NumPy-compatible statistics API.

Changes:

  • Adds fused kernels for contiguous and strided inputs, including multi-pass reductions.
  • Integrates the public API, output handling, bindings, exports, and documentation.
  • Adds tests for values, layouts, output buffers, errors, queues, and USM allocation types.
File Description
dpnp/​tests/​test_usm_type.py Checks output USM allocation types.
dpnp/​tests/​test_sycl_queue.py Checks output queue placement.
dpnp/​tests/​test_statistics.py Adds functional and edge-case tests for minmax.
dpnp/​tensor/​libtensor/​source/​reductions/​reduction_common.cpp Registers the new reduction binding.
dpnp/​tensor/​libtensor/​source/​reductions/​minmax.hpp Declares binding initialization.
dpnp/​tensor/​libtensor/​source/​reductions/​minmax.cpp Adds validation, dispatch, and Python binding.
dpnp/​tensor/​libtensor/​include/​kernels/​reductions.hpp Implements fused minimum/maximum kernels.
dpnp/​tensor/​CMakeLists.txt Includes the new binding source in the build.
dpnp/​tensor/​_reduction.py Adds private axis and output handling.
dpnp/​dpnp_iface_statistics.py Adds the public API and docstring.
dpnp/​__init__.py Exports dpnp.minmax.
doc/​reference/​math.rst Lists minmax among extrema functions.
CHANGELOG.md Records the new functionality.

Review effort: Balanced

Comment thread dpnp/dpnp_iface_statistics.py Resolved
@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Array API standard conformance tests for dpnp=0.22.0dev1=np2py314h8d9cdd5_24 ran successfully.
Passed: 1491
Failed: 0
Skipped: 6

@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

View rendered docs @ https://intelpython.github.io/dpnp/index.html

@coveralls

coveralls commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Coverage Status

coverage: 84.467% (+0.04%) from 84.424% — add_minmax_impl into master

Comment thread dpnp/dpnp_iface_statistics.py Resolved

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Casting a read-only output to a different dtype hides its read-only flag, allowing the reduction to mutate the input before failing; validate original output writability before reduction and add a mixed-dtype regression test.

1 open issue
1 resolved since last review

Review effort: Balanced

Comment thread dpnp/dpnp_iface_statistics.py Resolved
@antonwolfy

Copy link
Copy Markdown
Contributor Author

Follow-up proposal: the ASV suite has no reduction benchmarks yet, so nothing tracks the gain of the fused minmax kernel over separate min + max calls. Once this PR is merged, I plan to open a separate PR that adds benchmarks/benchmarks/bench_reductions.py, following bench_elementwise.py:

  • the function as a parameter: min, max, minmax
  • float dtypes and sizes from the shared axes in _utils.py
  • axis=None, 0 and -1 on a 2-D input, to time both the contiguous and the strided kernels

Keeping it out of this PR means the benchmark never runs against a dpnp build without dpnp.minmax.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed to weigh the performance impact of unnecessary device conversions and full-output memory traffic when output dtypes differ.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

Cover float16, NaN at the first, middle and last position (also a complex
NaN in the imaginary part only), integer limits of all integer types,
multi-pass reductions of several rows, axis=() with out, and read-only and
other-queue out errors. Link dpnp.minmax from dpnp.min and dpnp.max.
Comment thread dpnp/dpnp_iface_statistics.py Resolved
Comment thread dpnp/dpnp_iface_statistics.py Resolved
Comment thread dpnp/tensor/_reduction.py Resolved
Comment thread dpnp/tests/test_statistics.py Resolved
Comment thread dpnp/tests/test_statistics.py Resolved
Comment thread dpnp/libtensor/source/CMakeLists.txt Resolved
Comment thread dpnp/tensor/_reduction.py Resolved
antonwolfy and others added 2 commits November 10, 2026 18:57
Move `_minmax` from `dpnp.tensor` to `dpnp_utils_statistics` as `dpnp_minmax` and list `minmax.cpp` in a new `_dpnp_reduction_sources` group so that it gets the dpnp floating-point and coverage flags.

Run the checks of `py_minmax_over_axis` before the early return for an empty destination, make the unchanged locals `const`, depend only on the copy events in the 0-d reduction, and use inline `protected-access` disables.

Add a multi-pass test above the CPU work-group threshold with a complex dtype and a NaN, test the empty-destination checks of the binding, and expect only `ExecutionPlacementError` in the queue test.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues were identified for the new fused dpnp.minmax reduction.

No open issues 🎉

Review effort: Balanced

Comment thread dpnp/tests/test_statistics.py Resolved

@vlad-perevezentsev vlad-perevezentsev left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No more comments from my side
LGTM
Thank you @antonwolfy

@antonwolfy
antonwolfy merged commit 2f3bcef into master Nov 12, 2026
45 checks passed
@antonwolfy
antonwolfy deleted the add_minmax_impl branch November 12, 2026 10:49
github-actions Bot added a commit that referenced this pull request Nov 12, 2026
NumPy 2.6 adds `numpy.minmax`, which returns the `(min, max)` tuple of
an array in a single pass. This PR adds `dpnp.minmax` with the same
signature. It is backed by a new fused reduction kernel that reads the
input once and writes both extrema, instead of running `dpnp.min` and
`dpnp.max` one after another.

### Changes
- `kernels/reductions.hpp`: a new tree reduction (no atomics) that keeps
the running minimum and maximum per work-item and reduces both over the
work-group. It uses native group algorithms for integers,
`all_of`/`any_of` for bool, and NaN-aware native reductions for real
floats: if any work-item holds a NaN, the NaN of the first such
work-item is broadcast. Complex types use the custom SLM reduction.
There are sequential, one-work-group and multi-pass variants with
strided, axis-0 and axis-1 contiguous entry points, and the two outputs
have independent strides.
- `source/reductions/minmax.cpp`: new binding
`_tensor_reductions_impl._minmax_over_axis(src, trailing_dims_to_reduce,
dst_min, dst_max, sycl_queue, depends)`.
- `dpnp/tensor/_reduction.py`: private `_minmax` helper, modelled on
`_comparison_over_axis`.
- `dpnp.minmax(a, axis=None, out=None, keepdims=False, initial=None,
where=True)`: `out` is a tuple of two arrays, and an entry may be
`None`. `initial` and `where` are only supported with their default
values, as in `dpnp.min` and `dpnp.max`.
- Docs, exports, changelog, and tests in `test_statistics.py`,
`test_sycl_queue.py` and `test_usm_type.py`.

Two behaviours follow `dpnp.min`, `dpnp.max` and `dpnp.modf` rather than
NumPy, and are not changed here:
- An `out` of another dtype receives the result cast after the
reduction. NumPy casts the input first.
- `out` entries that overlap in memory are rejected, including
interleaved views such as `(r[:, 0], r[:, 1])`.

### Performance
Measured on an Intel Core Ultra 7 366H (Panther Lake H) with its
integrated GPU on Level Zero and its CPU on OpenCL. The timing is the
end-to-end time of `dpnp.minmax(a)` against `dpnp.min(a), dpnp.max(a)`,
each followed by `queue.wait()`. Runs are interleaved and the median is
reported. The speedup is the separate time divided by the fused time;
the results of both were checked to be equal in every case.

| dtype | GPU 1e4 | GPU 1e6 | GPU 1e8 | CPU 1e4 | CPU 1e6 | CPU 1e8 |
|---|---|---|---|---|---|---|
| bool | 1.67 | 1.82 | 2.17 | 2.18 | 3.01 | 4.00 |
| int8 | 1.53 | 1.52 | 1.81 | 1.31 | 1.77 | 1.83 |
| int32 | 1.24 | 1.23 | 1.55 | 1.59 | 1.11 | 2.03 |
| int64 | 1.10 | 1.27 | 1.35 | 1.37 | 1.15 | 1.88 |
| float16 | 1.53 | 1.82 | 2.41 | 2.60 | 3.20 | 4.89 |
| float32 | 1.26 | 1.47 | 2.22 | 1.55 | 2.43 | 4.60 |
| float64 | 1.28 | 1.36 | 1.41 | 1.47 | 2.74 | 4.69 |
| complex64 | 1.42 | 1.47 | 1.53 | 1.21 | 1.12 | 1.10 |

- GPU: the fused kernel is faster in all 84 measured cases, by 1.04x to
2.88x (geomean 1.56x). For example, float32 with 1e8 elements takes 7.2
ms instead of 15.9 ms.
- CPU: the speedup ranges from 0.93x to 13.9x (geomean 2.29x). The only
slower cases are int32 and int64 with 1e5 elements (0.93x and 0.97x),
where `dpnp.min` and `dpnp.max` use the single-kernel atomic path, and
complex128 with 1e3 elements (0.96x).
- Reductions over each axis of 2-D and 3-D arrays, and over a strided
input, gain 1.2x to 2.9x on the GPU and 1.4x to 13.9x on the CPU.

### Cost
Build times are for `-j6` on 16 CPUs.

- The full clean build time is unchanged at 15:59. The `_tensor_impl`
link remains the last step to finish.
- A clean rebuild of the `_tensor_reductions_impl` target alone goes
from 488 s to 511 s (+23 s), measured branch/master/master/branch. The
device link takes +20.6 s and `minmax.cpp` compiles in about 12 s.
Compile times of the other reduction sources are unchanged.
- The `_tensor_reductions_impl` shared object grows from 35.38 MB to
37.27 MB (+1.89 MB, +5.4%). Of that, 1.0 MB is SPIR-V and 0.45 MB is
host code. Compressed with zstd -19 it grows by 166 KB, with gzip -9 by
282 KB, and all dpnp shared objects together grow by 1.0%.
- Host RSS after `import dpnp` grows by about 1.6 MB. GPU memory is
unchanged, and so is the first-call JIT time of the existing `dpnp.min`
and `dpnp.max`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants