Repository navigation
Conversation
This comment was marked as outdated.
This comment was marked as outdated.
Merging this PR will degrade performance by 62.17%4 warnings, 1 note
Warning Please fix the performance issues or acknowledge them on CodSpeed. Performance Changes
Tip Investigate this regression by commenting Comparing Footnotes
|
|
@not-matthias (CodSpeed) Is the number and magnitude of reported regressions expected? |
2fbfac6 to
82f9420
Compare
|
clippy is sad :) |
82f9420 to
c2b1d9b
Compare
This comment was marked as outdated.
This comment was marked as outdated.
@xtqqczze Hey yes, we're now running each benchmark in it's own process which previously caused variance issues: https://github.com/CodSpeedHQ/codspeed-rust/releases/tag/v6.0.0 Looking at the flamegraphs, it looks like you have some The same seems to be happening in the memory benchmarks, where some now have a |
|
@xtqqczze Side note, you can also tag @codspeedbot to analyze the flamegraphs which can do the same analysis (in case I miss your ping) @codspeedbot demonstrate for me pls: fact check me here, analyze the flamegraphs and and give concrete details on what's causing this EDIT: ^ you can try this if you want |
|
@codspeedbot analyze the flamegraphs and and give concrete details on what's causing performance changes |
CC @sylvestre |
c2b1d9b to
a77be9e
Compare
In this case of |
|
@sylvestre once we merge this, we should likely rebase most PRs for new performance baseline :( |
|
CI is currently red on |
a77be9e to
dfe6aff
Compare
This comment was marked as outdated.
This comment was marked as outdated.
3241c28 to
0e6f033
Compare
This comment was marked as outdated.
This comment was marked as outdated.
0e6f033 to
ad59f60
Compare
|
@sylvestre ready |
ad59f60 to
bb5562b
Compare
|
/rerun OpenBSD |
7e3fdc7 to
31d1e05
Compare
|
@sylvestre It seems that adding benchmarks in #16439 has affected unrelated benchmarks in subsequent PRs. We need these changes to run benchmarks in separate processes to avoid such interference and ensure reliable results. |
31d1e05 to
9f55340
Compare
|
Binary size comparison: |
This comment was marked as outdated.
This comment was marked as outdated.
9f55340 to
bd09f97
Compare
|
GNU testsuite comparison: |
|
@sylvestre seems like good timing to merge now |
bd09f97 to
6c3f3b1
Compare
6c3f3b1 to
0fbc81f
Compare
|
Updated to v6.1.0. |
This PR contains the following updates:
5.0.0→6.1.0Release Notes
CodSpeedHQ/codspeed-rust (codspeed-divan-compat)
v6.1.0Compare Source
Highlights
telemetrymeasurement mode, supported by the divan, criterion and bencher compat cratesWhat's Changed
Full Changelog: CodSpeedHQ/codspeed-rust@v6.0.0...v6.1.0
v6.0.0Compare Source
CodSpeed for Rust 6.0.0 removes a source of measurement noise from your CPU Simulation benchmarks, so you can spend less time investigating unexpected changes in unrelated results.
Highlights
Your Divan and Criterion benchmarks now run in separate processes. Changes to one benchmark no longer carry allocator state into the benchmarks that follow it, improving consistency as your benchmark suite evolves.
Early results showed lower run-to-run variance, with roughly 70% fewer benchmarks varying by more than 1%.
No benchmark code changes required. Your WallTime results are unaffected.
Breaking Changes
Set
CODSPEED_PROCESS_ISOLATION=falseto revert to the previous behavior.What's Changed
Full Changelog: CodSpeedHQ/codspeed-rust@v5.0.3...v6.0.0
Contributes to #14872.