Skip to content

[RF][HS3] Restructure tests to avoid code duplication without losing coverage - #24260

Merged
guitargeek merged 9 commits into
root-project:masterfrom
guitargeek:hs3-test-restructure
Nov 20, 2026
Merged

guitargeek merged 9 commits into
root-project:masterfrom
guitargeek:hs3-test-restructure

Conversation

@guitargeek

Copy link
Copy Markdown
Contributor

The HS3 tests have grown a bit too much, so it is time for a refactoring that keeps the coverage the same or higher.

Assisted-by: claude-code:claude-fable-5-2

The tests in testRooFitHS3.cxx repeated the same steps by hand. They now
use the helpers of the new HS3TestUtils.h:

  * `Imported ws{json}` and `toJSON(ws)` for import and export. Flags of
    `RooJSONFactoryWSTool::config()` are passed to them instead of being
    set and reset around the call, which left a flag set if the call
    threw.
  * `Imported::warnings` for the warnings of an import.
  * gmock's `HasSubstr` for text that a document or a message contains.
  * `reexport(json)` for the check that a document is stable.
  * `addModelConfig()` for the plain ModelConfigs.

A test environment sets the message level for all tests.

No test changes what it checks.

Assisted-by: claude-code:claude-fable-5-2
The round trip of a workspace through HS3 was checked by hand in many
tests: by `validate()`, which compared values only along a variable
called "x", and none for objects that are no functions, or by exporting
and importing again and comparing documents and likelihoods.

The new `expectRoundTrip()` does this in one place. The workspace that
is read from the exported document must be exported to the same
document, have the same components, and give the same values for all
functions and pdfs: in the bins of the observables, for the states of
the categories, and with moved parameters. For a model with data, the
likelihood must be the same. The suite `RoundTrip` runs this for a list
of models, each with the export flags that apply to it.

The tests that only called `validate()` are models of the suite now. So
is the two-channel HistFactory model, with and without data, for the
four layouts of its constraint terms.

Assisted-by: claude-code:claude-fable-5-2
The tests inspected exported documents field by field, with a lookup
and an assertion for every node on the way, and wrote complete
documents where only a part mattered for the test. The helpers of
HS3TestUtils.h replace this:

  * `Doc` gives the node at a path like "distributions/model/samples/0".
  * `matches()` compares a node with an expectation that is written as
    JSON: the listed keys must match, and keys can be required to be
    absent.
  * `document()`, `defaultDomain()`, `channel()`, `HistFactoryDoc` and
    others write the common parts of documents.
  * Forms of older documents are compared with the current form by
    exporting both, where the current form is tested by itself.

Assisted-by: claude-code:claude-fable-5-2
testHS3AuxiliaryMeasurements.cxx exports, imports and reads sections of
documents with the helpers of HS3TestUtils.h now, instead of its own
copies.

Assisted-by: claude-code:claude-fable-5-2
Tests that wrote a model, read it back and then compared documents,
values or likelihoods by hand now register the model with the RoundTrip
suite or call `expectRoundTrip()`. They keep what is specific to them:
the layout of the document, facts about the imported workspace, warnings
and errors.

For models with constraint terms, the suite runs all four layouts of the
constraints, where the tests only covered the default one or looped over
one flag. Models whose first export differs from the later ones say so
with `stableFromSecondExport()`: hand-built HistFactory channels get the
binned-likelihood attributes on import, and an export with renamed
observables writes the attributes of a HistFactory channel under a name
that is not in the document.

The fixture of the HistFactory closure tests loses its dummy backend
parameter and unused members; it builds the same eight models.

Assisted-by: claude-code:claude-fable-5-2
The remaining tests of testRooFitHS3.cxx look into exported documents
with `Doc` and `matches()` and write their input documents with the
builders of HS3TestUtils.h. Checks that a text appears nowhere in a
document stay checks of the whole document.

Tests that repeated the same steps for several inputs loop over a table
of cases. The parametrized suite HistFactoryIncompatibleDuplicateModifiers
is one test with such a table, and ArrayConstraintTermsOfRenamedChannels
is merged into ConstraintTermsOfRenamedChannels.

The helpers of the test file that are left without users are removed.

Assisted-by: claude-code:claude-fable-5-2
A review of the restructured tests found gaps in `expectRoundTrip()`:

  * Variables have no bins by default since 8c50bad, so the values
    were compared at one point per observable only. Observables without
    bins are now scanned at ten points across their range. The
    `validate()` of the old tests had compared no values at all for
    such observables since then.
  * A second workspace without any function of the first one passed.
  * The document was only checked before the second workspace was
    evaluated. It is exported once more afterwards: what an evaluation
    leaves behind must not change it. The RooBinSamplingPdf is exempt,
    because it leaves a named range on its observable that is exported
    as a domain.
  * `likelihoodScan()` passed an empty set of global observables, for
    which RooFit drops the constraint terms of nested products.

Values are compared up to 4 ULPs by default, like `EXPECT_DOUBLE_EQ`,
since the two workspaces can evaluate along different paths. `matches()`
tells a boolean from a string, the option for numbers to compare is
called `comparing()` because it is also used for yields, and
`importError()` gives the message of a failing import.

Checks of the number formatting of bin contents and of a key that must
appear nowhere, which had become checks of values and of one node, are
checks of the text again.

Assisted-by: claude-code:claude-fable-5-2
`object<T>(ws, name)` throws if the workspace has no such object, which
fails the test like the `ASSERT_NE(..., nullptr)` that followed every
lookup before.

Assisted-by: claude-code:claude-fable-5-2
Several exporters wrote their parameters by name. For a literal
constant, that gave a string like `"rate": "10"`, which refers to an
object that is not in the document. RooFit reads this back, but the
document is not valid HS3.

These parameters are now written like the ones of other distributions,
as a number if they are a literal constant and by name otherwise:

  * the rate of a RooExtendPdf,
  * the observable and the mean of a RooPoisson,
  * the coefficients of the polynomials,
  * the lifetime of a RooDecay,
  * the mean and the width of a RooGaussModel,
  * the observable and the parameters of a RooLognormal in the standard
    parametrization,
  * the coefficient of a RooExponential that negates it,
  * the parameters of a ParamHistFunc.

The importer of the RooGaussModel asked for variables as mean and
width, which a number is not: a RooDecay with such a resolution model
could not be read back. It accepts any function now, like the class
itself.

Assisted-by: claude-code:claude-fable-5-2
@github-actions

Copy link
Copy Markdown

Test Results

    22 files      22 suites   3d 11h 18m 3s ⏱️
 3 946 tests  3 946 ✅ 0 💤 0 ❌
77 324 runs  77 324 ✅ 0 💤 0 ❌

Results for commit 5a59b4d.

@guitargeek
guitargeek merged commit cb01aa8 into root-project:master Nov 20, 2026
38 checks passed
@guitargeek
guitargeek deleted the hs3-test-restructure branch November 20, 2026 09:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants