Repository navigation
[RF][HS3] Restructure tests to avoid code duplication without losing coverage - #24260
Merged
Merged
Conversation
The tests in testRooFitHS3.cxx repeated the same steps by hand. They now
use the helpers of the new HS3TestUtils.h:
* `Imported ws{json}` and `toJSON(ws)` for import and export. Flags of
`RooJSONFactoryWSTool::config()` are passed to them instead of being
set and reset around the call, which left a flag set if the call
threw.
* `Imported::warnings` for the warnings of an import.
* gmock's `HasSubstr` for text that a document or a message contains.
* `reexport(json)` for the check that a document is stable.
* `addModelConfig()` for the plain ModelConfigs.
A test environment sets the message level for all tests.
No test changes what it checks.
Assisted-by: claude-code:claude-fable-5-2
The round trip of a workspace through HS3 was checked by hand in many tests: by `validate()`, which compared values only along a variable called "x", and none for objects that are no functions, or by exporting and importing again and comparing documents and likelihoods. The new `expectRoundTrip()` does this in one place. The workspace that is read from the exported document must be exported to the same document, have the same components, and give the same values for all functions and pdfs: in the bins of the observables, for the states of the categories, and with moved parameters. For a model with data, the likelihood must be the same. The suite `RoundTrip` runs this for a list of models, each with the export flags that apply to it. The tests that only called `validate()` are models of the suite now. So is the two-channel HistFactory model, with and without data, for the four layouts of its constraint terms. Assisted-by: claude-code:claude-fable-5-2
The tests inspected exported documents field by field, with a lookup
and an assertion for every node on the way, and wrote complete
documents where only a part mattered for the test. The helpers of
HS3TestUtils.h replace this:
* `Doc` gives the node at a path like "distributions/model/samples/0".
* `matches()` compares a node with an expectation that is written as
JSON: the listed keys must match, and keys can be required to be
absent.
* `document()`, `defaultDomain()`, `channel()`, `HistFactoryDoc` and
others write the common parts of documents.
* Forms of older documents are compared with the current form by
exporting both, where the current form is tested by itself.
Assisted-by: claude-code:claude-fable-5-2
testHS3AuxiliaryMeasurements.cxx exports, imports and reads sections of documents with the helpers of HS3TestUtils.h now, instead of its own copies. Assisted-by: claude-code:claude-fable-5-2
Tests that wrote a model, read it back and then compared documents, values or likelihoods by hand now register the model with the RoundTrip suite or call `expectRoundTrip()`. They keep what is specific to them: the layout of the document, facts about the imported workspace, warnings and errors. For models with constraint terms, the suite runs all four layouts of the constraints, where the tests only covered the default one or looped over one flag. Models whose first export differs from the later ones say so with `stableFromSecondExport()`: hand-built HistFactory channels get the binned-likelihood attributes on import, and an export with renamed observables writes the attributes of a HistFactory channel under a name that is not in the document. The fixture of the HistFactory closure tests loses its dummy backend parameter and unused members; it builds the same eight models. Assisted-by: claude-code:claude-fable-5-2
The remaining tests of testRooFitHS3.cxx look into exported documents with `Doc` and `matches()` and write their input documents with the builders of HS3TestUtils.h. Checks that a text appears nowhere in a document stay checks of the whole document. Tests that repeated the same steps for several inputs loop over a table of cases. The parametrized suite HistFactoryIncompatibleDuplicateModifiers is one test with such a table, and ArrayConstraintTermsOfRenamedChannels is merged into ConstraintTermsOfRenamedChannels. The helpers of the test file that are left without users are removed. Assisted-by: claude-code:claude-fable-5-2
A review of the restructured tests found gaps in `expectRoundTrip()`: * Variables have no bins by default since 8c50bad, so the values were compared at one point per observable only. Observables without bins are now scanned at ten points across their range. The `validate()` of the old tests had compared no values at all for such observables since then. * A second workspace without any function of the first one passed. * The document was only checked before the second workspace was evaluated. It is exported once more afterwards: what an evaluation leaves behind must not change it. The RooBinSamplingPdf is exempt, because it leaves a named range on its observable that is exported as a domain. * `likelihoodScan()` passed an empty set of global observables, for which RooFit drops the constraint terms of nested products. Values are compared up to 4 ULPs by default, like `EXPECT_DOUBLE_EQ`, since the two workspaces can evaluate along different paths. `matches()` tells a boolean from a string, the option for numbers to compare is called `comparing()` because it is also used for yields, and `importError()` gives the message of a failing import. Checks of the number formatting of bin contents and of a key that must appear nowhere, which had become checks of values and of one node, are checks of the text again. Assisted-by: claude-code:claude-fable-5-2
`object<T>(ws, name)` throws if the workspace has no such object, which fails the test like the `ASSERT_NE(..., nullptr)` that followed every lookup before. Assisted-by: claude-code:claude-fable-5-2
Several exporters wrote their parameters by name. For a literal
constant, that gave a string like `"rate": "10"`, which refers to an
object that is not in the document. RooFit reads this back, but the
document is not valid HS3.
These parameters are now written like the ones of other distributions,
as a number if they are a literal constant and by name otherwise:
* the rate of a RooExtendPdf,
* the observable and the mean of a RooPoisson,
* the coefficients of the polynomials,
* the lifetime of a RooDecay,
* the mean and the width of a RooGaussModel,
* the observable and the parameters of a RooLognormal in the standard
parametrization,
* the coefficient of a RooExponential that negates it,
* the parameters of a ParamHistFunc.
The importer of the RooGaussModel asked for variables as mean and
width, which a number is not: a RooDecay with such a resolution model
could not be read back. It accepts any function now, like the class
itself.
Assisted-by: claude-code:claude-fable-5-2
Test Results 22 files 22 suites 3d 11h 18m 3s ⏱️ Results for commit 5a59b4d. |
dpiparo
approved these changes
Nov 20, 2026
dpiparo
approved these changes
Nov 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The HS3 tests have grown a bit too much, so it is time for a refactoring that keeps the coverage the same or higher.
Assisted-by: claude-code:claude-fable-5-2