Skip to content

Require references for every library benefit and check - #606

Merged
prestoncabe merged 15 commits into
mainfrom
594-move-existing-library-benefits-and-checks-to-references-and-make-them-required
Nov 20, 2026
Merged

prestoncabe merged 15 commits into
mainfrom
594-move-existing-library-benefits-and-checks-to-references-and-make-them-required

Conversation

@prestoncabe

Copy link
Copy Markdown
Collaborator

Closes #594

Summary

Every exposed library benefit and check ({Name}Service under benefits/ or checks/) now carries a complete reference, and startup fails without one.

 ModelRegistry.initialize
+  validateUniqueModelNames            # moved first: the scan is keyed by model name
   buildModelCache
     scannedPath
+      exposed or has `reference` but scan missed its file → fail startup
     publishedReference
+      `reference` decision outside benefits/ or checks/ → fail startup
+      exposed benefit/check with no `reference` decision  → fail startup
       (programYear stays optional)

LibraryReferenceCoverageTest enforces what startup doesn't: a rationale for every check, a description for every parameter (on the type the parameters input uses, defined in the check's own file), and at least one Bruno example (docs { } block) per model.

Content added to all 26 models (6 PHL benefits + 20 checks):

summary               → model description
reference decision    → BDT.tBenefitReference / BDT.tEligibilityCheckReference (sources checked on 2026-11-19)
check rationale       → description on each `checks` context entry
parameter descriptions→ description on each tParameters component
implementation notes  → SCTF (income limit not checked), SctfAgeRequirement (ages measured Dec 31)
Bruno docs { } blocks → OpenAPI examples (new SomeoneMinAge tests)

Also:

  • The JAR scan now decodes the path, and the scan logs an error when it finds nothing.
  • new-dmn-benefit and new-dmn-check skills, templates, and library-api/AGENTS.md now say references are required.
  • openapi.snapshot.json regenerated.

Follow-ups found while writing references: #601 (SCTF income limit and spouse household), #602 (LOOP 150% limit for pre-2023 households).

Evidence

  • Before: a library benefit or check could ship with no reference. ReferenceOpenApiTest.modelsWithoutAReferenceDecisionPublishNone asserted that this was allowed.
    After:
    ReferenceEvaluationTest
      anExposedCheckWithoutAReferenceFailsStartup              ✓
      anExposedBenefitWithoutAReferenceFailsStartup            ✓
      aReferenceOutsideBenefitsAndChecksFailsStartupClearly    ✓
      anExposedModelTheScanMissedFailsStartup                  ✓
      anExposedUtilityOutsideBenefitsAndChecksNeedsNoReference ✓
      aBaseModuleWithoutAServiceNeedsNoReference               ✓
    LibraryReferenceCoverageTest
      everyExposedBenefitAndCheckIsFullyDocumented             ✓  (gaps == [])
    
  • devbox run test: all suites pass — library-api 49 JVM tests, Bruno 211 requests / 418 assertions, Playwright 25 passed, builder-frontend, builder-api, examples.
  • docs: npm test && npm run build pass.

Merge Danger

Door: two-way

The behaviour changes are a startup guard and documentation. Reverting restores the old optional behaviour. No production data is touched.

Blast Radius: contributors

  • Any new benefit or check without a full reference now fails startup or mvn test. The skills and templates explain what to add.
  • The OpenAPI contract (and the docs site built from it) gains descriptions and examples. Request and response schemas are unchanged, apart from added descriptions.
  • PersonNotEnrolledInBenefit now uses its own tParameters type instead of the one imported from PersonEnrolledInBenefit. The schema shape is the same, but the generated schema name can differ.

🤖 Generated with Claude Code

prestoncabe and others added 15 commits November 19, 2026 17:58
Add a benefit or eligibility check reference to the six Philadelphia
benefits and the twenty checks that lacked one, sourced from the City's
program pages (checked 2026-11-19): summaries, reference decisions, check
rationale, parameter descriptions and implementation notes. Mark
representative Bruno tests as examples with docs blocks, and add tests
for SomeoneMinAge, which had none.

Startup now fails when an exposed model under benefits/ or checks/ has
no reference decision. LibraryReferenceCoverageTest also requires a
rationale for each check, a description for each parameter and at least
one Bruno example. The DMN skills, their templates and library-api
AGENTS.md describe references as required, and the OpenAPI snapshot is
regenerated.

Closes #594

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add ComponentWidths for every expression in the new reference decisions
so they render as the DMN skills describe. Correct the explanations of
ApplicantOrSpouseMinAge and PropertyTaxesCurrentOrPaymentPlan, which pass
when one path is true even if the other is undecided, and of
TaxFreezeIncome, which reads an already-combined income field.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SctfAgeRequirement measures age on December 31 of the current year, so
an applicant who turns 65 later this year already shows as eligible. The
benefit reference said only "65 or older"; add a limitation saying so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The explanation said the check returns no result whenever a needed date
of birth or answer is missing. It actually passes as soon as any path
passes, fails when every path fails, and returns no result only when no
path passes and at least one cannot be decided.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Whether a model needs a reference depends on its file path, and the path
fell back to the bare model name when the classpath scan did not record
the file. A check whose file failed to parse, or every model after a
failed scan, then skipped the reference requirement silently. Now an
exposed model with no scanned path stops startup.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The rule (a {ModelName}Service under benefits/ or checks/) was written out
in ModelRegistry, the coverage test and, for its exposure half, the
OpenAPI filter. Move it into ModelInfo.isExposed and isLibraryExposed so
the startup check and the coverage test cannot drift apart.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The explanations named some inputs that leave a check undecided but not
all of them:
- TaxFreezeIncome also needs primaryPersonId to find a spouse.
- ApplicantOrSpouseMinAge cannot decide either path without
  primaryPersonId, and cannot decide the spouse path when the spouse is
  old enough but the household answer is missing, or when the spouse may
  live there and their date of birth is missing.
- PersonEnrolledInBenefit and PersonNotEnrolledInBenefit also return no
  result when the personId or benefit parameter is missing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The scan cut the JAR path out of the resource URL without decoding it, so
a JAR under a directory with a space in its name was not found. The scan
then came back empty, and since a missed exposed model now fails startup,
every benefit and check reported a missing DMN file. Now the path is
decoded as a URI, and that error says the scan probably failed and points
at the earlier scan log.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The test only collected gaps, so an empty registry, after a failed scan or
runtime lookup, passed with nothing checked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The classpath scan is keyed by model name, so a copied DMN that kept its
original name overwrote the other file's path and reference docs. The
registry is built before the duplicate check, and a missing reference
now fails it, so startup reported a misleading reference error and never
reached 'Duplicate DMN model names detected'. Now names are checked first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The reference kind came from the path, so any file outside benefits/ was
validated as a check reference. A reference outside both folders, which
nothing reads, then failed with a confusing unknown-field error. Now it
fails startup saying the file is not under benefits/ or checks/.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hing

The startup error for an exposed model the scan missed points to earlier
scan errors in the log. An unsupported resource URL or an empty scan
logged none, so that pointer led nowhere.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
scannedPath fell back to the bare model name for a model with a
reference decision but no Service, so publishedReference then reported
that its file was outside benefits/ and checks/. The real cause was
that the scan missed the file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ReferenceDocs read any local itemDefinition named tParameters, whether or
not the parameters input used it. PersonNotEnrolledInBenefit typed its
input as PersonEnrolledInBenefit.tParameters and documented an unused
local copy, so its published parameter docs could drift from the real
schema without failing a test.

ReferenceDocs now follows the parameters input's typeRef, and
LibraryReferenceCoverageTest fails when that type is not defined in the
check's own file. PersonNotEnrolledInBenefit now uses its local
tParameters, which has the same fields; the OpenAPI snapshot points at
it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MinYearsOwnerOccupant returns null when minYears is missing or negative,
but its explanation only listed the number of years.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@prestoncabe prestoncabe linked an issue Nov 20, 2026 that may be closed by this pull request
6 tasks
@prestoncabe
prestoncabe merged commit 68ecd49 into main Nov 20, 2026
7 checks passed
@prestoncabe
prestoncabe deleted the 594-move-existing-library-benefits-and-checks-to-references-and-make-them-required branch November 20, 2026 01:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Move existing library benefits and checks to references and make them required

1 participant