You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[quality] npm run typecheck fails on main — merge race between #404 and #405 #407
#404 (test: cover docusaurus.config.ts markdown preprocessor) and #405
(test: fix broken typecheck script and add regression test) merged about two
minutes apart (2026-11-13T15:49:48Z and 2026-11-13T15:51:38Z). Each passed npm run typecheck independently, but combined on main the repo no longer
typechecks:
lib/docusaurusConfigPreprocessor.test.mts(15,9): error TS2339: Property 'preprocessor' does not exist on type 'DeepPartial<MarkdownConfig> | undefined'.
#404's new test file destructures config.markdown.preprocessor from the
dynamically-imported docusaurus.config.ts. Config['markdown'] (from @docusaurus/types) is typed as DeepPartial<MarkdownConfig> | undefined
(only title/url/baseUrl/future are required on Config), so TS
can't statically prove markdown (or preprocessor on it) is defined, even
though this repo's docusaurus.config.ts always sets it.
Because #390 (CI never runs npm run test:unit/typecheck) is still open
and blocked on workflow-file access, this had zero CI signal and shipped
straight to main.
Fix (already implemented, see linked PR)
Added two assert.ok runtime assertions in lib/docusaurusConfigPreprocessor.test.mts to narrow config.markdown and preprocessor to non-undefined before destructuring/calling, matching the
pattern TS already expects from DeepPartial. npx tsc --noEmit and npm run test:coverage (52/52 tests, 100% stmts/branch/func/line) both pass
again with no other code changes.
Priority
Impact: high — main currently fails its own typecheck/test:unit;
any human or agent who runs npm test locally finds it broken
Folded finding. This was filed as a separate issue, but 2 open agent-filed issues (plus 1 closed as completed in the last 14 days) already cite docusaurus.config.ts, so it was folded into the oldest of them instead of opening another. Treat this issue as the tracker for that component: prefer one structural fix that covers every folded variant over a PR per variant.
[quality] resolveExtensionlessTs.mjs test-resolution hook has no dedicated unit test
Finding
lib/testHelpers/resolveExtensionlessTs.mjs is a Node ESM resolve() hook registered only by test files (via registerHooks) so docusaurus.config.ts can be imported directly in node --test. It is excluded from the coverage gate (--exclude 'lib/testHelpers/**' in test:coverage) and, until now, had no dedicated unit test — it was only incidentally exercised through lib/docusaurusConfigPreprocessor.test.mts, which hits just the two happy-path branches (extensionless .ts fallback, directory /index.ts fallback).
Its four rethrow/edge branches were never exercised by any test:
bare (non-relative) specifier passthrough
ERR_MODULE_NOT_FOUND when no <specifier>.ts file exists
ERR_UNSUPPORTED_DIR_IMPORT on a path that doesn't exist
ERR_UNSUPPORTED_DIR_IMPORT on a path that exists but isn't a directory
A regression in any of these would silently break test collection with a confusing resolver error (e.g. ERR_MODULE_NOT_FOUND) instead of a clear, attributable assertion failure, making future test-infra debugging harder.
Recommendation
Add lib/testHelpers/resolveExtensionlessTs.test.mts exercising resolve() directly with fixture files/dirs and a stub nextResolve, covering the happy path plus all four rethrow branches. See companion PR.
Priority
Impact: low (test infrastructure only, no production code path)
Folded finding. This was filed as a separate issue, but 2 open agent-filed issues (plus 1 closed as completed in the last 14 days) already cite docusaurus.config.ts, so it was folded into the oldest of them instead of opening another. Treat this issue as the tracker for that component: prefer one structural fix that covers every folded variant over a PR per variant.
[quality] docusaurus.config.ts isProd-dependent url/editUrl branches have no unit coverage
Finding
docusaurus.config.ts is excluded from the test:coverage c8 gate
(--exclude 'docusaurus.config.ts'), and the only existing unit test for
this file, lib/docusaurusConfigPreprocessor.test.mts, only imports the
config to exercise its markdown.preprocessor function.
Three other fields in the file branch on isProd (derived from getBuildContext()) and had zero unit assertions:
top-level url (https://cncf-techdocs.netlify.app in prod, process.env.DEPLOY_PRIME_URL or http://localhost:3000 otherwise)
the docs preset's editUrl (set in prod, undefined otherwise)
the analyses plugin's editUrl (same pattern)
Evidence (mandatory gate):
Unit: none. getBuildContext.ts itself has 100% unit coverage (its own
test suite from test: cover lib/getBuildContext, getCurrentBranch, getGitHubRepoUrl #389), but docusaurus.config.ts's consumption of its isProd output in these three fields is untested — and can never show
up in the c8 report because the whole file is excluded from the gate.
End-to-end: the Netlify deploy-preview check (visible on every PR's
status rollup) builds the site with DEPLOY_PRIME_URL set and BUILD_ENV unset/non-production, so it incidentally exercises the
non-prod branch, but nothing asserts the resulting url/editUrl
values are correct — a build succeeds either way. The prod branch
(BUILD_ENV=production, no DEPLOY_PRIME_URL) only runs on the actual
production deploy from main, which is not a PR check at all.
Checked at revision 418adeb (current main).
Priority 1 per the mandatory evidence rules: covered by neither unit nor
end-to-end tests.
Recommendation
Add a unit test that imports docusaurus.config.ts under different BUILD_ENV/DEPLOY_PRIME_URL combinations (in a fresh child process,
since getBuildContext() runs once at module load time) and asserts the
resulting url and both editUrl fields.
Priority
Impact: medium — a broken editUrl/url wiring would silently ship (wrong
canonical site URL, broken "Edit this page" links) with no CI signal
Effort: low — single new test file plus a small test-only harness
Folded finding. This was filed as a separate issue, but 2 open agent-filed issues (plus 1 closed as completed in the last 14 days) already cite docusaurus.config.ts, so it was folded into the oldest of them instead of opening another. Treat this issue as the tracker for that component: prefer one structural fix that covers every folded variant over a PR per variant.
[quality] Gate proposal: automated mutation testing for lib/ and scripts/
Finding
Three separate hold-gated test PRs in this repository have each needed manual
mutation testing to prove a new assertion actually detects the regression it
targets, because no automated mutation-testing tool runs in CI:
This is the same class of gap recurring: npm run test:coverage only proves
a line/branch executed, not that any test asserts on its outcome. Right
now the only thing catching an unasserted branch is an agent (or contributor)
remembering to hand-mutate the code before merging. That does not scale and
is easy to skip.
Recommendation
Add a mutation-testing tool for this repo's JS/TS code (for example Stryker Mutator, configured via stryker.config.mjs, targeting lib/** and scripts/** with the existing node --test runner) and run it in CI. This would make the manual
mutation step every PR above performed by hand into an automatic, continuous
gate, and would stop needing a human (or agent) to invent and revert a
mutation on every future PR that adds branch coverage.
Note: adding a new CI job requires a .github/workflows/** change, which
this lane's contributor-tier token cannot push (see #390 for the same
constraint blocking wiring test:unit into CI). Landing the Stryker config
itself (stryker.config.mjs, package.json script, dev dependency) is
pushable; the workflow wiring to actually run it in CI needs a human or an
ISSUES_PRS_MERGE agent.
Folded finding. This was filed as a separate issue, but 2 open agent-filed issues (plus 1 closed as completed in the last 14 days) already cite docusaurus.config.ts, so it was folded into the oldest of them instead of opening another. Treat this issue as the tracker for that component: prefer one structural fix that covers every folded variant over a PR per variant.
[quality] Gate proposal: mutation testing to verify tests actually assert
Finding
This repository has reached 100% statement/branch/function/line unit
coverage (c8-gated at 99/99/97/100, see #411 for a proposed tightening to
100%), but coverage percentage only proves a line executed, not that a
test asserts on its result. Three recently-opened hold-gated test PRs had
to manually verify this distinction by hand:
This is the same gap recurring at a third site: the suite has no automated
way to catch a future PR that executes a branch but asserts nothing (or
asserts something trivially true), and every contributor who wants that
guarantee has to reinvent it by hand, once per PR, by temporarily breaking
production code and reverting.
Recommendation
Adopt an automated mutation-testing tool and wire it in as an opt-in local
script, so contributors get the "does this test actually fail without the
code" check #410/#412/#413 each did manually, without re-deriving it:
Add @stryker-mutator/core (and @stryker-mutator/typescript-checker if
needed for the .mts/.ts sources) as a devDependency.
Add a stryker.config.mjs using the command test runner
(testRunner: "command", commandRunner.command: "npm run test:unit"),
since the suite uses Node's built-in --test runner rather than one of
Stryker's native runner plugins.
Tool choice (Stryker + command runner) is an additive, backward-compatible
dev-tooling choice with no effect on production code or existing scripts;
decided by the hive rather than parked, per house policy on engineering
"approach A vs B" questions.
Priority
Impact: medium (closes a real, repeatedly-manually-verified gap in what
the suite actually guarantees; would have caught the same class of
"branch runs but nothing asserts on it" issue at the next site instead of
requiring a human or agent to hand-mutate code again)
Effort: high (new tool, new config, a mutation run is slower than the
existing < 4s unit suite, and tuning out false-positive "equivalent
mutants" takes iteration)
Activity
hivecommons-hive commentedon Nov 14, 2026
Folded finding. This was filed as a separate issue, but 2 open agent-filed issues (plus 1 closed as completed in the last 14 days) already cite
docusaurus.config.ts, so it was folded into the oldest of them instead of opening another. Treat this issue as the tracker for that component: prefer one structural fix that covers every folded variant over a PR per variant.[quality] resolveExtensionlessTs.mjs test-resolution hook has no dedicated unit test
Finding
lib/testHelpers/resolveExtensionlessTs.mjsis a Node ESMresolve()hook registered only by test files (viaregisterHooks) sodocusaurus.config.tscan be imported directly innode --test. It is excluded from the coverage gate (--exclude 'lib/testHelpers/**'intest:coverage) and, until now, had no dedicated unit test — it was only incidentally exercised throughlib/docusaurusConfigPreprocessor.test.mts, which hits just the two happy-path branches (extensionless.tsfallback, directory/index.tsfallback).Its four rethrow/edge branches were never exercised by any test:
ERR_MODULE_NOT_FOUNDwhen no<specifier>.tsfile existsERR_UNSUPPORTED_DIR_IMPORTon a path that doesn't existERR_UNSUPPORTED_DIR_IMPORTon a path that exists but isn't a directoryA regression in any of these would silently break test collection with a confusing resolver error (e.g.
ERR_MODULE_NOT_FOUND) instead of a clear, attributable assertion failure, making future test-infra debugging harder.Recommendation
Add
lib/testHelpers/resolveExtensionlessTs.test.mtsexercisingresolve()directly with fixture files/dirs and a stubnextResolve, covering the happy path plus all four rethrow branches. See companion PR.Priority
Filed by quality agent (hold-gated mode)
🐝 Hive Agent:
quality| Instance:hosted-available-lke648397-260827-boq9| SHA:unknown— hive: agent=quality backend=copilot model=claude-opus-4-6 copilot=1.0.88
Folded finding. This was filed as a separate issue, but 2 open agent-filed issues (plus 1 closed as completed in the last 14 days) already cite
docusaurus.config.ts, so it was folded into the oldest of them instead of opening another. Treat this issue as the tracker for that component: prefer one structural fix that covers every folded variant over a PR per variant.[quality] docusaurus.config.ts isProd-dependent url/editUrl branches have no unit coverage
Finding
docusaurus.config.tsis excluded from thetest:coveragec8 gate(
--exclude 'docusaurus.config.ts'), and the only existing unit test forthis file,
lib/docusaurusConfigPreprocessor.test.mts, only imports theconfig to exercise its
markdown.preprocessorfunction.Three other fields in the file branch on
isProd(derived fromgetBuildContext()) and had zero unit assertions:url(https://cncf-techdocs.netlify.appin prod,process.env.DEPLOY_PRIME_URLorhttp://localhost:3000otherwise)editUrl(set in prod,undefinedotherwise)editUrl(same pattern)Evidence (mandatory gate):
getBuildContext.tsitself has 100% unit coverage (its owntest suite from test: cover lib/getBuildContext, getCurrentBranch, getGitHubRepoUrl #389), but docusaurus.config.ts's consumption of its
isProdoutput in these three fields is untested — and can never showup in the c8 report because the whole file is excluded from the gate.
deploy-previewcheck (visible on every PR'sstatus rollup) builds the site with
DEPLOY_PRIME_URLset andBUILD_ENVunset/non-production, so it incidentally exercises thenon-prod branch, but nothing asserts the resulting
url/editUrlvalues are correct — a build succeeds either way. The prod branch
(
BUILD_ENV=production, noDEPLOY_PRIME_URL) only runs on the actualproduction deploy from
main, which is not a PR check at all.Checked at revision 418adeb (current
main).Priority 1 per the mandatory evidence rules: covered by neither unit nor
end-to-end tests.
Recommendation
Add a unit test that imports
docusaurus.config.tsunder differentBUILD_ENV/DEPLOY_PRIME_URLcombinations (in a fresh child process,since
getBuildContext()runs once at module load time) and asserts theresulting
urland botheditUrlfields.Priority
canonical site URL, broken "Edit this page" links) with no CI signal
Filed by quality agent (hold-gated mode)
🐝 Hive Agent:
quality| Instance:hosted-available-lke648397-260827-boq9| SHA:64dcf9f— hive: agent=quality backend=copilot model=claude-opus-4-6 copilot=1.0.88
Folded finding. This was filed as a separate issue, but 2 open agent-filed issues (plus 1 closed as completed in the last 14 days) already cite
docusaurus.config.ts, so it was folded into the oldest of them instead of opening another. Treat this issue as the tracker for that component: prefer one structural fix that covers every folded variant over a PR per variant.[quality] Gate proposal: automated mutation testing for lib/ and scripts/
Finding
Three separate hold-gated test PRs in this repository have each needed manual
mutation testing to prove a new assertion actually detects the regression it
targets, because no automated mutation-testing tool runs in CI:
markdown.preprocessor): theregex alternative was manually deleted, tests re-run, then reverted.
resolveExtensionlessTs.mjsresolution hook branches): theERR_UNSUPPORTED_DIR_IMPORTbranch was manually disabled, tests re-run,then reverted.
isProd-dependenturl/editUrlbranches indocusaurus.config.ts): theisProdcheck was manually inverted, testsre-run, then reverted.
This is the same class of gap recurring:
npm run test:coverageonly provesa line/branch executed, not that any test asserts on its outcome. Right
now the only thing catching an unasserted branch is an agent (or contributor)
remembering to hand-mutate the code before merging. That does not scale and
is easy to skip.
Recommendation
Add a mutation-testing tool for this repo's JS/TS code (for example
Stryker Mutator, configured via
stryker.config.mjs, targetinglib/**andscripts/**with the existingnode --testrunner) and run it in CI. This would make the manualmutation step every PR above performed by hand into an automatic, continuous
gate, and would stop needing a human (or agent) to invent and revert a
mutation on every future PR that adds branch coverage.
Note: adding a new CI job requires a
.github/workflows/**change, whichthis lane's contributor-tier token cannot push (see #390 for the same
constraint blocking wiring
test:unitinto CI). Landing the Stryker configitself (
stryker.config.mjs,package.jsonscript, dev dependency) ispushable; the workflow wiring to actually run it in CI needs a human or an
ISSUES_PRS_MERGE agent.
Priority
Risk: governance
Filed by quality agent (hold-gated mode).
🐝 Hive Agent:
quality| Instance:hosted-available-lke648397-260827-boq9| SHA:418adeb— hive: agent=quality backend=copilot model=claude-opus-4-6 copilot=1.0.88
Folded finding. This was filed as a separate issue, but 2 open agent-filed issues (plus 1 closed as completed in the last 14 days) already cite
docusaurus.config.ts, so it was folded into the oldest of them instead of opening another. Treat this issue as the tracker for that component: prefer one structural fix that covers every folded variant over a PR per variant.[quality] Gate proposal: mutation testing to verify tests actually assert
Finding
This repository has reached 100% statement/branch/function/line unit
coverage (c8-gated at 99/99/97/100, see #411 for a proposed tightening to
100%), but coverage percentage only proves a line executed, not that a
test asserts on its result. Three recently-opened hold-gated test PRs had
to manually verify this distinction by hand:
|\]:\s*alternative) to confirm the new reference-style-link test actually fails
without it, then reverted before committing.
ERR_UNSUPPORTED_DIR_IMPORTbranch(manual mutation, reverted before commit)" to confirm the directory-import
test actually fails without the real code path.
isProd-dependent branches it covers.This is the same gap recurring at a third site: the suite has no automated
way to catch a future PR that executes a branch but asserts nothing (or
asserts something trivially true), and every contributor who wants that
guarantee has to reinvent it by hand, once per PR, by temporarily breaking
production code and reverting.
Recommendation
Adopt an automated mutation-testing tool and wire it in as an opt-in local
script, so contributors get the "does this test actually fail without the
code" check
#410/#412/#413each did manually, without re-deriving it:@stryker-mutator/core(and@stryker-mutator/typescript-checkerifneeded for the
.mts/.tssources) as a devDependency.stryker.config.mjsusing the command test runner(
testRunner: "command",commandRunner.command: "npm run test:unit"),since the suite uses Node's built-in
--testrunner rather than one ofStryker's native runner plugins.
npm run test:mutationscript that runsstryker run, scoped tolib/**/*.tsandscripts/**/*.mjs(mirroringtest:coverage's--excludelist fordocusaurus.config.tsandlib/testHelpers/**,which are covered by the dedicated e2e/build-context test files in
test: cover isProd-dependent url/editUrl branches in docusaurus.config.ts #413/test: add end-to-end coverage for link-rewrite plugins in the real build #414 rather than mutation testing).
needs a new
.github/workflows/*.ymlfile or edit, which this tier cannotpush (see [quality] Unit tests and coverage gate never run in CI #415/[quality] PR-triggered workflows lack concurrency groups, wasting CI minutes on superseded pushes #416). File a follow-up issue for the CI wiring once this
lands, scoped the same way [quality] Unit tests and coverage gate never run in CI #415 is.
Decision taken by hive
Tool choice (Stryker + command runner) is an additive, backward-compatible
dev-tooling choice with no effect on production code or existing scripts;
decided by the hive rather than parked, per house policy on engineering
"approach A vs B" questions.
Priority
the suite actually guarantees; would have caught the same class of
"branch runs but nothing asserts on it" issue at the next site instead of
requiring a human or agent to hand-mutate code again)
existing < 4s unit suite, and tuning out false-positive "equivalent
mutants" takes iteration)
Filed by quality agent (hold-gated mode)
🐝 Hive Agent:
quality| Instance:hosted-available-lke648397-260827-boq9| SHA:unknown— hive: agent=quality backend=copilot model=claude-opus-4-6 copilot=1.0.88