Repository navigation
Finish the roadmap short of running in the browser: GraphQL, link-check, migration-safety, sanitizers, CMake layout checks, follow-ups - #179
Merged
Conversation
A format recognised by intent: a `.graphqls` file, a `schema.graphql`, a `.graphql` or `.gql` file holding type definitions, or a GraphQL Config file; the operations documents count once a schema does. CI builds the schema and validates the operations (`schema-validation`: `graphql-inspector validate`, `introspect` or `diff`; GraphQL Code Generator), lints it (`static-analysis`: graphql-schema-linter, eslint with `@graphql-eslint/eslint-plugin`), formats it (`code-formatting`: prettier) and diffs it for breaking changes (`schema-compatibility`: `graphql-inspector diff`, the `graphql-hive/graphql-inspector` action unless `fail-on-breaking` is false, `rover graph check`). A scoped npm package (`@redocly/cli`, `@graphql-inspector/cli`) is now one program name as well as its own (`@biomejs/biome` is still `biome`), so `npx @redocly/cli lint` is recognised as redocly; the OpenAPI fixture gains that tool run. Part of cpp-best-practices#89. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
CI checks each documentation site's links and fails on a broken one: a new aspect of the tool model, `links` (facts/links.py), with an advised policy (none warns: a broken link misleads a reader without breaking a build). The documentation profile's tools: lychee (and lycheeverse/ lychee-action, which fails the job unless `fail: false`; `--dump` only lists), Sphinx's linkcheck builder (`-b`/`-M linkcheck`, `make linkcheck`), markdown-link-check (and its actions), mdBook's build with `[output.linkcheck]` in book.toml or mdbook-linkcheck itself, MkDocs' build with the htmlproofer plugin, linkinator, muffet, htmltest, HTMLProofer, LinkChecker. A scheduled workflow's jobs were already read, so a scheduled run counts. The documentation fixture gains a `links` job running the lychee action on a schedule as well as on push; every fixture's results and answers snapshots gain the check and the question. Part of cpp-best-practices#91. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
The last row of cpp-best-practices#86's schema-declared: each document of one of the project's XSDs (its root element in the schema's targetNamespace, or a global element of a schema with none) names the schema's location with `xsi:schemaLocation`, `xsi:noNamespaceSchemaLocation` or an `xml-model` processing instruction. A second `Declares` of the XML Schema format finds them; the answer joins what a format's files are asked ("their targetNamespace or schema location"). Other XML (a build file, a document of another schema) is not asked; one that does not parse declares nothing that is read. `elementFormDefault` is a choice with a defined default, not a declaration, and is not asked; the XML schemas page says so, and no longer lists the example documents of #65 as unbuilt. The XML fixtures' documents (and the XML Schema layout's) name their schema. Part of cpp-best-practices#86. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
Which sanitizers a project uses, where ci-build-types asks that each platform builds with and without any. A practice (facts/practices.py) judged by the settings policy, now `judge_settings`, which the hardening check shares (its passed wording takes the settings found). C/C++: AddressSanitizer and UndefinedBehaviorSanitizer, each a flag requirement met by `-fsanitize=address,undefined` (MSVC's `/fsanitize=address`) among the build files' options or on a CI command line, by cmake_template's ENABLE_SANITIZER_* options (prefixed or not) on by default or set by CI, or by Meson's b_sanitize; judged across the build, not per compiler, since MSVC has no UBSan. One of the two warns, neither fails. Rust: `cargo miri test` or a nightly -Zsanitizer build. Go: `go test -race` (-asan and -msan count too). Nothing fails. The C/C++ fixture's CMakeLists gains the ENABLE_SANITIZERS option its CI already toggles; the Rust and Go fixtures already sanitize. Questions for sanitizers and cxx-extensions (which had none) join `--answers`. Part of #10. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
CI checks a project's database migrations for changes that are unsafe to apply to a live database: a new aspect of the tool model, `migrations` (facts/migrations.py), whose linters define what they check, the migrations themselves (Flyway-named files, or .sql under migrations/, db/migrate/ or db/migrations/: the files the SQL profile already recognised a migrations project by, now shared). A project without migrations is not judged (skipped); one whose CI checks nothing warns, as the migrations may be run against a copy first. The SQL profile's linters: squawk (now one tool for `static-analysis` and this check, as `squawk` or npm's `squawk-cli`; sbdchd/squawk-action counts only with `fail-on-violations: true`, since by default it comments on the pull request), `atlas migrate lint` (and ariga/atlas-action/migrate/lint), `liquibase checks run`. The SQL fixtures gain a Flyway migration; the well-configured one runs squawk on it. Part of cpp-best-practices#92. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
tools/scanners.py knows what CodeQL, GitLab SAST and semgrep read (CODEQL_LANGUAGES, GITLAB_SAST_LANGUAGES, SEMGREP_LANGUAGES), and tools/audits.py what osv-scanner, trivy, snyk and GitLab Dependency Scanning have lockfile readers for. A profile asks for the tools of its languages and gets those that read one of them, for those languages only; the hand-written filters of the Lua, Julia, OCaml, Clojure, Dart, BEAM, Scala and PHP profiles go, and the CodeQL layout test derives what CodeQL does not read from the table instead of its own list. What the tables change: CodeQL reads C#, not F#; nothing reads rebar.lock, so Erlang has no auditor (mix_audit is a Mix project's, Elixir's) and an Erlang-only project's dependency-audit is skipped rather than warned. The BEAM snapshots follow. Part of cpp-best-practices#143. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
Three practices on how a C/C++ build is laid out (ecosystems/c_cpp/layout.py), each met by any one setting and warning when none is, skipped without CMake or Meson build files: - out-of-source-build: a CMake `if()` compares the source and binary directories (cmake_template's PreventInSourceBuilds), or sets CMAKE_DISABLE_IN_SOURCE_BUILD or CMAKE_DISABLE_SOURCE_CHANGES; Meson builds out of source by itself. - compile-commands: CMAKE_EXPORT_COMPILE_COMMANDS on in the CMake files, a preset or on the command line; Meson always writes the file. - top-level-guard: an `if()` on PROJECT_IS_TOP_LEVEL (or <project>_IS_TOP_LEVEL, or CMAKE_SOURCE_DIR STREQUAL PROJECT_SOURCE_DIR), a cmake_dependent_option depending on it, or Meson's meson.is_subproject(). What is guarded is not yet read. What an `if()` guards on is now a fact: both evaluators record each condition as written (a `guard` setting of CMake or Meson), reached in some build or not; the CMake evaluator hands `if()` and `elseif()` to its observer like any command. The policy for a practice met by any one setting learns to warn (`Wording.warns`). The C/C++ fixture's CMakeLists gains the in-source check, the export and a PROJECT_IS_TOP_LEVEL guard around its tests (still found: the evaluator knows the project is top level). Part of #18, #23, #25. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
A CI job's service containers (GitHub's `services:`, GitLab's) are read as the images it runs beside itself (`CiJob.services`, `Job.services`): what a database project is tested against. The SQL profile's platform axis is the database engine versions those images name (`postgres:16` is PostgreSQL 16; MySQL, MariaDB, SQL Server, CockroachDB, ClickHouse, TimescaleDB; PostGIS counts as PostgreSQL), and an axis may leave the OS out (`ToolchainAxis.operating_systems`): the platform policy then asks for two engine versions, whatever runs them, since a database project has no use for several operating systems. platform-matrix applies to SQL, where it was skipped; the SQL fixture tests against two PostgreSQL versions. Part of cpp-best-practices#148. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
`outputs_of` returned the outputs of the build systems in use or, only when none had any, the language-wide ones, so a profile tying one output to a build system hid its language-wide outputs for that build system: a Maven or Gradle project was never asked to ignore `*.class`, and BEAM had to leave deps/ untied so _build/ still counted for Elixir. The two sets now add up: a build of the language writes the language's own outputs whatever builds it. BEAM ties deps/ to Mix, which alone fetches into it; the well-configured JVM fixture ignores `*.class`, and the bare one's committed class file counts beside its target/. The other half of cpp-best-practices#145, a shared output reported under one language of an ecosystem, no longer shows: the languages' answers agree and fold into one, as the bare BEAM fixture's do; where they differ, each language lists the outputs it owns. Part of cpp-best-practices#145. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
Four small things noted while building the matrix and the grid: - the support matrix lists a tool several ecosystems share (CodeQL, osv-scanner, lizard: in the cells of three or more) once below the table, with the ecosystems it reads; a cell it alone filled shows `↓` (SupportMatrix.shared, the JSON's "shared"); - the per-project grid (`--format markdown`) says when no ecosystem was recognised, instead of an empty section; - `--list --format markdown` renders the checks as a table of check, ecosystems and description, where it printed the text list; - on a pull request, the self-validation job writes the diff of the support matrix against the base branch to the job summary, so a profile's or a check's effect shows without reading the snapshot. Part of cpp-best-practices#149. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
A toolchain axis may read a job's toolchains with the project's facts
(`ToolchainAxis.of_job_in`, used by the platform question where given),
since a job alone does not tell what its build names. The Scala axis
reads the versions an sbt build cross-builds (`crossScalaVersions :=
Seq(...)` in build.sbt or another .sbt file) or a Mill build's
`Cross[...]("...")`, from a settings reader of the build files, and
counts them for a job that cross-builds: an sbt task with the `+`
prefix (`sbt +test`; `++3.3.5` only switches), or a Mill selector over
every module (`mill __.test`). Each runs on each JDK the job names, as
the switched-to versions did.
Part of cpp-best-practices#146.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
A toolchain's patterns may name the actions whose inputs name a version
(`ToolchainPatterns.steps`: the action's repository and a reader of its
inputs); `detect_toolchains` reads the job's `ActionStep` facts with
them, beside the texts. The Dart profile reads dart-lang/setup-dart's
`sdk` and subosito/flutter-action's `flutter-version` so, and the
action's `channel` when no version is given ("Flutter stable"), where
the text patterns could not match a key as generic as `channel:` and a
job giving both would have counted two toolchains; an action given
neither sets up the stable channel.
Part of cpp-best-practices#147.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt
Read service containers as CI facts; SQL's platforms are engine versions
Let a build system's outputs add to the language's own
Polish the support matrix and the markdown grid
Count the Scala versions a cross-build runs as toolchains
Read a setup action's inputs structurally for the toolchain it names
Let the shared scanners and auditors declare the languages they read
Add the out-of-source-build, compile-commands and top-level-guard checks
Add the sanitizers check
Add the migration-safety check
Add the link-check check
Add the GraphQL profile
Ask the documents of an XSD to name the schema's location
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #179 +/- ##
========================================
Coverage 99.77% 99.77%
========================================
Files 258 273 +15
Lines 13560 14063 +503
Branches 1495 1544 +49
========================================
+ Hits 13529 14031 +502
- Misses 31 32 +1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The remaining roadmap, short of running in the browser (#127): twelve PRs reviewed and merged bottom-up on the fork (leftibot#67–#78), each CI-green.
New profile and checks
.graphqls, ambiguous.graphql/.gqlconfirmed by a type definition or a.graphqlrc; graphql-inspector (validate, diff), graphql-codegen, rover, graphql-schema-linter, graphql-eslint, prettier, and the two inspector Actions. A scoped npm package (@redocly/cli,@graphql-inspector/cli) is now one program name and its basename, which also fixesnpx @redocly/cli lint.xsi:schemaLocation,noNamespaceSchemaLocation,<?xml-model?>).elementFormDefaultis deliberately not asked: it is a style choice, not a practice.linksaspect; lychee, markdown-link-check, linkinator, muffet, htmltest, html-proofer, linkchecker, sphinx-b linkcheck, mdbook-linkcheck, mkdocs-htmlproofer. Advised:fail: falseon the lychee Action counts as nothing.migrationsaspect; squawk, atlas migrate lint, liquibase checks and their Actions. Flyway-named files andmigrations/directories are migrations; a project with none is skipped.sanitizerscheck: ASan/UBSan (or Miri) are used #10): AddressSanitizer and UndefinedBehaviorSanitizer in some configuration of a C/C++ build (one of the two warns), including cmake_template'sENABLE_SANITIZER_*options and Meson'sb_sanitize; Miri or-Zsanitizerfor Rust;go test -racefor Go.out-of-source-buildcheck: in-source builds are prevented #18, Addcompile-commandscheck: compile_commands.json is exported #23, Addtop-level-guardcheck: the project can be consumed as a subproject #25): both build evaluators now record what eachif()guards on, as written, so the practices readPROJECT_IS_TOP_LEVEL,CMAKE_SOURCE_DIR STREQUAL CMAKE_BINARY_DIRandmeson.is_subproject()structurally. All three warn. Header-only projects are skipped.Follow-ups
channelcounts as a toolchain when no version is given.--list --format markdown; CI shows the matrix diff on pull requests.*.classas well.platform-matrixasks for two database engine versions, whatever the OS.History item 14 records the round.
🤖 Generated with Claude Code
https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt