Skip to content

Scope Doors preview browser checks to changed pages - #1278

Open
LukasWallrich wants to merge 10 commits into
rebrand-doorsfrom
rebrand-page-scoped-ci
Open

LukasWallrich wants to merge 10 commits into
rebrand-doorsfrom
rebrand-page-scoped-ci

Conversation

@LukasWallrich

@LukasWallrich LukasWallrich commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Doors preview resolves the production data and announcements artifacts from successful main workflow runs. The candidate, PR base and main reference build in parallel from those pinned inputs. The Contributors page comes from the pinned data artifact. For scoped PRs, the base job reuses a successful push build at the exact base SHA when its manifest records the same artifacts and Contributors provenance. The comparison waits for both sites and checks URLs, structured data, SEO tags and internal links across the whole site.

The browser checks compare rendered HTML. CSS and JS changes add the core sample and retain all changed articles, including generated articles outside that sample. Changes to either the home page or Resources run the topic/facet assertion across both pages. Pushes, check infrastructure changes, static files and non-bundle assets, the full-checks label and diffs over 60 pages run the full suite.

Parallel reference benchmark (UTC)

Draft #1286's Doors preview run changed one Markdown page and selected only /replication-hub/submit/. The temporary base enabled the workflow trigger and had no reusable push build. The draft was closed and both temporary branches were deleted.

Job Start End Result
Resolve build inputs 13:37:25 13:37:28 Pass
Build 13:37:31 13:40:35 Pass
Build PR base 13:37:31 13:40:35 Pass
Build main reference 13:37:31 13:39:11 Pass
Compare with main 13:40:38 13:41:55 Pass
Designed screens 13:40:38 13:41:44 Pass
Accessibility rules 13:40:38 13:41:44 Pass
Deploy preview 13:40:36 13:40:36 Skipped
Check results 13:41:58 13:42:00 Pass

The gate passed 4m 35s after the first job started. Compare with main took 1m 17s, down from 2m 27s in the earlier content-only run. Candidate build times also differed between the runs.

Local validation passed for actionlint, 209 unit tests, the Hugo build, SEO HTML, sitemap lastmod against source history, and the comparison with main using production artifacts.

Earlier job timings on 2026-11-21 (UTC)

The draft PRs targeted a temporary base branched from this head, with deployment disabled and only the branch triggers changed. All three reused the same successful push build. In the previous cold runs the base job took 4m 32s for content and 5m 36s for CSS. With reuse it took 22–24s, and the browser jobs started as soon as the candidate was ready.

Job Content only CSS only CSS + content
Resolve build inputs 13:10:27–13:10:30 (3s) 13:17:12–13:17:15 (3s) 13:17:11–13:17:15 (4s)
Build 13:10:38–13:15:26 (4m 48s) 13:17:17–13:21:54 (4m 37s) 13:17:18–13:21:41 (4m 23s)
Build PR base 13:10:33–13:10:55 (22s) 13:17:17–13:17:41 (24s) 13:17:18–13:17:40 (22s)
Designed screens 13:15:29–13:16:17 (48s) 13:21:57–13:26:42 (4m 45s) 13:21:44–13:26:08 (4m 24s)
Accessibility rules 13:15:29–13:16:19 (50s) 13:21:57–13:25:09 (3m 12s) 13:21:44–13:25:32 (3m 48s)
Compare with main 13:15:29–13:17:56 (2m 27s) 13:21:57–13:24:51 (2m 54s) 13:21:45–13:24:57 (3m 12s)
Check results 13:17:59–13:18:03 (4s) 13:26:44–13:26:46 (2s) 13:26:11–13:26:20 (9s)
First job start to gate end 7m 36s 9m 34s 9m 09s

All three Doors checks passed. The content trial selected only the edited educators-corner page. The CSS trial selected core mode with zero changed HTML pages and 46 article pages. The mixed trial retained the core sections, added the edited article to the article set for 47 pages, and ran its call-out and phone-reading sections. The initial CSS fixture did not change the emitted bundle, so the table uses the corrected stylesheet-token trials. All three drafts were closed and their branches and the temporary base removed.

Earlier workflow totals, including queueing
Workflow Content-only PR CSS-only PR Full
Before #1278 ~11 min ~11 min ~11 min
First #1278 version 7m 51s 10m 28s —
Updated, cold cache 6m 43s 11m 00s 11m 13s
Updated, warm-cache rerun 7m 00s 5m 45s, Contributors only —

Implemented with GPT-6.2 Sol in Codex.

@LukasWallrich
LukasWallrich requested a review from a team as a code owner November 21, 2026 11:30
@github-actions

Copy link
Copy Markdown
Contributor

👍 All image files/references (if any) are in webp format, in line with our policy.

@github-actions

github-actions Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

✅ Spell Check Passed

No spelling issues found when checking 13 changed file(s)! 🎉

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 6 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="scripts/rebrand/check_design.py">

<violation number="1" location="scripts/rebrand/check_design.py:1730">
P2: The cross-page topic/facet check no longer runs when either `/` or `/resources/` changes alone, so a changed topic URL or removed facet can ship without validation. Schedule `check_directory` when either page changes and run this assertion against the unchanged counterpart as well.</violation>
</file>

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread scripts/rebrand/check_design.py Outdated
"check_conference": (CONFERENCE,),
"check_review_pages": ("publications/", "clusters/", "disciplines/", "booklet/", "community-map/"),
"check_contributor_network": (NETWORK_PATH,),
"check_directory": ("resources/",),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The cross-page topic/facet check no longer runs when either / or /resources/ changes alone, so a changed topic URL or removed facet can ship without validation. Schedule check_directory when either page changes and run this assertion against the unchanged counterpart as well.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At scripts/rebrand/check_design.py, line 1730:

<comment>The cross-page topic/facet check no longer runs when either `/` or `/resources/` changes alone, so a changed topic URL or removed facet can ship without validation. Schedule `check_directory` when either page changes and run this assertion against the unchanged counterpart as well.</comment>

<file context>
@@ -1647,7 +1707,36 @@ def main():
+        "check_conference": (CONFERENCE,),
+        "check_review_pages": ("publications/", "clusters/", "disciplines/", "booklet/", "community-map/"),
+        "check_contributor_network": (NETWORK_PATH,),
+        "check_directory": ("resources/",),
+    }
+    if mode == "changed":
</file context>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 GPT-6.2 Sol responding on behalf of Lukas Wallrich

check_directory now runs when either / or /resources/ changes and opens both pages for the topic/facet assertion.

Comment thread scripts/rebrand/changed_pages.py Resolved
Comment thread .github/workflows/rebrand-preview.yaml Resolved

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 1 file (changes from recent commits).

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread .github/workflows/rebrand-preview.yaml Resolved

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 2 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name=".github/workflows/rebrand-preview.yaml">

<violation number="1" location=".github/workflows/rebrand-preview.yaml:398">
P2: This selection skips browser checks when a PR changes only a static image or other non-HTML/CSS/JS visual asset. Include referenced visual assets in the comparison, or force at least the affected pages into the selection.</violation>
</file>

Tip: Review your code locally with the cubic CLI to iterate faster.

View guided diff | Turn on auto-fix | Re-trigger cubic

tar -xf base/site.tar -C base
mkdir public
tar -xf site.tar -C public
python3 scripts/rebrand/changed_pages.py base public --out changed.json | tee selection.log

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: This selection skips browser checks when a PR changes only a static image or other non-HTML/CSS/JS visual asset. Include referenced visual assets in the comparison, or force at least the affected pages into the selection.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At .github/workflows/rebrand-preview.yaml, line 398:

<comment>This selection skips browser checks when a PR changes only a static image or other non-HTML/CSS/JS visual asset. Include referenced visual assets in the comparison, or force at least the affected pages into the selection.</comment>

<file context>
@@ -388,40 +369,51 @@ jobs:
+            tar -xf base/site.tar -C base
+            mkdir public
+            tar -xf site.tar -C public
+            python3 scripts/rebrand/changed_pages.py base public --out changed.json | tee selection.log
+          else
+            python3 scripts/rebrand/changed_pages.py --full --out changed.json | tee selection.log
</file context>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 GPT-6.2 Sol responding on behalf of Lukas Wallrich

Changes under static/ or to non-bundle assets under assets/ and page bundles under content/ now trigger the full browser suite.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 6 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread .github/workflows/rebrand-preview.yaml Resolved
Comment thread .github/workflows/rebrand-preview.yaml Resolved
Comment thread scripts/rebrand/check_design.py Resolved

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 3 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name=".github/workflows/README.md">

<violation number="1" location=".github/workflows/README.md:64">
P3: This summary overstates the PR-base path: pushes and full-mode PRs do not build a PR base, and those browser jobs use the full suite rather than a PR-base HTML diff. Qualify these statements to scoped same-repository PRs.</violation>
</file>

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

- Compares each build with its merge base with `main`: URL list, structured data, SEO tags and internal links. See `scripts/rebrand/README.md`
- A failed check marks the run red but still updates the preview
- After the build, the comparison, the designed-screens check and the accessibility check run as three jobs side by side; the designed-screens check runs four browsers at once
- The candidate, PR base and main reference build in parallel from the same pinned production artifacts, including generated Contributors content. Browser checks use the PR-base HTML diff. Changes to static files or non-bundle assets run the full browser suite. The comparison with the merge base with `main` covers the whole site.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: This summary overstates the PR-base path: pushes and full-mode PRs do not build a PR base, and those browser jobs use the full suite rather than a PR-base HTML diff. Qualify these statements to scoped same-repository PRs.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At .github/workflows/README.md, line 64:

<comment>This summary overstates the PR-base path: pushes and full-mode PRs do not build a PR base, and those browser jobs use the full suite rather than a PR-base HTML diff. Qualify these statements to scoped same-repository PRs.</comment>

<file context>
@@ -61,7 +61,7 @@ Features:
 - Compares each build with its merge base with `main`: URL list, structured data, SEO tags and internal links. See `scripts/rebrand/README.md`
 - A failed check marks the run red but still updates the preview
-- PRs build their base commit alongside the candidate with the same artifacts and regenerated Contributors page. The browser checks use the resulting HTML diff. The comparison with the merge base with `main` still covers the whole site.
+- The candidate, PR base and main reference build in parallel from the same pinned production artifacts, including generated Contributors content. Browser checks use the PR-base HTML diff. Changes to static files or non-bundle assets run the full browser suite. The comparison with the merge base with `main` covers the whole site.
 - CSS or JS changes add the core design and axe samples to the changed-page checks. Core design covers the named screens, breakpoints, sample pages, home, floating controls at 390 and 1440 px, and every fourth article in the usual article set.
 - Pushes, PRs labeled `full-checks`, PRs changing this workflow or the check scripts, and diffs over 60 pages run the full browser suite. With no relevant changes, both browser jobs report success without starting Chromium.
</file context>
Suggested change
- The candidate, PR base and main reference build in parallel from the same pinned production artifacts, including generated Contributors content. Browser checks use the PR-base HTML diff. Changes to static files or non-bundle assets run the full browser suite. The comparison with the merge base with `main` covers the whole site.
For scoped same-repository PRs, the candidate, PR base and main reference build in parallel from the same pinned production artifacts, including generated Contributors content; browser checks use the PR-base HTML diff. Full-mode runs use the full browser suite, while the comparison with the merge base with `main` covers the whole site.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant