Repository navigation
Retry gh-pages pushes rejected by concurrent writers - #3146
Conversation
The master docs deploy, the PR preview publish and the clean job each pushed to gh-pages once, so a run that raced another writer failed with "rejected (fetch first)" (3 of ~100 Sphinx runs, two of them master deploys). Route every gh-pages write through .github/scripts/update_gh_pages.sh, which rebuilds its change on the latest branch tip and retries a rejected push. It works on a blobless, shallow clone and only touches the index, so the 6 GB site is never checked out. The clean job only ran when the docs build on close succeeded, so 118 previews of closed PRs (4.7 GB) were left behind. Add a weekly workflow that removes previews of closed and merged PRs.
There was a problem hiding this comment.
🟢 Approval recommended
The review identifies no active issues in the documentation publishing and preview-cleanup changes.
No open issues 🎉
What changed in this PR
This PR makes documentation publishing more resilient to concurrent gh-pages updates and adds routine cleanup of previews for closed pull requests.
Changes:
- Centralizes publishing and removal in a shared script that retries pushes on the latest branch tip without checking out the site.
- Routes existing docs deployment and cleanup steps through the script; republished previews replace their previous contents.
- Adds weekly and manually triggered pruning of closed-PR previews, retaining previews when PR lookups fail.
| File | Description |
|---|---|
.github/workflows/prune-pr-docs.yml |
Adds scheduled and manual cleanup of closed-PR previews. |
.github/workflows/build-sphinx.yml |
Uses the shared script for all docs writes and removes the full-history cleanup checkout. |
.github/scripts/update_gh_pages.sh |
Implements index-only publishing, removal, and pruning with push retries. |
.github/AGENTS.md |
Documents the requirement to route all gh-pages writes through the script. |
Review effort: Balanced
|
View rendered docs @ https://intelpython.github.io/dpnp/index.html |
|
Array API standard conformance tests for dpnp=0.22.0dev1=np2py314h8d9cdd5_16 ran successfully. |
ndgrigorian
left a comment
There was a problem hiding this comment.
A few comments, mostly on how the script behaves when something fails
| # Unlike read-tree --prefix, this doesn't fetch blobs of existing entries | ||
| GIT_INDEX_FILE="${src_index}" git ls-files -s -z | | ||
| sed -z "s|\t|\t${prefix}|" | | ||
| git update-index -z --index-info |
There was a problem hiding this comment.
peaceiris/actions-gh-pages adds a .nojekyll by default, and nothing does that now. It's on gh-pages only because the action left it there, so if it's ever removed or the branch gets recreated, Pages runs Jekyll, drops _static/ and _sources/, and the docs lose their CSS and JS. Could we add an empty one at the root when it's missing? Something like:
if [[ -z "$(git ls-files -- .nojekyll)" ]]; then
git update-index --add --cacheinfo \
"100644,$(git hash-object -w --stdin </dev/null),.nojekyll"
fiThere was a problem hiding this comment.
The docs build already writes it: doc/conf.py enables sphinx.ext.githubpages, which puts an empty .nojekyll into doc/_build/html/, and the script publishes dotfiles too. This PR's own preview has it: pull/3146/.nojekyll. Every master deploy publishes that output to the site root, so the root .nojekyll is added back on each one, with no change while it's there.
The only gap is a recreated branch that gets PR previews before the next master deploy, so I'd rather not add it to the script as well. Does that work for you?
Retry a failed push only when the remote tip has moved, so a failure other than a concurrent writer stops at once instead of after all attempts. Check the arguments before the clone. In prune, check that pull/ exists, let set -e catch a failed listing, warn about each failed pull request lookup and fail when all of them fail. Fix the script path in .github/AGENTS.md.
vlad-perevezentsev
left a comment
There was a problem hiding this comment.
LGTM
Thank you @antonwolfy
Three jobs write to
gh-pages: themasterdocs deploy, the PR preview publish, and thecleanjob that removes a closed PR's preview. Each pushed once with no retry (peaceiris/actions-gh-pagesmakes a singlegit push, and so didclean), so a run that raced another writer failed with! [rejected] gh-pages -> gh-pages (fetch first). This happened in 3 of the ~100 Sphinx runs since 2026-10-26, two of themmasterdeploys, which left the published docs stale until the next merge.This PR routes every
gh-pageswrite through a new.github/scripts/update_gh_pages.sh:publishadds files to the site root (keeping other files, as before) or replacespull/<N>;removedeletes directories;prunedeletespull/<N>for every closed or merged PR.build-sphinx.ymlnow calls the script for all three writes, which dropspeaceiris/actions-gh-pagesand the token-in-URL push inclean.cleanno longer needs a full-history checkout.The new
prune-pr-docs.ymlrunspruneweekly and on manual dispatch. Until #3144,cleanonly ran when the docs build on close succeeded, so 118 previews of closed PRs (4.7 GB of current files) are left ongh-pages. The first prune run will remove them; only numericpull/<N>directories are considered, and a failed PR lookup keeps the preview.Two behaviour differences from the old action:
pull/<N>, so pages removed from the docs disappear (the action kept old files).Tested locally against a scratch remote that allows blobless clones like GitHub:
Also tested against a local mirror of the real
gh-pages: republishing a 2,010-file preview with one file changed committed exactly that change, andprunewith live PR states removed exactly the 118 previews of closed PRs and kept the 17 open ones.The
masterdeploy andprunewill first run after merge; this PR's own Sphinx run exercises the preview publish.