Skip to content

CI: upload execution reports from the build that executes notebooks - #660

Open
mmcky wants to merge 1 commit into
mainfrom
ci-execution-reports-path
Open

mmcky wants to merge 1 commit into
mainfrom
ci-execution-reports-path

Conversation

@mmcky

@mmcky mmcky commented Nov 11, 2026

Copy link
Copy Markdown
Contributor

When a notebook fails to execute in CI, the execution-reports artifact that should hold its traceback comes up empty. This mirrors QuantEcon/lecture-python-advanced.myst#410, which found the same mismatch there.

Cause

Notebooks execute in whichever jb build runs them first, and each build writes its execution reports under its own output directory. Since #588, the tojupyter build ("Build Download Notebooks") is the first build and the gating step. So a CellExecutionError stops the job there, and its traceback is written only to _build/jupyter/reports. The upload step looks in _build/html/reports, which the HTML build never reaches to create.

This workflow doesn't upload _build when a build fails, so in practice the traceback of a failing notebook is lost, and the only way to see it is to reproduce the failure locally.

I confirmed where the reports land with a throwaway two-page book whose only notebook raises. I built it with this workflow's tojupyter command (jb build lectures --path-output ./ --builder=custom --custom-builder=jupyter -n -W --keep-going), using jupyter-book 1.0.4.post1 and myst-nb 1.4.0. The traceback went to _build/jupyter/reports/broken.err.log, and no _build/html/reports directory was created.

Change

The upload step now lists the reports directories of all three builds: _build/jupyter/reports, _build/latex/reports and _build/html/reports. Whichever build fails, its report is captured. upload-artifact roots a multi-path upload at the common ancestor (_build), so the artifact keeps the sets apart as jupyter/reports/… and so on. I also added a comment explaining the paths. This matches the change in QuantEcon/lecture-python-advanced.myst#410.

Testing

This PR's own CI run won't exercise the step, because it runs only if: failure(). I checked that the workflow parses and that path is a valid multi-line list. The first failing notebook on any PR will confirm the upload.

🤖 Generated with Claude Code

The "Upload Execution Reports" step only looked in _build/html/reports.
Since #588 made the tojupyter build the gating step, a CellExecutionError
stops the job in that build, which writes its reports to
_build/jupyter/reports, so the execution-reports artifact came up empty.
Upload the reports directories of all three builds.

Mirrors QuantEcon/lecture-python-advanced.myst#410.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings November 11, 2026 01:19

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Copilot was unable to complete this review at the selected tier due to model policy or plan restrictions.

🟢 Approval recommended

The change is a small, well-scoped CI workflow adjustment with clear documentation that broadens failure-only artifact collection across known Jupyter Book output paths, and the review evidence reports no outstanding issues.

No open issues 🎉

What changed in this PR

Updates CI artifact uploading so notebook execution tracebacks are preserved regardless of which jb build step runs notebooks first, preventing empty execution-reports artifacts when failures occur early.

Changes:

  • Upload execution reports from multiple build output directories (_build/jupyter/reports, _build/latex/reports, _build/html/reports) on failure.
  • Add an inline comment documenting why reports may be produced under different _build/*/reports paths.
File Description
.github/​workflows/​ci.yml Adjusts the failure-only artifact upload to collect execution reports from all Jupyter Book build output roots so failing notebook tracebacks are captured consistently.

Review effort: Balanced

@github-actions

Copy link
Copy Markdown

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants