Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 77 additions & 0 deletions .github/workflows/build-kernel.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
name: Build kernel and wheel
# Rebuild the two binaries the site and the cppjit CI job install from a release of this repository: the browser kernel
# (xeus-cpp + cppjit + CppInterOp with Clad on wasm32, as kernel-overlay-<tag>.tar.gz) and the native cppjit wheel with
# Clad inside. Everything comes from tools/build-wasm-kernel.sh, the locks and the patches in the repository; the runner's
# 16 GB cover binaryen's wasm-opt pass (about 11.5 GB). About an hour. The digests go back as a pull request, because the
# Pages and cppjit jobs verify the assets against lite/kernel/SHA256SUMS in the repository, not against a file beside them.
on:
workflow_dispatch:
inputs:
tag:
description: release tag to create or update (kernel-YYYY-MM-DD)
required: true
default: kernel-2026-11-03
permissions:
contents: write
pull-requests: write
jobs:
build:
runs-on: ubuntu-24.04
timeout-minutes: 300
env:
SP: ${{ github.workspace }}/.kernel-build
steps:
- uses: actions/checkout@v4
- name: micromamba (pinned, digest checked, the same as tools/setup-lite-env.sh)
run: |
mkdir -p "$SP/bin"
curl -sSL -o "$SP/bin/micromamba" https://github.com/mamba-org/micromamba-releases/releases/download/2.9.0-0/micromamba-linux-64
(cd "$SP/bin" && echo "366cd9cd8be14df1ab8ed50352a82111082a36686b2d389fdb79a92c3fafb3e3 micromamba" | sha256sum -c -) && chmod +x "$SP/bin/micromamba"
- name: emsdk 4.0.9
run: |
git clone -q --depth 1 https://github.com/emscripten-core/emsdk.git "$SP/emsdk"
(cd "$SP/emsdk" && ./emsdk install 4.0.9 >/dev/null && ./emsdk activate 4.0.9 >/dev/null)
echo "EMSDK=$SP/emsdk" >> "$GITHUB_ENV"
- name: Environments (wasm32 from the lock, native LLVM 23 tools)
run: bash tools/build-wasm-kernel.sh envs
- name: Sources at the pins, patches applied
run: bash tools/build-wasm-kernel.sh sources
- name: Kernel (CppInterOp with Clad, cppjit, xeus-cpp; linked at -O1)
run: bash tools/build-wasm-kernel.sh build
- name: wasm-opt -Oz (binaryen, about 11.5 GB)
run: bash tools/build-wasm-kernel.sh wasm-opt
- name: Kernel overlay
run: bash tools/build-wasm-kernel.sh overlay "${{ inputs.tag }}"
- name: cppjit wheel with Clad
run: bash tools/build-wasm-kernel.sh wheel
- name: Digests
run: |
mkdir -p out && cp "$SP"/kernel-overlay-*.tar.gz "$SP"/cppjit-*.whl out/
(cd out && sha256sum kernel-overlay-*.tar.gz cppjit-*.whl | tee SHA256SUMS)
- uses: actions/upload-artifact@v4
with:
name: kernel-and-wheel-${{ inputs.tag }}
path: out/
- name: Release assets
if: ${{ !env.ACT }}
env:
GH_TOKEN: ${{ github.token }}
run: |
gh release view "${{ inputs.tag }}" --repo "$GITHUB_REPOSITORY" >/dev/null 2>&1 \
|| gh release create "${{ inputs.tag }}" --repo "$GITHUB_REPOSITORY" --title "Browser kernel and cppjit wheel with Clad, ${{ inputs.tag }}" --notes "Built by the Build kernel and wheel workflow from tools/build-wasm-kernel.sh; see lite/kernel/README.md."
gh release upload "${{ inputs.tag }}" --repo "$GITHUB_REPOSITORY" --clobber out/kernel-overlay-*.tar.gz out/cppjit-*.whl out/SHA256SUMS
- name: Pull request with the new digests
if: ${{ !env.ACT }}
env:
GH_TOKEN: ${{ github.token }}
run: |
cp out/SHA256SUMS lite/kernel/SHA256SUMS
sed -i "s/KERNEL_RELEASE: kernel-[0-9-]*/KERNEL_RELEASE: ${{ inputs.tag }}/" .github/workflows/deploy-pages.yml
sed -i "s/gh release download kernel-[0-9-]* /gh release download ${{ inputs.tag }} /" .github/workflows/check-notebooks.yml
git config user.name "github-actions[bot]"; git config user.email "github-actions[bot]@users.noreply.github.com"
git checkout -b "digests/${{ inputs.tag }}"
git commit -am "Kernel and wheel digests for release ${{ inputs.tag }}"
git push -u origin "digests/${{ inputs.tag }}"
gh pr create --repo "$GITHUB_REPOSITORY" --base "${{ github.ref_name }}" --head "digests/${{ inputs.tag }}" \
--title "Kernel and wheel digests for ${{ inputs.tag }}" \
--body "Built by the Build kernel and wheel workflow (run ${{ github.run_id }}); the assets are on release ${{ inputs.tag }}. Merging this makes the Pages and cppjit jobs use them."
39 changes: 39 additions & 0 deletions .github/workflows/check-notebooks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -122,3 +122,42 @@ jobs:
name: executed-cuda-notebooks
path: executed/
if-no-files-found: ignore

execute-notebooks-cppjit:
name: cppjit
runs-on: ubuntu-latest
defaults:
run:
shell: bash -el {0}
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up micromamba environment
uses: mamba-org/setup-micromamba@v2
with:
environment-file: environment.yml
cache-environment: true
- name: List available kernels
run: jupyter kernelspec list
- name: cppjit wheel with Clad from the release (digest checked against lite/kernel/SHA256SUMS)
# Under nektos/act (ACT=true) the wheel is expected in lite/kernel-artifacts already; see lite/README-build.md.
env:
GH_TOKEN: ${{ github.token }}
run: |
mkdir -p lite/kernel-artifacts
if [ -z "$ACT" ]; then
gh release download kernel-2026-11-03 --repo "$GITHUB_REPOSITORY" --dir lite/kernel-artifacts --pattern 'cppjit-*.whl'
fi
(cd lite/kernel-artifacts && sha256sum -c --ignore-missing ../kernel/SHA256SUMS)
$CONDA_PREFIX/bin/pip install lite/kernel-artifacts/cppjit-*.whl
- name: Run cppjit notebook tests (Python kernel, C++ compiled by cppjit)
run: |
mkdir -p executed
$CONDA_PREFIX/bin/pytest tests/test_notebooks.py::CppjitNotebookTests -sv
- name: Upload executed cppjit notebooks as artifact
if: always()
uses: actions/upload-artifact@v4
with:
name: executed-cppjit-notebooks
path: executed/
if-no-files-found: ignore
71 changes: 71 additions & 0 deletions .github/workflows/deploy-pages.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
name: Pages
# Build the JupyterLite site (the C++ series and the cppjit series running in the browser) and publish it on GitHub Pages.
# The kernel binaries come from a release of this repository (see lite/kernel/README.md); everything else is rebuilt here.
on:
push:
branches: [master]
paths: ['cppjit/**', 'xeus-cpp/**', 'lite/**', 'tools/**', '.github/workflows/deploy-pages.yml']
workflow_dispatch:
permissions:
contents: read
pages: write
id-token: write
concurrency:
group: pages
cancel-in-progress: false
env:
KERNEL_RELEASE: kernel-2026-11-03
jobs:
build:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.12'
- name: Kernel binaries from the release
# Under nektos/act (ACT=true) the assets are expected in lite/kernel-artifacts already; see lite/README-build.md.
if: ${{ !env.ACT }}
env:
GH_TOKEN: ${{ github.token }}
run: |
mkdir -p lite/kernel-artifacts
gh release download "$KERNEL_RELEASE" --repo "$GITHUB_REPOSITORY" --dir lite/kernel-artifacts \
--pattern 'kernel-overlay-*.tar.gz'
- name: Verify the kernel binaries against the digests committed in lite/kernel/SHA256SUMS
run: (cd lite/kernel-artifacts && sha256sum -c --ignore-missing ../kernel/SHA256SUMS) && ls -la lite/kernel-artifacts
- name: Build environment (micromamba, host venv, wasm32 env from the lock, kernel overlay)
run: SP="$RUNNER_TEMP/lite" bash tools/setup-lite-env.sh
- name: Build the site
run: SP="$RUNNER_TEMP/lite" OUT="$PWD/lite/dist" bash tools/build-lite-site.sh
- name: The lazy packages are all there
run: test "$(ls lite/dist/lazy/*.tar.gz | wc -l)" -eq "$(grep -v '^#' lite/env/lazy-remove.txt | wc -w)"
- name: Smoke test in a headless browser (cppjit notebooks 1 and 3, the second one with SciPy, matplotlib and a figure)
run: |
"$RUNNER_TEMP/lite/site-venv/bin/pip" install -q playwright==1.63.0
"$RUNNER_TEMP/lite/site-venv/bin/playwright" install --with-deps chromium
# serve one level up so the site sits under /dist/, a sub-path like the Pages URL, not the domain root
(cd lite && python3 -m http.server 8942 >/dev/null 2>&1 &)
sleep 2
for nb in 1_call_it 3_fit_it; do
python3 - "$nb" <<'PY'
import json, sys; nb = json.load(open(f"lite/content/cppjit/{sys.argv[1]}.ipynb"))
json.dump([''.join(c['source']) for c in nb['cells'] if c['cell_type'] == 'code'], open("cells.json", "w"))
PY
"$RUNNER_TEMP/lite/site-venv/bin/python" tools/lite_run.py --site http://127.0.0.1:8942/dist --cells cells.json --timeout 600 | tee run.log
grep -q '"text": "Traceback' run.log && exit 1 || true
done
- uses: actions/upload-pages-artifact@v3
if: ${{ !env.ACT }}
with:
path: lite/dist
deploy:
needs: build
if: ${{ !github.event.act }}
runs-on: ubuntu-latest
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
steps:
- id: deployment
uses: actions/deploy-pages@v4
8 changes: 8 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
lite/dist/
lite/content/
lite/labext/
lite/lazy-packages/
lite/kernel-artifacts/
lite/.jupyterlite.doit.db
executed/
__pycache__/
20 changes: 20 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,29 @@ An introduction to modern C++ through live, interactive examples. Topics include
### OpenMP (`openmp/`)
CPU parallelism using OpenMP pragmas. Covers `#pragma omp parallel`, loop-level parallelism, reductions, and thread synchronization using the OpenMP-enabled `xeus-cpp` kernel. The notebooks guide you step-by-step from foundational concepts like spawning threads and managing shared vs. private data variables, all the way to advanced patterns like task-based parallelism, critical sections, and optimizing performance loops directly in an interactive environment.

### cppjit (`cppjit/`)
The other direction: a Python kernel with a C++ compiler inside it. `cppjit` carries Clang and LLVM, reads a C++ header
the way a compiler does, and lets Python call what it finds. Five notebooks from the SC26 tutorial: call a header from
Python, use the derivative Clad wrote for it, fit a C++ simulator with SciPy on that gradient, fit the Higgs mass on CMS
open data without ROOT, and calibrate a sixty-line tumour twin. one wheel is the whole setup (the release's `cppjit-*.whl`, Clad inside; see `cppjit/README.md`).

### CUDA (`cuda/`)
GPU programming from first principles using CUDA C++. This section explores how to accelerate compute-heavy workloads by utilizing graphics cards. It covers essential GPU architecture concepts, kernel declarations, memory allocation (`cudaMalloc`), host-to-device data transfers, and thread organization across grids and blocks to give you hands-on experience writing raw parallel code optimized for parallel hardware.

## Licences

This repository is Apache-2.0. The cppjit notebooks use [Clad](https://github.com/vgvassilev/clad) (LGPL-3.0) through
the compiler they run on: the browser kernel and the cppjit wheel of this repository's release carry it inside
CppInterOp; nothing of Clad is kept here.

## In the browser

The C++ and cppjit series also run with nothing installed, on GitHub Pages:
https://compiler-research.org/live-cpp-tutorials/ . The kernel is xeus-cpp compiled to WebAssembly (LLVM 23), hosting
Python through cppjit for the cppjit series, with Clad built in: a C++ cell can call `clad::gradient` and a Python cell
can do the same through `cppjit.cppdef`. The first visit downloads the compiler once (about 42 MB), after that
everything runs in the tab. OpenMP and CUDA need the native kernels below. How the site is built: `lite/README-build.md`.

## Installation
To ensure a clean setup, install `live-cpp-tutorials` inside a fresh environment using Micromamba.

Expand Down
Loading
Loading