[marcleblanc2]

Stacked on #1468. CI passed while checking less than it looked like: it validated one Compose file out of ten, never ran the executor test, and counted containers against a hard-coded 25. This PR closes those gaps.

Gap	Fix
db-only-migrate.docker-compose.yaml was a hand-copied pgsql and codeintel-db, with a "MUST be kept in sync" comment. It had drifted: 2g memory instead of 4g, no shm_size, no /conf mount, shorter health check and stop grace period.	The file now extends both services from docker-compose.yaml, so it cannot drift.
CI ran docker-compose config on docker-compose.yaml only.	validate-docker-compose.sh runs docker compose config on every Compose file, combined the way its docs say to run it: db-only, jaeger, dev, executor (with the base file, and standalone), and each examples/*/docker-compose.override.yaml.
The standalone command in executors/README.md fails on Compose v2: service "executor" refers to undefined network sourcegraph.	executor.docker-compose.yaml declares the network.
test/executor-smoke-test.sh was never run by CI.	The smoke test VM runs it after smoke-test.sh. The VM image has no jq, so the job installs it with apt-get first, from the Ubuntu repositories only: the image lacks the key for its Chrome repository, so a full apt-get update fails.
smoke-test.sh hard-coded expect_containers="25".	It counts the services in docker-compose.yaml, minus migrator, which exits when it is done.
"Checking every 10s that containers are running for 5 minutes" checked twice, one second apart.	Removed. The count check now counts running containers, and #1468's restart check stays.
Every job on an agent host shared /tmp/e2e-builder.json, and never refreshed or removed it.	Each job fetches the key into its own mktemp file, and removes it when the job ends.
The Vagrantfile appended the Docker Hub password to /root/.profile on the VM.	The credentials reach the provisioner as environment variables.
batch-change/release.yaml still had the bare git tag, which fails when a finalize job is retried.	Same retry-safe block as release.yaml.
No lint for shell scripts.	New :bash: step runs shellcheck on every *.sh and the Buildkite hook. shellcheck 0.11.0 is added to .tool-versions.
dev/run.sh used docker-compose; the README uses docker compose.	docker compose.

Things to know

• Behavior change for backup and restore. Customers who start the databases with db-only-migrate.docker-compose.yaml now get the same pgsql and codeintel-db as the full deployment: mem_limit 4g (was 2g), shm_size 1g, the ../<db>/conf:/conf/ mount, health check retries: 10 and start_period: 1h (was 3 and 15s), stop_grace_period: 120s.
• extends works on Compose 1.29.2, 2.18.1, 2.24.1, and 5.7.0. The release tooling skips services without an image: key, so it leaves the file alone.
• The validation step uses docker compose (the plugin) on the Buildkite agent, and prints its version: 2.18.1. That version accepts the !override and !reset tags in three examples (aws-alb, aws-rds-databases, external-databases), but ignores them. So CI checks that those examples parse, not that the merged result is what a customer on a current Compose gets. Compose 2.24.1 rejects !override; 5.7.0 handles both tags.
• executor.docker-compose.yaml pins executor:6.0.0, while every other image is 8.1.x. The release tooling has not been updating it. It still registers with the 8.1 frontend in CI. Not changed here.
• The Docker Hub password still ends up in the VM's Docker config, because the smoke test runs docker login. The Vagrantfile change removes a second copy and nine lines; it does not make the VM hold fewer secrets.
• smoke-test.sh keeps using docker-compose (v1), because that is what the VM image ships.

Not in this PR

• A pipeline gate which stops release finalize jobs when the smoke test did not run. Builds of non-release commits on release branches are cancelled by newer commits, and --config-from-commit already fails closed.
• Prettier for YAML and Markdown. It would reformat most of the repo.
• catch_errors in smoke-test.sh lists containers with docker ps, which hides exited containers. Adding --all would flag migrator, which is meant to exit.

Checklist

• All images have a valid tag and SHA256 sum (no image changes)

Test plan

• docker compose config on every file and combination in the new validation script, with Compose 5.7.0 (all pass), 2.24.1 (fails only the two !override examples), and 2.18.1 (all pass)
• Resolved pgsql and codeintel-db from the db-only file and from docker-compose.yaml on Compose 1.29.2, 2.18.1, 2.24.1, and 5.7.0: identical on each
• docker-compose config --services on 1.29.2 lists 25 services other than migrator, the number the test used to hard-code
• Ran .buildkite/vagrant-run.sh against stubbed vagrant, gcloud, and curl: the key file is removed in every case; the test's exit code is kept; a rejected delete request falls back to vagrant destroy; a failed vagrant destroy fails the job
• shellcheck 0.11.0 is clean on every script; ruby -c test/Vagrantfile; every edited YAML file parses
• The two git:tag blocks in release.yaml and batch-change/release.yaml are identical
• Buildkite builds 16830 and 16837 (after the rebase onto ci: Speed up the smoke test #1468) passed: all 11 docker compose config calls; shellcheck, installed by mise from .tool-versions; Login Succeeded with the credentials passed as provisioner environment variables; 25 running containers against 25 expected; jq installed; the executor registered after 10s. The smoke test job took about 290s, executor test included
• Build 16825 failed on the full apt-get update, which is why the install skips /etc/apt/sources.list.d

[github-actions]

Open the Diff Tour for this PR in Sourcegraph

[marcleblanc2]

Closing in favor of #1489, which validates every Compose file and runs shellcheck. The branch stays.