[marcleblanc2]

Lets agents and people test changes to this repo's Docker Compose deployment in Amp orbs. A thread in the sourcegraph/deploy-sourcegraph-docker Amp project can start Sourcegraph, Jaeger, and an executor from the Compose files it's changing, behind a portal URL, with a site admin already created. To develop Sourcegraph itself, sg is still the tool.

Risk: none for customers. Nothing under docker-compose/ changes; the new files only take effect in an Amp orb.

Files

• .agents/setup: starts Docker, installs the Compose plugin if missing, and adds orb guidance to the orb's AGENTS.md. Pulls no images, so the snapshot holds no build artifacts.
• .amp/services.yaml: declares the sourcegraph service, its Executors and Jaeger portal links, and a health check that passes once the site admin exists.
• .amp/sourcegraph.sh: starts the containers, creates the admin from the project env vars SOURCEGRAPH_ADMIN_USERNAME, SOURCEGRAPH_ADMIN_EMAIL, and SOURCEGRAPH_ADMIN_PASSWORD, sets five site config keys when it has no executor token, an older one, or another portal URL (externalURL, executors.accessToken, executors.frontendURL, codeIntelAutoIndexing.enabled, observability.tracing), prunes unused images when the deployment's images change, then follows the logs. It also uses docker-compose/docker-compose.override.yaml when it exists, and stops the containers when it fails or stops.
• .amp/docker-compose.override.yaml: Caddy publishes Amp's port and port 80; the executor gets its token and all three queues; both zoekt services drop to 4 CPUs so a 4 vCPU orb works.
• .amp/Caddyfile: caddy/builtins/http.Caddyfile minus Sec-Fetch-Dest and X-Frame-Options, so Sourcegraph loads in the Portal tab's iframe.

Worth a look

• .amp/Caddyfile turns off Sourcegraph's clickjacking protection in the orb.
• EXECUTOR_DOCKER_ADD_HOST_GATEWAY=true lets job containers reach the orb's ports; they need it to reach the frontend at http://host.docker.internal, since the portal URL is behind Amp's sign-in.
• The admin password is a project env var, not a secret, so the agent can give it to the user. It also shows in the portal note. Only people who can open the thread can open its portal.
• Writing the site config back drops its comments. That only happens on the first start, or when the executor token or portal URL changed.
• The first start pulls 8.3 GB of images, in 54 to 66 s. Past Amp's 60 s port check, amp orb services ensure exits 1, but the service keeps going and a second ensure exits 0.

Checklist

• All images have a valid tag and SHA256 sum (no image changes)

Test plan

In a 16 vCPU, 31 GB orb with the 8.2.0 images, from no containers, images, or volumes:

• amp orb services ensure exits 0; 27 of 27 containers run, with no restarts.
• The admin signs in through the portal, also inside a cross-origin iframe.
• The executor is active on batches, batches-hooks, and codeintel. A server-side batch change and an auto-indexing job complete, and precise code navigation works on the index.
• A search with ?trace=1 shows in Jaeger.
• Restarts keep the admin, the site config ID, and settings added by hand. Removing the executor token puts it back.
• Also works in an 8 vCPU orb, and in a 4 vCPU orb at an earlier commit with the same CPU change.
• shellcheck is clean. Buildkite build 16951 passed.

Not tested: a batches-hooks job (needs published changesets), and a site config the script can't parse.

At b0c18d8, in a fresh 16 vCPU orb (thread), all of the above passed again, plus: a restart doesn't re-run site-init, rewrite the site config, or prune images, and the job images survive it; a failed start stops every container; deleting the executor token gets a new one into the site config; a changed image list prunes once, and docker-compose/docker-compose.override.yaml is picked up; an unused volume survives restarts; setup run 4 times leaves one guidance section. feebb26 fixes the two problems that run found, tested in the same orb: the health check waits for the site admin (/healthz passed about 2 s early), and a wrong password now logs sign-in's "Authentication failed".

Closes FE-600

[github-actions]

Open the Diff Tour for this PR in Sourcegraph

[michaellzc]

cool, but can you clarify the use case? I don't think the goal is to get a running sourcegraph (that's what sg/sg is for), but to assist with development the docker compose distro. you might want to put some wording around that in agents.md

[marcleblanc2]

cool, but can you clarify the use case? I don't think the goal is to get a running sourcegraph (that's what sg/sg is for), but to assist with development the docker compose distro. you might want to put some wording around that in agents.md

@michaellzc yes, the use case is to test and verify changes to the Docker Compose deployment files, which requires a running Sourcegraph instance, Jaeger, and Executors to ensure that changes to the deployment files don't break anything, eg. ensuring a Precise auto-indexing job, and a Batch Change job can complete on the Executor.

Will add to the agents file.