[marcleblanc2]

Open issues

Found in review on 2026-11-13 (review thread), at 05bb302. The labels work as described; this PR is a draft until the first three are settled.

1. A user's own app.kubernetes.io/instance label leaves both Jaeger Services with no endpoints. main pins that label on the Jaeger pod to the release name; this branch lets sourcegraph.labels, sourcegraph.podLabels, and jaeger.podLabels replace it, and the Services still select the release name. Reproduced on EKS with jaeger.enabled: true and jaeger.podLabels."app.kubernetes.io/instance": prod: jaeger-collector and jaeger-query went from 1 endpoint each to 0. Fix: merge the release name last in the pod's labels, and add a test
2. The executor Service deletion and the kubernetes-services job removal, both taken from fix/resources: Drop unused ports, Services, Role verbs, and settings #1016, are in neither changelog nor this description. "No object is recreated" below is true, but the upgrade deletes the executor-<queueName> Service, and executor.serviceAnnotations, serviceLabels, and serviceType stop working. Fix: merge fix/resources: Drop unused ports, Services, Role verbs, and settings #1016 first, or copy its changelog entries
3. "$@" is the wrong advice for one form of custom collector command. With the script in openTelemetry.gateway.args after command: ["/bin/sh", "-c"], the form tests/command_test.yaml uses, the chart's argument arrives as $0 and "$@" is empty, so the collector keeps serving metrics on 127.0.0.1. With the script inside command and a placeholder after it, "$@" works: that is what the test instance runs. Fix: document "$0" "$@" for the first form in values.yaml and the changelog
4. otel-collector restarts on a config change only when openTelemetry.gateway.config.traces is set. The checksum annotation is absent with default values, so "otel-collector now restarts when the chart changes its configMap content" overstates it. prometheus gets its annotation in every case
5. Small ones:
   • PVCs made from volumeClaimTemplates still get no labels, on new installs too
   • the EKS results under "Testing" are from 4dc65cf, which is no longer on the branch; the run below repeats most of them on the current head

Tested in this review on the EKS test instance m5, by upgrading from main to this branch merged onto main:

• 83 objects before and 82 after: only the executor Service was removed. No UID, Service IP, or PVC volume changed, and every pod was replaced once
• Every object Helm manages, and every pod template, has the labels
• A chart-version-only upgrade replaced 0 pods
• The otel-collector target is up, with the instance's own sh -c command
• All 28 pods Ready, and search and an executor batch spec work

Not re-checked: which labels metrics gained and lost, helm test, and the two dind charts on a cluster.

Problem

• Customer needs the ability to label all resources; Add opt-in selector labels to adopt existing workloads #1007 only went part way

• Many objects didn't render sourcegraph.labels (PVCs, ServiceAccounts, RBAC objects, Services, ConfigMaps, Secrets, the Ingress), and some had no per-component labels at all

Solution

All four charts now render these labels on every object:

• sourcegraph.labels, the standard Helm labels, deploy, and app.kubernetes.io/component
• Per-component labels, empty by default:
  • <serviceName>.labels now also covers the component's ConfigMaps, Secrets, and PodDisruptionBudget
  • New: <serviceName>.storageLabels (PVCs), <serviceName>.serviceAccount.labels (ServiceAccounts and RBAC), storageClass.labels, priorityClasses[].labels
  • Executor charts: executor.labels now also covers the ConfigMaps, Secret, ServiceAccount, and RBAC objects, and new privateDockerRegistry.serviceLabels labels the dind registry Service

On upgrade:

• No object is recreated, and no selector changes
• Pods don't get helm.sh/chart or app.kubernetes.io/version, so a chart-only change doesn't restart every pod
• A new Prometheus labeldrop rule keeps unused labels off metrics
  • Customers using prometheus.existingConfig can add this rule themselves
• prometheus and otel-collector now restart when the chart changes their configMap content

Issues found along the way, and fixed:

• jaeger.serviceAccount.create: true failed to render
• redisStore.serviceAccount.annotations was dropped
• The OpenTelemetry Collector "pods available" panel and alert had no data
  • The otel-collector Service gets an app label, and the collector serves its metrics on all interfaces through one extra --config argument
  • The argument also goes to a custom openTelemetry.gateway.command, which has to pass "$@" through
• Prometheus couldn't scrape jaeger, so its scrape annotations are removed, as fix/resources: Drop unused ports, Services, Role verbs, and settings #1016 does

Open question: the --config argument replaces service.telemetry.metrics.readers in any config file a custom command loads. Should the readers be a Helm value instead?

Testing

• helm unittest and helm lint pass on all four charts. New tests render every template, and fail on any object missing the labels
• Add opt-in selector labels to adopt existing workloads #1007's golden snapshot only gained the two pod labels. No selector or volume claim template changed
• On EKS (27 pods, Helm 4, bundled databases, k8s executor), at 4dc65cf:
  • Upgrade: 0 objects recreated, no Service IP or PVC volume changed, each pod replaced once
  • Chart-version-only upgrade and back: 0 pods replaced
  • Metrics lost only app_kubernetes_io_managed_by and gained nothing. No loaded rule or dashboard uses a dropped label
  • otel-collector targets went from down to up, and the "pods available" alert from firing to inactive
  • helm test, search, an executor batch spec, and jaeger.enabled work

Amp threads

• Design and implementation
• EKS upgrade, metrics, and label tests
• Matching the removals in #1016

Changelog

• sourcegraph.labels, the standard Helm labels, deploy, and app.kubernetes.io/component are now on every object in all four charts. Existing releases gain them on upgrade
• New per-component labels: storageLabels, serviceAccount.labels, storageClass.labels, priorityClasses[].labels, and privateDockerRegistry.serviceLabels. <serviceName>.labels and executor.labels now cover more objects
• If using prometheus.existingConfig, add the new labeldrop rule to avoid new labels in metrics
• Fixed jaeger.serviceAccount.create, redisStore.serviceAccount.annotations, the OpenTelemetry Collector "pods available" alert

[github-actions]

Open the Diff Tour for this PR in Sourcegraph