diff --git a/charts/sourcegraph-executor/CHANGELOG.md b/charts/sourcegraph-executor/CHANGELOG.md index 5e2f4fb18..3aa5f4a35 100644 --- a/charts/sourcegraph-executor/CHANGELOG.md +++ b/charts/sourcegraph-executor/CHANGELOG.md @@ -3,6 +3,9 @@ ## Unreleased +- Every object the charts create now has the `deploy` and `app.kubernetes.io/component` labels: the `sourcegraph-executor-k8s` Secret did not have them. `executor.labels` is now also rendered on the `sourcegraph-executor-k8s` ConfigMap, Secret, ServiceAccount, Role, and RoleBinding, and on the `sourcegraph-executor-dind` `docker-config` ConfigMap, not only on the Deployment. Added `privateDockerRegistry.serviceLabels` to the `sourcegraph-executor-dind` chart to label the `private-docker-registry` Service. +- `sourcegraph.labels` and the charts' standard labels (`helm.sh/chart`, `app.kubernetes.io/name`, `app.kubernetes.io/instance`, `app.kubernetes.io/version`, `app.kubernetes.io/managed-by`) are now rendered on every object the charts create. The `sourcegraph-executor-k8s` chart rendered them on none of its objects (ConfigMap, Deployment, Role, RoleBinding, Secret, Service, ServiceAccount), and the `sourcegraph-executor-dind` chart did not render them on the `docker-config` ConfigMap and the `private-docker-registry` Service. Pods get them too, except `helm.sh/chart` and `app.kubernetes.io/version`, which change with every chart version and would restart every pod on every upgrade. The `private-docker-registry` pod gains `app.kubernetes.io/component`. Existing releases gain these labels on upgrade. No object is recreated. Every pod restarts once, on this upgrade. + ## 8.0.0 * Added a `network-policy` example to the `sourcegraph-executor-k8s` and `sourcegraph-executor-dind` charts, with `CiliumNetworkPolicy` resources which limit Executor and job pods to the frontend API. diff --git a/charts/sourcegraph-executor/dind/README.md b/charts/sourcegraph-executor/dind/README.md index 14bff60a9..29726ab31 100644 --- a/charts/sourcegraph-executor/dind/README.md +++ b/charts/sourcegraph-executor/dind/README.md @@ -90,6 +90,7 @@ In addition to the documented values, the `executor` and `private-docker-registr | privateDockerRegistry.image.registry | string | `"index.docker.io"` | | | privateDockerRegistry.image.repository | string | `"registry"` | | | privateDockerRegistry.image.tag | int | `3` | | +| privateDockerRegistry.serviceLabels | object | `{}` | Add extra labels to the `private-docker-registry` Service | | privateDockerRegistry.storageSize | string | `"10Gi"` | | | queues | list | `[]` | Optional list of queues to deploy as standalone Deployments. When set, the single executor Deployment is not rendered. Each entry supports: name (required) — used as the deployment name suffix (executor-) queueName — sets EXECUTOR_QUEUE_NAME; defaults to name if omitted queueNames — sets EXECUTOR_QUEUE_NAMES (comma-joined); takes precedence over queueName when set replicaCount, resources, env (merged with executor.env, queue overrides) | | sourcegraph.affinity | object | `{}` | Affinity, learn more from the [Kubernetes documentation](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity) | diff --git a/charts/sourcegraph-executor/dind/templates/_helpers.tpl b/charts/sourcegraph-executor/dind/templates/_helpers.tpl index 8b87b5d24..1143009eb 100644 --- a/charts/sourcegraph-executor/dind/templates/_helpers.tpl +++ b/charts/sourcegraph-executor/dind/templates/_helpers.tpl @@ -12,29 +12,6 @@ Create chart name and version as used by the chart label. {{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }} {{- end }} -{{/* -Common labels -*/}} -{{- define "sourcegraph.labels" -}} -helm.sh/chart: {{ include "sourcegraph.chart" . }} -{{ include "sourcegraph.selectorLabels" . }} -{{- if .Chart.AppVersion }} -app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} -{{- end }} -app.kubernetes.io/managed-by: {{ .Release.Service }} -{{- if .Values.sourcegraph.labels }} -{{ toYaml .Values.sourcegraph.labels }} -{{- end }} -{{- end }} - -{{/* -Selector labels -*/}} -{{- define "sourcegraph.selectorLabels" -}} -app.kubernetes.io/name: {{ include "sourcegraph.name" . }} -app.kubernetes.io/instance: {{ .Release.Name }} -{{- end }} - {{/* Create the name of the service account to use */}} @@ -98,7 +75,6 @@ tolerations: {{- end }} {{- end }} - {{- define "executor.name" -}} {{- if .Values.executor.queueName -}} executor-{{.Values.executor.queueName}} @@ -107,13 +83,6 @@ executor-{{join "-" .Values.executor.queueNames }} {{- end }} {{- end }} -{{- define "executor.labels" -}} -app: {{ include "executor.name" . }} -deploy: sourcegraph -sourcegraph-resource-requires: no-cluster-admin -app.kubernetes.io/component: executor -{{- end}} - {{- define "dind.daemonConfig" -}} {{- $config := .Values.dind.daemonConfig | deepCopy }} {{- if .Values.dind.gVisor.enabled }} @@ -169,13 +138,7 @@ spec: {{- toYaml $r.Values.executor.podAnnotations | nindent 8 }} {{- end }} labels: - {{- include "sourcegraph.selectorLabels" $r | nindent 8 }} - {{- if $r.Values.sourcegraph.podLabels }} - {{- toYaml $r.Values.sourcegraph.podLabels | nindent 8 }} - {{- end }} - {{- if $r.Values.executor.podLabels }} - {{- toYaml $r.Values.executor.podLabels | nindent 8 }} - {{- end }} + {{- include "sourcegraph.podLabels" (list $r $r.Values.executor) | nindent 8 }} app: {{ .name }} deploy: sourcegraph sourcegraph-resource-requires: no-cluster-admin diff --git a/charts/sourcegraph-executor/dind/templates/_helpers/_labels.tpl b/charts/sourcegraph-executor/dind/templates/_helpers/_labels.tpl new file mode 100644 index 000000000..f4c0197ef --- /dev/null +++ b/charts/sourcegraph-executor/dind/templates/_helpers/_labels.tpl @@ -0,0 +1,47 @@ +{{/* +Common labels +*/}} +{{- define "sourcegraph.labels" -}} +helm.sh/chart: {{ include "sourcegraph.chart" . }} +{{ include "sourcegraph.selectorLabels" . }} +{{- if .Chart.AppVersion }} +app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} +{{- end }} +app.kubernetes.io/managed-by: {{ .Release.Service }} +{{- if .Values.sourcegraph.labels }} +{{ toYaml .Values.sourcegraph.labels }} +{{- end }} +{{- end }} + +{{/* +Selector labels +*/}} +{{- define "sourcegraph.selectorLabels" -}} +app.kubernetes.io/name: {{ include "sourcegraph.name" . }} +app.kubernetes.io/instance: {{ .Release.Name }} +{{- end }} + +{{/* +Labels of a pod template: the common labels, then sourcegraph.podLabels, then +the component's podLabels, then the selector labels. The later one wins when a +key is set more than once, so each key is rendered once and the selector always +matches the pods. + +`helm.sh/chart` and `app.kubernetes.io/version` are left out: they change with +every chart version, and a changed pod template restarts the pods. + +{{ include "sourcegraph.podLabels" (list . .Values.executor) }} +*/}} +{{- define "sourcegraph.podLabels" -}} +{{- $top := index . 0 -}} +{{- $component := index . 1 -}} +{{- $labels := omit (include "sourcegraph.labels" $top | fromYaml) "helm.sh/chart" "app.kubernetes.io/version" -}} +{{- mergeOverwrite $labels ($top.Values.sourcegraph.podLabels | default dict) ($component.podLabels | default dict) (include "sourcegraph.selectorLabels" $top | fromYaml) | toYaml -}} +{{- end }} + +{{- define "executor.labels" -}} +app: {{ include "executor.name" . }} +deploy: sourcegraph +sourcegraph-resource-requires: no-cluster-admin +app.kubernetes.io/component: executor +{{- end}} diff --git a/charts/sourcegraph-executor/dind/templates/executor/docker-daemon.ConfigMap.yaml b/charts/sourcegraph-executor/dind/templates/executor/docker-daemon.ConfigMap.yaml index ecd96f99b..6ffb86250 100644 --- a/charts/sourcegraph-executor/dind/templates/executor/docker-daemon.ConfigMap.yaml +++ b/charts/sourcegraph-executor/dind/templates/executor/docker-daemon.ConfigMap.yaml @@ -2,6 +2,10 @@ apiVersion: v1 kind: ConfigMap metadata: labels: + {{- include "sourcegraph.labels" . | nindent 4 }} + {{- if .Values.executor.labels }} + {{- toYaml .Values.executor.labels | nindent 4 }} + {{- end }} app: executor deploy: sourcegraph app.kubernetes.io/component: executor diff --git a/charts/sourcegraph-executor/dind/templates/private-docker-registry/private-docker-registry.Service.yaml b/charts/sourcegraph-executor/dind/templates/private-docker-registry/private-docker-registry.Service.yaml index 356293ca8..cf472ab58 100644 --- a/charts/sourcegraph-executor/dind/templates/private-docker-registry/private-docker-registry.Service.yaml +++ b/charts/sourcegraph-executor/dind/templates/private-docker-registry/private-docker-registry.Service.yaml @@ -7,9 +7,13 @@ metadata: {{- toYaml .Values.privateDockerRegistry.serviceAnnotations | nindent 4 }} {{- end }} labels: + {{- include "sourcegraph.labels" . | nindent 4 }} app: private-docker-registry deploy: sourcegraph app.kubernetes.io/component: private-docker-registry + {{- if .Values.privateDockerRegistry.serviceLabels }} + {{- toYaml .Values.privateDockerRegistry.serviceLabels | nindent 4 }} + {{- end }} name: private-docker-registry spec: ports: diff --git a/charts/sourcegraph-executor/dind/templates/private-docker-registry/private-docker-registry.Statefulset.yaml b/charts/sourcegraph-executor/dind/templates/private-docker-registry/private-docker-registry.Statefulset.yaml index 71f3194e8..84fcae9f3 100644 --- a/charts/sourcegraph-executor/dind/templates/private-docker-registry/private-docker-registry.Statefulset.yaml +++ b/charts/sourcegraph-executor/dind/templates/private-docker-registry/private-docker-registry.Statefulset.yaml @@ -30,15 +30,10 @@ spec: {{- toYaml .Values.privateDockerRegistry.podAnnotations | nindent 8 }} {{- end }} labels: - {{- include "sourcegraph.selectorLabels" . | nindent 8 }} - {{- if .Values.sourcegraph.podLabels }} - {{- toYaml .Values.sourcegraph.podLabels | nindent 8 }} - {{- end }} - {{- if .Values.privateDockerRegistry.podLabels }} - {{- toYaml .Values.privateDockerRegistry.podLabels | nindent 8 }} - {{- end }} + {{- include "sourcegraph.podLabels" (list . .Values.privateDockerRegistry) | nindent 8 }} deploy: sourcegraph app: private-docker-registry + app.kubernetes.io/component: private-docker-registry spec: containers: - name: private-docker-registry diff --git a/charts/sourcegraph-executor/dind/tests/labels_test.yaml b/charts/sourcegraph-executor/dind/tests/labels_test.yaml new file mode 100644 index 000000000..9a9dfc7ed --- /dev/null +++ b/charts/sourcegraph-executor/dind/tests/labels_test.yaml @@ -0,0 +1,107 @@ +suite: sourcegraph.labels on every object +# No `templates` list, so every template in the chart is rendered, including +# ones added later. The test fails with "no manifest found" for a template that +# renders nothing, so enable a new optional template here. +tests: + - it: should add sourcegraph.labels to every object the chart renders + set: + sourcegraph: + labels: + example.com/installation: production + example.com/project: sourcegraph + executor: + queueName: batches + privateDockerRegistry: + enabled: true + asserts: + - isSubset: + path: metadata.labels + content: + app.kubernetes.io/managed-by: Helm + deploy: sourcegraph + example.com/installation: production + example.com/project: sourcegraph + - exists: + path: metadata.labels["helm.sh/chart"] + - exists: + path: metadata.labels["app.kubernetes.io/name"] + - exists: + path: metadata.labels["app.kubernetes.io/instance"] + - exists: + path: metadata.labels["app.kubernetes.io/version"] + - exists: + path: metadata.labels["app.kubernetes.io/component"] + - it: should add the labels of a component to every object of it + set: + executor: + queueName: batches + labels: + example.com/service: component + privateDockerRegistry: + enabled: true + labels: + example.com/service: component + serviceLabels: + example.com/service: component + asserts: + - equal: + path: metadata.labels["example.com/service"] + value: component + - it: should add sourcegraph.labels and the chart's labels to the pod template, and let the selector labels win + template: executor/executor.Deployment.yaml + set: + sourcegraph: + labels: + example.com/project: sourcegraph + podLabels: + example.com/project: pods + # A selector label: a different value would stop the selector from matching the pods. + app.kubernetes.io/instance: other + executor: + queueName: batches + asserts: + - isSubset: + path: spec.template.metadata.labels + content: + app.kubernetes.io/instance: RELEASE-NAME + app.kubernetes.io/managed-by: Helm + deploy: sourcegraph + example.com/project: pods + - notExists: + path: spec.template.metadata.labels["helm.sh/chart"] + - exists: + path: spec.template.metadata.labels["app.kubernetes.io/name"] + - notExists: + path: spec.template.metadata.labels["app.kubernetes.io/version"] + - exists: + path: spec.template.metadata.labels["app.kubernetes.io/component"] + - it: should add sourcegraph.labels and the chart's labels to the pod template, and let the selector labels win + template: private-docker-registry/private-docker-registry.Statefulset.yaml + set: + sourcegraph: + labels: + example.com/project: sourcegraph + podLabels: + example.com/project: pods + # A selector label: a different value would stop the selector from matching the pods. + app.kubernetes.io/instance: other + executor: + queueName: batches + privateDockerRegistry: + enabled: true + asserts: + - isSubset: + path: spec.template.metadata.labels + content: + app.kubernetes.io/instance: RELEASE-NAME + app.kubernetes.io/managed-by: Helm + deploy: sourcegraph + example.com/project: pods + - notExists: + path: spec.template.metadata.labels["helm.sh/chart"] + - exists: + path: spec.template.metadata.labels["app.kubernetes.io/name"] + - notExists: + path: spec.template.metadata.labels["app.kubernetes.io/version"] + - exists: + path: spec.template.metadata.labels["app.kubernetes.io/component"] diff --git a/charts/sourcegraph-executor/dind/values.yaml b/charts/sourcegraph-executor/dind/values.yaml index 571ebe624..10b74fae0 100644 --- a/charts/sourcegraph-executor/dind/values.yaml +++ b/charts/sourcegraph-executor/dind/values.yaml @@ -217,6 +217,8 @@ privateDockerRegistry: repository: registry tag: 3 storageSize: 10Gi + # -- Add extra labels to the `private-docker-registry` Service + serviceLabels: {} # -- Extra environment variables to set on the registry container. # Registry can be configured with environment variables using the pattern # REGISTRY_
_ (uppercase). Full reference: diff --git a/charts/sourcegraph-executor/k8s/templates/_helpers.tpl b/charts/sourcegraph-executor/k8s/templates/_helpers.tpl index 70f75b28c..cb8bb442b 100644 --- a/charts/sourcegraph-executor/k8s/templates/_helpers.tpl +++ b/charts/sourcegraph-executor/k8s/templates/_helpers.tpl @@ -12,29 +12,6 @@ Create chart name and version as used by the chart label. {{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }} {{- end }} -{{/* -Common labels -*/}} -{{- define "sourcegraph.labels" -}} -helm.sh/chart: {{ include "sourcegraph.chart" . }} -{{ include "sourcegraph.selectorLabels" . }} -{{- if .Chart.AppVersion }} -app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} -{{- end }} -app.kubernetes.io/managed-by: {{ .Release.Service }} -{{- if .Values.sourcegraph.labels }} -{{ toYaml .Values.sourcegraph.labels }} -{{- end }} -{{- end }} - -{{/* -Selector labels -*/}} -{{- define "sourcegraph.selectorLabels" -}} -app.kubernetes.io/name: {{ include "sourcegraph.name" . }} -app.kubernetes.io/instance: {{ .Release.Name }} -{{- end }} - {{/* Create the name of the service account to use */}} @@ -106,9 +83,3 @@ executor-{{join "-" .Values.executor.queueNames }} {{- end }} {{- end }} -{{- define "executor.labels" -}} -app: {{ include "executor.name" . }} -deploy: sourcegraph -sourcegraph-resource-requires: no-cluster-admin -app.kubernetes.io/component: executor -{{- end}} diff --git a/charts/sourcegraph-executor/k8s/templates/_helpers/_labels.tpl b/charts/sourcegraph-executor/k8s/templates/_helpers/_labels.tpl new file mode 100644 index 000000000..f4c0197ef --- /dev/null +++ b/charts/sourcegraph-executor/k8s/templates/_helpers/_labels.tpl @@ -0,0 +1,47 @@ +{{/* +Common labels +*/}} +{{- define "sourcegraph.labels" -}} +helm.sh/chart: {{ include "sourcegraph.chart" . }} +{{ include "sourcegraph.selectorLabels" . }} +{{- if .Chart.AppVersion }} +app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} +{{- end }} +app.kubernetes.io/managed-by: {{ .Release.Service }} +{{- if .Values.sourcegraph.labels }} +{{ toYaml .Values.sourcegraph.labels }} +{{- end }} +{{- end }} + +{{/* +Selector labels +*/}} +{{- define "sourcegraph.selectorLabels" -}} +app.kubernetes.io/name: {{ include "sourcegraph.name" . }} +app.kubernetes.io/instance: {{ .Release.Name }} +{{- end }} + +{{/* +Labels of a pod template: the common labels, then sourcegraph.podLabels, then +the component's podLabels, then the selector labels. The later one wins when a +key is set more than once, so each key is rendered once and the selector always +matches the pods. + +`helm.sh/chart` and `app.kubernetes.io/version` are left out: they change with +every chart version, and a changed pod template restarts the pods. + +{{ include "sourcegraph.podLabels" (list . .Values.executor) }} +*/}} +{{- define "sourcegraph.podLabels" -}} +{{- $top := index . 0 -}} +{{- $component := index . 1 -}} +{{- $labels := omit (include "sourcegraph.labels" $top | fromYaml) "helm.sh/chart" "app.kubernetes.io/version" -}} +{{- mergeOverwrite $labels ($top.Values.sourcegraph.podLabels | default dict) ($component.podLabels | default dict) (include "sourcegraph.selectorLabels" $top | fromYaml) | toYaml -}} +{{- end }} + +{{- define "executor.labels" -}} +app: {{ include "executor.name" . }} +deploy: sourcegraph +sourcegraph-resource-requires: no-cluster-admin +app.kubernetes.io/component: executor +{{- end}} diff --git a/charts/sourcegraph-executor/k8s/templates/executor.ConfigMap.yaml b/charts/sourcegraph-executor/k8s/templates/executor.ConfigMap.yaml index 6e773607b..9963cfea1 100644 --- a/charts/sourcegraph-executor/k8s/templates/executor.ConfigMap.yaml +++ b/charts/sourcegraph-executor/k8s/templates/executor.ConfigMap.yaml @@ -3,7 +3,11 @@ kind: ConfigMap metadata: name: {{ include "executor.name" . }} labels: -{{- include "executor.labels" . | nindent 4 }} + {{- include "sourcegraph.labels" . | nindent 4 }} + {{- include "executor.labels" . | nindent 4 }} + {{- if .Values.executor.labels }} + {{- toYaml .Values.executor.labels | nindent 4 }} + {{- end }} data: EXECUTOR_USE_FIRECRACKER: "false" EXECUTOR_FRONTEND_URL: "{{ .Values.executor.frontendUrl }}" diff --git a/charts/sourcegraph-executor/k8s/templates/executor.Deployment.yaml b/charts/sourcegraph-executor/k8s/templates/executor.Deployment.yaml index 1cb35d2c4..a001db219 100644 --- a/charts/sourcegraph-executor/k8s/templates/executor.Deployment.yaml +++ b/charts/sourcegraph-executor/k8s/templates/executor.Deployment.yaml @@ -6,6 +6,7 @@ metadata: description: Runs a sourcegraph executor kubectl.kubernetes.io/default-container: executor labels: + {{- include "sourcegraph.labels" . | nindent 4 }} {{- include "executor.labels" . | nindent 4 }} {{- if .Values.executor.labels }} {{- toYaml .Values.executor.labels | nindent 4 }} @@ -37,13 +38,7 @@ spec: {{- toYaml .Values.executor.podAnnotations | nindent 8 }} {{- end }} labels: - {{- include "sourcegraph.selectorLabels" . | nindent 8 }} - {{- if .Values.sourcegraph.podLabels }} - {{- toYaml .Values.sourcegraph.podLabels | nindent 8 }} - {{- end }} - {{- if .Values.executor.podLabels }} - {{- toYaml .Values.executor.podLabels | nindent 8 }} - {{- end }} + {{- include "sourcegraph.podLabels" (list . .Values.executor) | nindent 8 }} {{- include "executor.labels" . | nindent 8 }} spec: securityContext: diff --git a/charts/sourcegraph-executor/k8s/templates/executor.Role.yaml b/charts/sourcegraph-executor/k8s/templates/executor.Role.yaml index 7a3d1ef4d..68caac8a5 100644 --- a/charts/sourcegraph-executor/k8s/templates/executor.Role.yaml +++ b/charts/sourcegraph-executor/k8s/templates/executor.Role.yaml @@ -4,6 +4,10 @@ kind: Role metadata: name: sg-executor-role labels: + {{- include "sourcegraph.labels" . | nindent 4 }} + {{- if .Values.executor.labels }} + {{- toYaml .Values.executor.labels | nindent 4 }} + {{- end }} category: rbac deploy: sourcegraph sourcegraph-resource-requires: cluster-admin diff --git a/charts/sourcegraph-executor/k8s/templates/executor.RoleBinding.yaml b/charts/sourcegraph-executor/k8s/templates/executor.RoleBinding.yaml index f13311c5f..e7e6f2df7 100644 --- a/charts/sourcegraph-executor/k8s/templates/executor.RoleBinding.yaml +++ b/charts/sourcegraph-executor/k8s/templates/executor.RoleBinding.yaml @@ -4,6 +4,10 @@ kind: RoleBinding metadata: name: sg-executor-role-binding labels: + {{- include "sourcegraph.labels" . | nindent 4 }} + {{- if .Values.executor.labels }} + {{- toYaml .Values.executor.labels | nindent 4 }} + {{- end }} category: rbac deploy: sourcegraph sourcegraph-resource-requires: cluster-admin diff --git a/charts/sourcegraph-executor/k8s/templates/executor.Secret.yaml b/charts/sourcegraph-executor/k8s/templates/executor.Secret.yaml index 669afae55..936b4ea1a 100644 --- a/charts/sourcegraph-executor/k8s/templates/executor.Secret.yaml +++ b/charts/sourcegraph-executor/k8s/templates/executor.Secret.yaml @@ -2,6 +2,12 @@ apiVersion: v1 kind: Secret metadata: + labels: + {{- include "sourcegraph.labels" . | nindent 4 }} + {{- include "executor.labels" . | nindent 4 }} + {{- if .Values.executor.labels }} + {{- toYaml .Values.executor.labels | nindent 4 }} + {{- end }} name: {{ include "executor.name" . }} type: Opaque data: diff --git a/charts/sourcegraph-executor/k8s/templates/executor.Service.yaml b/charts/sourcegraph-executor/k8s/templates/executor.Service.yaml deleted file mode 100644 index 33d841b3b..000000000 --- a/charts/sourcegraph-executor/k8s/templates/executor.Service.yaml +++ /dev/null @@ -1,22 +0,0 @@ -apiVersion: v1 -kind: Service -metadata: - annotations: - {{- if .Values.executor.serviceAnnotations }} - {{- toYaml .Values.executor.serviceAnnotations | nindent 4 }} - {{- end }} - labels: -{{ include "executor.labels" . | indent 4 }} - {{- if .Values.executor.serviceLabels }} - {{- toYaml .Values.executor.serviceLabels | nindent 4 }} - {{- end }} - name: {{ include "executor.name" . }} -spec: - ports: - - name: http-debug - port: 6060 - targetPort: http-debug - selector: - {{- include "sourcegraph.selectorLabels" . | nindent 4 }} - app: {{ include "executor.name" . }} - type: {{ .Values.executor.serviceType | default "ClusterIP" }} diff --git a/charts/sourcegraph-executor/k8s/templates/executor.ServiceAccount.yaml b/charts/sourcegraph-executor/k8s/templates/executor.ServiceAccount.yaml index 6a0f44627..39f2a0fb7 100644 --- a/charts/sourcegraph-executor/k8s/templates/executor.ServiceAccount.yaml +++ b/charts/sourcegraph-executor/k8s/templates/executor.ServiceAccount.yaml @@ -4,6 +4,10 @@ kind: ServiceAccount metadata: name: sg-executor labels: + {{- include "sourcegraph.labels" . | nindent 4 }} + {{- if .Values.executor.labels }} + {{- toYaml .Values.executor.labels | nindent 4 }} + {{- end }} category: rbac deploy: sourcegraph sourcegraph-resource-requires: cluster-admin diff --git a/charts/sourcegraph-executor/k8s/tests/executor_test.yaml b/charts/sourcegraph-executor/k8s/tests/executor_test.yaml index 256638deb..dea6138f9 100644 --- a/charts/sourcegraph-executor/k8s/tests/executor_test.yaml +++ b/charts/sourcegraph-executor/k8s/tests/executor_test.yaml @@ -1,10 +1,9 @@ suite: executor templates: - executor.Deployment.yaml - - executor.Service.yaml - executor.ConfigMap.yaml tests: - - it: should render the Deployment, Service, ConfigMap if executor is enabled + - it: should render the Deployment, ConfigMap if executor is enabled set: executor: queueName: "test" @@ -14,11 +13,6 @@ tests: apiVersion: apps/v1 name: executor-test template: executor.Deployment.yaml - - containsDocument: - kind: Service - apiVersion: v1 - name: executor-test - template: executor.Service.yaml - containsDocument: kind: ConfigMap apiVersion: v1 diff --git a/charts/sourcegraph-executor/k8s/tests/labels_test.yaml b/charts/sourcegraph-executor/k8s/tests/labels_test.yaml new file mode 100644 index 000000000..1874a0cd1 --- /dev/null +++ b/charts/sourcegraph-executor/k8s/tests/labels_test.yaml @@ -0,0 +1,71 @@ +suite: sourcegraph.labels on every object +# No `templates` list, so every template in the chart is rendered, including +# ones added later. The test fails with "no manifest found" for a template that +# renders nothing, so enable a new optional template here. +tests: + - it: should add sourcegraph.labels to every object the chart renders + set: + sourcegraph: + labels: + example.com/installation: production + example.com/project: sourcegraph + executor: + queueName: batches + configureRbac: true + asserts: + - isSubset: + path: metadata.labels + content: + app.kubernetes.io/managed-by: Helm + deploy: sourcegraph + example.com/installation: production + example.com/project: sourcegraph + - exists: + path: metadata.labels["helm.sh/chart"] + - exists: + path: metadata.labels["app.kubernetes.io/name"] + - exists: + path: metadata.labels["app.kubernetes.io/instance"] + - exists: + path: metadata.labels["app.kubernetes.io/version"] + - exists: + path: metadata.labels["app.kubernetes.io/component"] + - it: should add executor.labels to every object + set: + executor: + queueName: batches + configureRbac: true + labels: + example.com/service: executor + asserts: + - equal: + path: metadata.labels["example.com/service"] + value: executor + - it: should add sourcegraph.labels and the chart's labels to the pod template, and let the selector labels win + template: executor.Deployment.yaml + set: + sourcegraph: + labels: + example.com/project: sourcegraph + podLabels: + example.com/project: pods + # A selector label: a different value would stop the selector from matching the pods. + app.kubernetes.io/instance: other + executor: + queueName: batches + asserts: + - isSubset: + path: spec.template.metadata.labels + content: + app.kubernetes.io/instance: RELEASE-NAME + app.kubernetes.io/managed-by: Helm + deploy: sourcegraph + example.com/project: pods + - notExists: + path: spec.template.metadata.labels["helm.sh/chart"] + - exists: + path: spec.template.metadata.labels["app.kubernetes.io/name"] + - notExists: + path: spec.template.metadata.labels["app.kubernetes.io/version"] + - exists: + path: spec.template.metadata.labels["app.kubernetes.io/component"] diff --git a/charts/sourcegraph-migrator/templates/_helpers.tpl b/charts/sourcegraph-migrator/templates/_helpers.tpl index f6ce9b36d..2de962a1e 100644 --- a/charts/sourcegraph-migrator/templates/_helpers.tpl +++ b/charts/sourcegraph-migrator/templates/_helpers.tpl @@ -12,29 +12,6 @@ Create chart name and version as used by the chart label. {{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }} {{- end }} -{{/* -Common labels -*/}} -{{- define "sourcegraph.labels" -}} -helm.sh/chart: {{ include "sourcegraph.chart" . }} -{{ include "sourcegraph.selectorLabels" . }} -{{- if .Chart.AppVersion }} -app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} -{{- end }} -app.kubernetes.io/managed-by: {{ .Release.Service }} -{{- if .Values.sourcegraph.labels }} -{{ toYaml .Values.sourcegraph.labels }} -{{- end }} -{{- end }} - -{{/* -Selector labels -*/}} -{{- define "sourcegraph.selectorLabels" -}} -app.kubernetes.io/name: {{ include "sourcegraph.name" . }} -app.kubernetes.io/instance: {{ .Release.Name }} -{{- end }} - {{/* Create the name of the service account to use */}} diff --git a/charts/sourcegraph-migrator/templates/_helpers/_labels.tpl b/charts/sourcegraph-migrator/templates/_helpers/_labels.tpl new file mode 100644 index 000000000..a73d9a7bb --- /dev/null +++ b/charts/sourcegraph-migrator/templates/_helpers/_labels.tpl @@ -0,0 +1,40 @@ +{{/* +Common labels +*/}} +{{- define "sourcegraph.labels" -}} +helm.sh/chart: {{ include "sourcegraph.chart" . }} +{{ include "sourcegraph.selectorLabels" . }} +{{- if .Chart.AppVersion }} +app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} +{{- end }} +app.kubernetes.io/managed-by: {{ .Release.Service }} +{{- if .Values.sourcegraph.labels }} +{{ toYaml .Values.sourcegraph.labels }} +{{- end }} +{{- end }} + +{{/* +Selector labels +*/}} +{{- define "sourcegraph.selectorLabels" -}} +app.kubernetes.io/name: {{ include "sourcegraph.name" . }} +app.kubernetes.io/instance: {{ .Release.Name }} +{{- end }} + +{{/* +Labels of a pod template: the common labels, then sourcegraph.podLabels, then +the component's podLabels, then the selector labels. The later one wins when a +key is set more than once, so each key is rendered once and the selector always +matches the pods. + +`helm.sh/chart` and `app.kubernetes.io/version` are left out: they change with +every chart version, and a changed pod template restarts the pods. + +{{ include "sourcegraph.podLabels" (list . .Values.migrator) }} +*/}} +{{- define "sourcegraph.podLabels" -}} +{{- $top := index . 0 -}} +{{- $component := index . 1 -}} +{{- $labels := omit (include "sourcegraph.labels" $top | fromYaml) "helm.sh/chart" "app.kubernetes.io/version" -}} +{{- mergeOverwrite $labels ($top.Values.sourcegraph.podLabels | default dict) ($component.podLabels | default dict) (include "sourcegraph.selectorLabels" $top | fromYaml) | toYaml -}} +{{- end }} diff --git a/charts/sourcegraph-migrator/templates/migrator/sourcegraph-migrator.Job.yaml b/charts/sourcegraph-migrator/templates/migrator/sourcegraph-migrator.Job.yaml index c77ec3b33..4e41b3a28 100644 --- a/charts/sourcegraph-migrator/templates/migrator/sourcegraph-migrator.Job.yaml +++ b/charts/sourcegraph-migrator/templates/migrator/sourcegraph-migrator.Job.yaml @@ -24,15 +24,10 @@ spec: {{- toYaml .Values.migrator.podAnnotations | nindent 8 }} {{- end }} labels: - {{- include "sourcegraph.selectorLabels" . | nindent 8 }} - {{- if .Values.sourcegraph.podLabels }} - {{- toYaml .Values.sourcegraph.podLabels | nindent 8 }} - {{- end }} - {{- if .Values.migrator.podLabels }} - {{- toYaml .Values.migrator.podLabels | nindent 8 }} - {{- end }} + {{- include "sourcegraph.podLabels" (list . .Values.migrator) | nindent 8 }} job: migrator deploy: sourcegraph + app.kubernetes.io/component: migrator spec: containers: - name: migrator diff --git a/charts/sourcegraph-migrator/tests/labels_test.yaml b/charts/sourcegraph-migrator/tests/labels_test.yaml new file mode 100644 index 000000000..47c6571a0 --- /dev/null +++ b/charts/sourcegraph-migrator/tests/labels_test.yaml @@ -0,0 +1,55 @@ +suite: sourcegraph.labels on every object +# No `templates` list, so every template in the chart is rendered, including +# ones added later. The test fails with "no manifest found" for a template that +# renders nothing, so enable a new optional template here. +tests: + - it: should add sourcegraph.labels to every object the chart renders + set: + sourcegraph: + labels: + example.com/installation: production + example.com/project: sourcegraph + asserts: + - isSubset: + path: metadata.labels + content: + app.kubernetes.io/managed-by: Helm + deploy: sourcegraph + example.com/installation: production + example.com/project: sourcegraph + - exists: + path: metadata.labels["helm.sh/chart"] + - exists: + path: metadata.labels["app.kubernetes.io/name"] + - exists: + path: metadata.labels["app.kubernetes.io/instance"] + - exists: + path: metadata.labels["app.kubernetes.io/version"] + - exists: + path: metadata.labels["app.kubernetes.io/component"] + - it: should add sourcegraph.labels and the chart's labels to the pod template, and let the selector labels win + template: migrator/sourcegraph-migrator.Job.yaml + set: + sourcegraph: + labels: + example.com/project: sourcegraph + podLabels: + example.com/project: pods + # A selector label: a different value would stop the selector from matching the pods. + app.kubernetes.io/instance: other + asserts: + - isSubset: + path: spec.template.metadata.labels + content: + app.kubernetes.io/instance: RELEASE-NAME + app.kubernetes.io/managed-by: Helm + deploy: sourcegraph + example.com/project: pods + - notExists: + path: spec.template.metadata.labels["helm.sh/chart"] + - exists: + path: spec.template.metadata.labels["app.kubernetes.io/name"] + - notExists: + path: spec.template.metadata.labels["app.kubernetes.io/version"] + - exists: + path: spec.template.metadata.labels["app.kubernetes.io/component"] diff --git a/charts/sourcegraph/CHANGELOG.md b/charts/sourcegraph/CHANGELOG.md index a8b444320..f2daf8eb6 100644 --- a/charts/sourcegraph/CHANGELOG.md +++ b/charts/sourcegraph/CHANGELOG.md @@ -9,6 +9,15 @@ Use `**BREAKING**:` to denote a breaking change ## Unreleased - The `correct-data-dir-permissions` init containers of `pgsql`, `codeintel-db`, and `codeinsights-db` now run the database's own image instead of `sourcegraph/alpine-3.14`, which was based on an end-of-life Alpine release. The chart no longer pulls `alpine-3.14`, so private registries no longer need to mirror it. `alpine.image` has been removed and is ignored if set; `alpine.containerSecurityContext`, `alpine.resources`, and `codeInsightsDB.init.containerSecurityContext` still apply. +- Fixed the "pods available" panel and alert of the OpenTelemetry Collector dashboard, which query `up{app=~".*otel-collector"}`, having no data. The `otel-collector` Service was missing the `app` label that every other scraped Service has, so its metrics had no `app` and `job` labels. They now have `app="otel-collector"` and `job="otel-collector"`. The collector also served its own metrics on 127.0.0.1 only, unless `jaeger.enabled` was set, so Prometheus could not scrape them. The chart now always passes the container one argument, after `openTelemetry.gateway.args`, that serves them on all interfaces, port 8888. A command set in `openTelemetry.gateway.command` gets the argument too, so a shell wrapper must pass its arguments on to the collector (`"$@"`). The argument replaces `service.telemetry.metrics.readers` of the configuration files the command loads. +- Every object the chart creates now has the `deploy` and `app.kubernetes.io/component` labels. PodDisruptionBudgets get the component of their workload, PriorityClasses get `priority-class`, the StorageClass gets `storage` and keeps `deploy: sourcegraph-storage`, and the `sg-test-connection` Pod gets `test-connection`. `extraResources` get `deploy: sourcegraph` unless they set their own; set `app.kubernetes.io/component` on them yourself. The `jaeger` ConfigMap, Deployment, and Services gain `app.kubernetes.io/instance` and keep `app.kubernetes.io/name: jaeger`, which their selectors use. Labels on these objects can be changed in place; the `jaeger` pod restarts once, because its `checksum/jaeger-config` annotation covers the ConfigMap's labels. +- `.labels` is now also rendered on the component's ConfigMaps, Secrets, and PodDisruptionBudget, not only on its workload, so releases that set it gain these labels on upgrade. Added `storageClass.labels` and a `labels` field on each `priorityClasses` entry. +- Fixed `redisStore.serviceAccount.annotations` being ignored, so the `redis-store` ServiceAccount was created without them, e.g. without a workload identity annotation. They are now rendered, as on every other ServiceAccount. +- The `prometheus` pod now restarts when an upgrade changes the chart's `prometheus.yml`, through a `checksum/prometheus-config` pod annotation. Prometheus does not reload a changed ConfigMap by itself, so such a change took effect only at the pod's next restart. The annotation is not set with `prometheus.existingConfig`. +- The `otel-collector` pod now restarts when `openTelemetry.gateway.config.traces` changes, through a `checksum/otel-collector-config` pod annotation. The collector does not reload a changed ConfigMap by itself, so a changed exporter or processor took effect only at the pod's next restart. The annotation is set only with `openTelemetry.gateway.config.traces.exporters`, which is when the chart renders that ConfigMap, so the default install is not affected. +- `sourcegraph.labels` and the chart's standard labels (`helm.sh/chart`, `app.kubernetes.io/name`, `app.kubernetes.io/instance`, `app.kubernetes.io/version`, `app.kubernetes.io/managed-by`) are now rendered on every object the chart creates: also on pods, Services, ConfigMaps, Secrets, the Ingress, PodDisruptionBudgets, PriorityClasses, and `extraResources`, so existing releases gain these labels on upgrade. Pods do not get `helm.sh/chart` and `app.kubernetes.io/version`, which change with every chart version and would restart every pod on every upgrade, and they also gain `app.kubernetes.io/component`. Labels that an `extraResources` entry, `serviceLabels`, or `podLabels` set take precedence, except that `podLabels` cannot change a selector label. No object is recreated. Every pod restarts once, on this upgrade. Prometheus copies the labels of a Service or pod to the metrics it scrapes from it, so the bundled Prometheus configuration now drops the five standard labels and the keys of `sourcegraph.labels`. No bundled dashboard or alert queries them. Metrics lose `app_kubernetes_io_managed_by`, which Helm added to every Service, `otel-collector` metrics lose the other standard labels, which its Service already had, and `cadvisor` metrics, scraped from pods, lose `app_kubernetes_io_name` and `app_kubernetes_io_instance` and gain `app_kubernetes_io_component`. To add a label to the metrics of a Service, set it in `.serviceLabels`, with a key that is not in `sourcegraph.labels`. If you set `prometheus.existingConfig`, add the same `labeldrop` rule after each `__meta_kubernetes_service_label_(.+)` and `__meta_kubernetes_pod_label_(.+)` `labelmap` rule in your `prometheus.yml`, as in the [k3s example](examples/prometheus/prometheus-override-k3s.ConfigMap.yaml), and append the keys of `sourcegraph.labels` to its regex. +- Added `storageLabels` on `blobstore`, `codeInsightsDB`, `codeIntelDB`, `pgsql`, `prometheus`, `redisCache`, and `redisStore` to label each component's PersistentVolumeClaim, and `.serviceAccount.labels` to label a component's ServiceAccount and, for `frontend` and `prometheus`, its Role, RoleBinding, ClusterRole, and ClusterRoleBinding. `sourcegraph.labels` and the chart's standard labels (`helm.sh/chart`, `app.kubernetes.io/name`, `app.kubernetes.io/instance`, `app.kubernetes.io/version`, `app.kubernetes.io/managed-by`) are now also rendered on these PersistentVolumeClaims, on every ServiceAccount, and on the `prometheus` RBAC objects, as they already are on every workload and on the `frontend` Role and RoleBinding, so existing releases gain these labels on upgrade. Labels on these objects can be changed in place, so none is recreated. See [Adopting existing workloads](README.md#adopting-existing-workloads). +- Fixed the `jaeger` ServiceAccount failing to render with `jaeger.serviceAccount.create: true`. Its name and annotations were read from a `tracing` value that does not exist, and are now read from `jaeger.serviceAccount`, as the `jaeger` Deployment's `serviceAccountName` already was. - Added `sourcegraph.selectorLabelsOverride`, `.extraSelectorLabels`, and `storageLabels` on `gitserver`, `grafana`, `indexedSearch`, and `searcher`, so the chart can render the selectors and volume claim templates of workloads created outside it, such as with Kustomize, and a Helm release can take them over without deleting them. All default to empty, so existing releases render the same selectors. Never change them on an installed release: Kubernetes does not allow changing these fields. See [Adopting existing workloads](README.md#adopting-existing-workloads). - Fixed `frontend.command` and `migrator.command` rendering a list as a single element, e.g. `["sh", "-c", "