Repository navigation
Conversation
The executor parses EXECUTOR_KUBERNETES_NODE_TOLERATIONS, the node
affinity match expressions and fields, and the pod affinity and
anti-affinity with json.Unmarshal. The ConfigMap rendered them as
"{{ .Values... }}", which broke both ways of setting them:
- A JSON encoded string, the documented form, put unescaped double
quotes inside a double-quoted YAML scalar, so the render failed
with "did not find expected key".
- A YAML list was printed with Go formatting, e.g.
"[map[effect:NoSchedule key:foo operator:Equal value:bar]]", so the
executor failed with "invalid character 'm' looking for beginning
of value".
Add an executor.jsonEnv helper that quotes a string unchanged and
encodes a list or map with toJson, and use it for all five values.
Empty values still render as "". The values.yaml examples escaped
their quotes inside single-quoted YAML, so copied as written they
were not valid JSON; they now are.
Fixes sourcegraph#349
Author
|
Closing in favour of #902, which fixed #349 first and which I missed before opening this. #902 is also the safer fix: it keeps the documented pre-escaped form ( |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #349.
The executor reads
EXECUTOR_KUBERNETES_NODE_TOLERATIONS,EXECUTOR_KUBERNETES_NODE_REQUIRED_AFFINITY_MATCH_EXPRESSIONS,EXECUTOR_KUBERNETES_NODE_REQUIRED_AFFINITY_MATCH_FIELDS,EXECUTOR_KUBERNETES_POD_AFFINITY, andEXECUTOR_KUBERNETES_POD_ANTI_AFFINITYwithjson.Unmarshal. Thesourcegraph-executor-k8sConfigMap rendered them as"{{ .Values... }}", which broke both ways of setting them:"[{"key":"foo",...}]", unescaped quotes inside a double-quoted scalarhelm templatefails:yaml: line 26: did not find expected key"[map[effect:NoSchedule key:foo operator:Equal value:bar]]"invalid EXECUTOR_KUBERNETES_NODE_TOLERATIONS, failed to parse: invalid character 'm' looking for beginning of valueChanges
executor.jsonEnvhelper: a string is quoted unchanged, a list or map is encoded withtoJsonand quoted, and an empty or unset value renders"".main.values.yamlexamples escaped their quotes inside single-quoted YAML ('[{\"key\":...}]'), so copied as written they were not valid JSON. They are now, and the descriptions say a YAML list also works. README regenerated with./scripts/helm-docs.sh.Checklist
Test plan
charts/sourcegraph-executor/k8s/tests/executor_test.yaml: default, JSON string, and YAML list, each across all five values. Onmain, 2 fail and 1 errors; all pass here.helm unitteston all four charts, as CI runs it (helm-unittest v1.0.2): 284 tests pass.helm lint charts/sourcegraph-executor/k8spasses.json.Unmarshalinto thek8s.io/api/core/v1types the executor declares ([]Toleration,[]NodeSelectorRequirement,[]PodAffinityTerm): both forms decode here; onmainthe YAML list fails with the error above and the JSON string does not render.sourcegraph/executor-kubernetes:8.1.4873(the digest pinned invalues.yaml) with the rendered ConfigMap as its environment and an unreachable frontend URL:main, YAML listinvalid ... failed to parseerrors, one per valueTested on a cluster
k3s v1.35.8, Helm v3.19.0,
executor-kubernetes:8.1.4873, in a throwaway namespace with an unreachable frontend URL, so a working executor passes config validation and then retries connecting to the frontend.tolerationsmainhelm upgrade[{"effect":"NoSchedule","key":"dedicated",...}]; pod Running, no parse errorsmain[map[effect:NoSchedule ...]]; pod inCrashLoopBackOffwithinvalid EXECUTOR_KUBERNETES_NODE_TOLERATIONS, failed to parse: invalid character 'm' looking for beginning of valuemainhelm upgradefails:YAML parse error ... did not find expected keySteps 3, 4 and 6 used
kubectl rollout restart, because the Deployment has no checksum annotation for its ConfigMap, so a values change alone does not restart the executor.Not tested: job pods being scheduled with these tolerations. The executor creates job pods only after dequeuing a job from a Sourcegraph frontend, and none was deployed.