Skip to content

ci(python): check generated Python Connexion server code in a GitHub workflow - #25654

Open
khayashi4337 wants to merge 4 commits into
OpenAPITools:masterfrom
khayashi4337:ci/python-connexion-generated-code-workflow
Open

khayashi4337 wants to merge 4 commits into
OpenAPITools:masterfrom
khayashi4337:ci/python-connexion-generated-code-workflow

Conversation

@khayashi4337

@khayashi4337 khayashi4337 commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Follow-up to #25556, as promised in #25556 (comment).

#25556 asked for the unit tests to stop depending on a local python3. #25579 (merged) did the Java side: it removed testJsonContentHeaderParamsGeneratedTestsCompile and re-enabled testInfoTitleEmbeddedAsPythonStringLiteral with Java-only assertions, so ./mvnw test no longer needs python3. Those Java assertions pin the generated text line by line; they cannot tell whether the result is valid Python or what the string literals evaluate to. This PR adds a GitHub workflow that checks exactly that.

What the workflow does

.github/workflows/samples-python-connexion-generated-code.yaml builds the CLI (JDK 11, with the same ~/.m2/repository cache as openapi-generator.yaml), sets up Python 3.11, and then runs one command, bin/utils/test-python-connexion-generated.sh. That script:

  • generates two test specs that are already in the repository, 3_0/python/connexion-json-header-test-values.yaml and 3_0/python/info-title-special-chars.yaml, with each of python-flask, python-flask with useConnexion3=true, python-aiohttp and python-blueplanet (8 outputs);
  • runs bin/utils/check-python-connexion-generated.py on each output, and stops at the first output that fails. For one output, the check script:
    • compiles the generated test_default_controller.py, treating warnings such as invalid escape sequences as errors;
    • parses the generated setup.py with Python's ast, evaluates the keyword arguments of setup(), and compares three of them with the values the generator should produce from the spec: description (info.title), author_email (info.contact.email, or "" when absent) and long_description (info.description, or the generator's "No description provided …" text when absent);
    • reports every mismatch in that output and exits with 1; a missing or ambiguous file is reported as a one-line error.

The script also runs locally once the CLI is built; it needs java and python3 with PyYAML.

It does not run the generated tests with pytest. On the python-flask output with useConnexion3=true (the only variant I tried), plain pytest passes when a generated test contains an invalid escape such as "\d" (only a DeprecationWarning) and when the setup.py description is wrong, and two of the generated tests for connexion-json-header-test-values.yaml already fail on master (400 '7' is not of type 'integer').

Unlike the other samples-python-* workflows, this one generates its inputs instead of checking committed samples. The two specs exist to exercise edge cases: JSON content header parameters, and a title, description and contact email full of quotes, triple quotes, backslashes, &, <> and control characters. Committing their output would add 8 configs and 226 generated files; the trade-off is discussed in the review.

It runs on push and pull_request when any of these change:

  • the committed samples of these generators: samples/server/petstore/python-flask, python-flask-connexion3, python-aiohttp-srclayout and python-blueplanet;
  • the two test specs;
  • either script, or the workflow itself.

The petstore samples do not contain the special characters the check is about. A change that only affects how those characters are generated may therefore leave the samples unchanged and not trigger the workflow.

Verified

  • On my fork, the workflow ran as written on GitHub Actions at 24db256 and passed: the CLI built, and all 8 outputs reported both checks as OK (https://github.com/khayashi4337/openapi-generator/actions/runs/43744174428).
  • Locally, with Python 3.11.15, bin/utils/test-python-connexion-generated.sh run from another directory exits with 0, and all 16 checks report OK. With the CLI jar missing, it exits with 1 and says how to build the jar.
  • Locally (Python 3.14; all 8 outputs and the invalid-escape case also with Python 3.11), the check script fails with exit code 1 for each of the following, and passes again once the file is restored:
    • a syntax error in a generated test;
    • an invalid escape sequence such as "\d" in a generated test;
    • a changed description in a generated setup.py;
    • a changed author_email in the setup.py generated from the spec without a contact email;
    • an empty output directory.
  • actions/checkout@v7, actions/setup-java@v6, actions/setup-python@v7 and actions/cache@v6 match the versions used by the existing workflows.

No Java code, template or sample changes.

cc @wing328

PR checklist

  • Read the contribution guidelines.
  • Run the following to build the project and update samples — not applicable: this PR only adds a workflow and two helper scripts, so no sample or doc output changes. The CLI was built locally and on the fork's Actions run to exercise the workflow.
    ./mvnw clean package || exit
    ./bin/generate-samples.sh ./bin/configs/*.yaml || exit
    ./bin/utils/export_docs_generators.sh || exit
    
    (For Windows users, please run the script in WSL)
    Commit all changed files.
    This is important, as CI jobs will verify all generator outputs of your HEAD commit as it would merge with master.
    These must match the expectations made by your contribution.
    You may regenerate an individual generator by passing the relevant config(s) as an argument to the script, for example ./bin/generate-samples.sh bin/configs/java*.
    IMPORTANT: Do NOT purge/delete any folders/files (e.g. tests) when regenerating the samples as manually written tests may be removed.
  • If your PR is targeting a particular programming language, @mention the technical committee members, so they are more likely to review the pull request. (The Python technical committee is not cc'd: this is CI-only and was requested by the maintainer in Disabled python tests that require python3 #25556.)

🤖 Generated with Claude Code

…workflow

The unit tests must not require a python3 toolchain (OpenAPITools#25556). OpenAPITools#25579
removed testJsonContentHeaderParamsGeneratedTestsCompile and re-enabled
testInfoTitleEmbeddedAsPythonStringLiteral with Java-only assertions;
this workflow restores the Python-level checks (py_compile of the
generated tests and evaluating the setup() literals with ast):

- bin/utils/check-python-connexion-generated.py runs py_compile on the
  generated test_default_controller.py (with warnings promoted to
  errors) and evaluates the setup() kwargs
  (description/author_email/long_description) with ast, comparing all
  three against the values the generator derives from the input spec's
  info object -- spec values where present, generator defaults
  otherwise.
- The workflow builds the CLI, generates the two Connexion test specs
  under python-flask, python-flask useConnexion3=true, python-aiohttp
  and python-blueplanet (8 combinations), and runs the check script over
  each output.

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 2 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name=".github/workflows/samples-python-connexion-generated-code.yaml">

<violation number="1" location=".github/workflows/samples-python-connexion-generated-code.yaml:11">
P2: The path filters miss `DefaultCodegen.java` and `DefaultGenerator.java`. The Python Connexion escaping and the default `long_description` both come from those files. Add them to both `push` and `pull_request` paths so changes to them run this check.</violation>
</file>

Tip: Get a Slack DM when cubic finishes reviewing your PRs. Connect Slack

View guided diff | Re-trigger cubic

- modules/openapi-generator/src/main/java/org/openapitools/codegen/languages/PythonAiohttpConnexionServerCodegen.java
- modules/openapi-generator/src/main/java/org/openapitools/codegen/languages/PythonBluePlanetServerCodegen.java
- modules/openapi-generator/src/main/java/org/openapitools/codegen/languages/PythonFlaskConnexionServerCodegen.java
- modules/openapi-generator/src/main/java/org/openapitools/codegen/languages/PythonStringUtils.java

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The path filters miss DefaultCodegen.java and DefaultGenerator.java. The Python Connexion escaping and the default long_description both come from those files. Add them to both push and pull_request paths so changes to them run this check.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At .github/workflows/samples-python-connexion-generated-code.yaml, line 11:

<comment>The path filters miss `DefaultCodegen.java` and `DefaultGenerator.java`. The Python Connexion escaping and the default `long_description` both come from those files. Add them to both `push` and `pull_request` paths so changes to them run this check.</comment>

<file context>
@@ -0,0 +1,97 @@
+      - modules/openapi-generator/src/main/java/org/openapitools/codegen/languages/PythonAiohttpConnexionServerCodegen.java
+      - modules/openapi-generator/src/main/java/org/openapitools/codegen/languages/PythonBluePlanetServerCodegen.java
+      - modules/openapi-generator/src/main/java/org/openapitools/codegen/languages/PythonFlaskConnexionServerCodegen.java
+      - modules/openapi-generator/src/main/java/org/openapitools/codegen/languages/PythonStringUtils.java
+      - modules/openapi-generator/src/main/resources/python-flask/**
+      - modules/openapi-generator/src/main/resources/python-aiohttp/**
</file context>
Suggested change
- modules/openapi-generator/src/main/java/org/openapitools/codegen/languages/PythonStringUtils.java
- modules/openapi-generator/src/main/java/org/openapitools/codegen/DefaultCodegen.java
- modules/openapi-generator/src/main/java/org/openapitools/codegen/DefaultGenerator.java
- modules/openapi-generator/src/main/java/org/openapitools/codegen/languages/PythonStringUtils.java

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded by df29e14: following the maintainer's review, the workflow now triggers on the committed sample paths (as the other samples-* workflows do) instead of listing generator sources. A change to DefaultCodegen, DefaultGenerator or PythonStringUtils that affects the Connexion output also changes those regenerated samples, so the check still runs.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correction: this was too strong. The petstore samples do not contain the special characters the check is about, so a change that only affects how those characters are escaped can leave the samples unchanged, and then the check does not run. 24db256 also triggers on the two test specs. The committed-sample alternative, which would close this gap, is in the thread with wing328 above.

- modules/openapi-generator/src/test/resources/3_0/python/connexion-json-header-test-values.yaml
- modules/openapi-generator/src/test/resources/3_0/python/info-title-special-chars.yaml
- bin/utils/check-python-connexion-generated.py
- .github/workflows/samples-python-connexion-generated-code.yaml

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and to run the tests, can we simply do something similar to https://github.com/OpenAPITools/openapi-generator/blob/master/.github/workflows/samples-python-server.yaml#L27 by running a single script/command? maybe as simple as just running pytest

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, done in df29e14: the workflow now triggers on the sample paths of the four generators it checks (samples/server/petstore/python-flask, python-flask-connexion3, python-aiohttp-srclayout and python-blueplanet), plus the check script and the workflow file itself, following samples-python-server.yaml / samples-python-petstore.yaml.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, and sorry I missed this one: it crossed with my reply above. Done in 8d55e90 and 24db256. The generate-and-check loop is now bin/utils/test-python-connexion-generated.sh, and the job's test step is the single line run: /bin/bash ./bin/utils/test-python-connexion-generated.sh, the way misc-test.yaml calls the bin/utils scripts. The script does not run the generated tests. It checks two things:

  • the generated Python compiles without warnings;
  • the setup.py metadata matches the spec.

I did not use plain pytest. I tried it on the python-flask output with useConnexion3=true, and it misses both of those checks:

  • An invalid escape such as "\d" in a generated test only produces a DeprecationWarning.
  • No generated test reads setup.py.

Separately, two of the generated tests for connexion-json-header-test-values.yaml already fail on master with 400 '7' is not of type 'integer'. I tried only that one variant.

The workflow still differs from samples-python-server.yaml in two ways:

  • It builds the CLI on every run.
  • It triggers on the petstore samples of the same generators, and those do not contain the special characters it checks.

To narrow the second gap, 24db256 also triggers on the two test specs. A template change that only affects those characters can still leave the petstore samples unchanged, though, so my earlier note to cubic ("so the check still runs") was too strong.

The alternative is committed samples:

  • 8 configs in bin/configs (2 specs × 4 generators), plus 226 generated files, for example under samples/server/others/.
  • The workflow would then run the check script in each sample directory, with no CLI build, and trigger on exactly what it checks.
  • Running pytest there as well would first need a fix for the two failing tests.

Which would you prefer?

khayashi4337 and others added 3 commits November 16, 2026 01:53
Use the generated server sample paths for push/pull_request triggers,
per wing328's review on PR OpenAPITools#25654.

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Per wing328's review on PR OpenAPITools#25654, replace the inline test step with a
single bin/utils/test-python-connexion-generated.sh call so the job runs
the same check with one command, like the other sample workflows.

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Add the two Connexion test specs to the workflow trigger paths
- Document the java and PyYAML prerequisites and that a relative
  output_dir resolves against the repository root
- Print the output directory at the start of the test script

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants