Repository navigation
Compile installed packages and pytest's plugins when the image is built (~2.7x faster) - #128
Conversation
The packages are installed with --no-compile and PYTHONDONTWRITEBYTECODE, so the image held no .pyc files for pytest or its dependencies. Every run compiled them from source before it ran any tests. Compiling them once at image build takes about 40% off each run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks for this. I think the failing CI tests have to do with the Python version mis-match and error cases. The line number changes per python version. I will see if its an easy re-gen fix. According to what I've read, Standard precompilation via But I do wan tot dig a little further, so please bear with me. 🙂 |
pytest rewrites the assert statements in its plugins (such as pytest-subtests) when it imports them, and caches the rewritten copies separately from normal bytecode, so compileall can't create them. Running pytest once with --collect-only on a dummy test at image build creates those cached copies, so each run loads them instead of rewriting the plugins again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks for looking at this, Bethany 🙂 This reply is from Claude, on Jeremy's behalf. You're right that Your On CI: the failing |
|
On looking at this further, I think we could make the following changes, and maybe avoid having to compile the entire Pip will automatically look for a wheel and compile while installing (unless prevented by the --no-compile flag), so that will take care of pytest, black, and all dependancies. This may or may not be faster than just compiling Another suggestion: The |
|
Thanks Bethany, these are great suggestions! Letting pip compile during the install is simpler than a separate step over the whole of I'll benchmark your version against this PR on the same 20 example solutions: pip compiling during the install, |
Pip now compiles the packages while installing them, and compileall only handles the runner (with checked-hash files). The pytest warm-up unsets PYTHONDONTWRITEBYTECODE so it can still write its rewritten plugin files. Co-authored-by: iHiD <286476+iHiD@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks again, Bethany. I benchmarked your approach, and it's the best version, so I've switched the PR to it. Letting pip compile during the install is as fast as compiling all of There's one small adjustment. With The only failing golden test is still |
|
Thank you for the changes and the benchmark numbers!
Since we well know why that's failing (and it is getting late for me) - I am going to go ahead and merge this so we can take advantage of the speed bump-up. 😄 |
|
Thank you, Bethany! Your suggestions made this both faster and simpler than where it started, so thanks for digging into it. I'll check the production run times once it's deployed. |
|
Thanks again for merging this, @BethanyG, and for your suggestions. Here are the production numbers since it was deployed at 10:22 UTC.
The "after" figures cover about 300 runs. Most of the gain comes from your idea of warming pytest with |
Hello 🙂 I'm working with Claude to try and get our tooling faster.
This PR makes this project around 2.7x faster. I've run it in production to benchmark things in the real world against the most recent few hundred solutions, so I'm confident that it works robustly. However, I've not written the code. I have tried my hardest to get Claude to write sensible messages and comments that explains stuff, but I'd ask for your patience in the fact I'm sending AI code to you. Everything below here is from your friendly neighbourhood LLM...
What changed
The image now includes precompiled
.pycfiles for pytest and its dependencies. The Dockerfile used to install the packages with--no-compileandPYTHONDONTWRITEBYTECODE=1, so the image had no.pycfiles insite-packages. Every run compiled pytest and its dependencies from source before running any tests, and the compiled files were lost when the container stopped. As suggested in review, pip now compiles the packages as it installs them, andcompileallcompiles only the runner (with--invalidation-mode=checked-hash).The image also caches pytest's rewritten plugins. pytest rewrites the
assertstatements in its plugins (such as pytest-subtests) when it imports them, and caches the rewritten copies separately from normal bytecode. A new step, also suggested in review, runspytest --collect-onlyonce on a dummy test when the image is built, so each run loads those cached copies instead of rewriting the plugins again. That step unsetsPYTHONDONTWRITEBYTECODE, because pytest won't write the files while it's set.The image grows from about 59 MB to 70 MB.
This has been tested in production, and has reduced the median run from 2.5s to 0.9s.
🤖 Generated with Claude Code