Repository navigation
Conversation
The old pin declares python_requires<3.13, so the extra cannot install on Python 3.13 or 3.14. Remove vLLM from the generated GPU requirements. vLLM 0.32.0 pins torch==2.13.0, and the transformers extra caps torch<2.12.0.
bvolpato
force-pushed
the
bvolpato/bump-vllm
branch
from
November 5, 2026 02:05
ffe75f8 to
b2fedd8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR upgrades the
vllmextra from 0.10.1.1 to 0.32.0. The old pin dates from August 2025 and declarespython_requires<3.13, so the extra cannot install on Python 3.13 or 3.14. vLLM 0.32.0 declares support for Python versions below 3.15.The new version needs newer dependencies, so this PR makes these changes:
setup.py: Pinvllm==0.32.0andopenai>=2.25.0,<3.0.0. Remove thetritonpin, becausetorchnow selectstriton. Theopenaicap matches theopenai==2.44.0pin in the vLLM CUDA test requirements.vllm_tests_requirements.txt: Replacetransformers==4.57.1and raiseopenai, because the old values conflict with vLLM 0.32.0.common.gradleandgpu_image_requirements.txt: Remove vLLM from the generated GPU requirements.Decision for reviewers: The GPU requirements no longer include vLLM. vLLM 0.32.0 pins
torch==2.13.0, and thetransformersextra capstorch<2.12.0because newer Linux wheels crash when imported after TensorFlow. A list with both extras has no solution. I kept the cap and removed vLLM from the list. No file in this repository reads the GPU requirements, and the vLLM benchmark image installs[gcp,vllm]. People who use the list to get vLLM must now install the extra. If vLLM must stay in the list, then the list must drop TensorFlow or the cap must change.The generated files: I did not run
generatePythonRequirementsAll, because it needs Linux x86_64 and a large install. I resolved the list withuv pip compileforx86_64-manylinux_2_28, with the current pins as preferences. The result removes vLLM and the 88 packages that only vLLM needed, and it changes no other pin. The next scheduled regeneration will move the remaining pins, includingtorch.Validated:
[gcp,vllm]resolves on Python 3.11 and 3.12 for Linux x86_64, with and withoutvllm_tests_requirements.txt..[vllm]from this branch installs on Python 3.11 in a Linux arm64 container.uv pip checkreports only thatnvidia-cusparselt-cu13was built for a different platform.vllm_inference_test.pypasses in that container.--model,--port,--max-num-seqs,--gpu-memory-utilization,--served-model-name, and--max-model-lenoptions. The container has no GPU, so I forced the vLLM CPU platform plugin for this check.VLLMCompletionsModelHandlerandVLLMChatModelHandlercomplete requests against a fake OpenAI endpoint withopenai2.54.0.Not validated: A real vLLM server, GPU inference, the Dynamo path, the benchmark image build, and Dataflow. The vLLM wheel cannot start a server without a GPU. Also, I did not install the extra on Python 3.13 or 3.14. The vLLM benchmark workflow is the best check for the GPU path.