Skip to content

[Python] Upgrade the vLLM extra to 0.32.0 - #41005

Draft
bvolpato wants to merge 1 commit into
apache:masterfrom
bvolpato:bvolpato/bump-vllm
Draft

bvolpato wants to merge 1 commit into
apache:masterfrom
bvolpato:bvolpato/bump-vllm

Conversation

@bvolpato

@bvolpato bvolpato commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

This PR upgrades the vllm extra from 0.10.1.1 to 0.32.0. The old pin dates from August 2025 and declares python_requires<3.13, so the extra cannot install on Python 3.13 or 3.14. vLLM 0.32.0 declares support for Python versions below 3.15.

The new version needs newer dependencies, so this PR makes these changes:

  • setup.py: Pin vllm==0.32.0 and openai>=2.25.0,<3.0.0. Remove the triton pin, because torch now selects triton. The openai cap matches the openai==2.44.0 pin in the vLLM CUDA test requirements.
  • vllm_tests_requirements.txt: Replace transformers==4.57.1 and raise openai, because the old values conflict with vLLM 0.32.0.
  • common.gradle and gpu_image_requirements.txt: Remove vLLM from the generated GPU requirements.

Decision for reviewers: The GPU requirements no longer include vLLM. vLLM 0.32.0 pins torch==2.13.0, and the transformers extra caps torch<2.12.0 because newer Linux wheels crash when imported after TensorFlow. A list with both extras has no solution. I kept the cap and removed vLLM from the list. No file in this repository reads the GPU requirements, and the vLLM benchmark image installs [gcp,vllm]. People who use the list to get vLLM must now install the extra. If vLLM must stay in the list, then the list must drop TensorFlow or the cap must change.

The generated files: I did not run generatePythonRequirementsAll, because it needs Linux x86_64 and a large install. I resolved the list with uv pip compile for x86_64-manylinux_2_28, with the current pins as preferences. The result removes vLLM and the 88 packages that only vLLM needed, and it changes no other pin. The next scheduled regeneration will move the remaining pins, including torch.

Validated:

  • [gcp,vllm] resolves on Python 3.11 and 3.12 for Linux x86_64, with and without vllm_tests_requirements.txt.
  • .[vllm] from this branch installs on Python 3.11 in a Linux arm64 container. uv pip check reports only that nvidia-cusparselt-cu13 was built for a different platform.
  • vllm_inference_test.py passes in that container.
  • The vLLM 0.32.0 server CLI lists the --model, --port, --max-num-seqs, --gpu-memory-utilization, --served-model-name, and --max-model-len options. The container has no GPU, so I forced the vLLM CPU platform plugin for this check.
  • VLLMCompletionsModelHandler and VLLMChatModelHandler complete requests against a fake OpenAI endpoint with openai 2.54.0.

Not validated: A real vLLM server, GPU inference, the Dynamo path, the benchmark image build, and Dataflow. The vLLM wheel cannot start a server without a GPU. Also, I did not install the extra on Python 3.13 or 3.14. The vLLM benchmark workflow is the best check for the GPU path.

The old pin declares python_requires<3.13, so the extra cannot install on
Python 3.13 or 3.14.

Remove vLLM from the generated GPU requirements. vLLM 0.32.0 pins
torch==2.13.0, and the transformers extra caps torch<2.12.0.
@github-actions github-actions Bot added python docker size/m M: 50-249 lines changed labels Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docker python size/m M: 50-249 lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant