diff --git a/docs/contributing/ci/update_pytorch_version.md b/docs/contributing/ci/update_pytorch_version.md index 735bb2e205332..74c0beb779c7d 100644 --- a/docs/contributing/ci/update_pytorch_version.md +++ b/docs/contributing/ci/update_pytorch_version.md @@ -77,25 +77,20 @@ This complicates the process as we cannot use the out-of-the-box - `.buildkite/release-pipeline.yaml` - `.buildkite/scripts/upload-wheels.sh` -## Address long vLLM build time +## Manually running vLLM builds on BuildKiteCI -When building vLLM with a new PyTorch/CUDA version, no cache will exist -in the vLLM sccache S3 bucket, causing the build job on CI to potentially take more than 5 hours -and timeout. Additionally, since vLLM's fastcheck pipeline runs in read-only mode, -it doesn't populate the cache, so re-running it to warm up the cache -is ineffective. +When building vLLM with a new PyTorch/CUDA version, the vLLM sccache S3 bucket +will not have any cached artifacts, which can cause CI build jobs to exceed 5 hours. +Furthermore, vLLM's fastcheck pipeline operates in read-only mode and does not +populate the cache, making it ineffective for cache warm-up purposes. -While ongoing efforts like -address the long build time at its source, the current workaround is to set `VLLM_CI_BRANCH` -to a custom branch provided by @khluu (`VLLM_CI_BRANCH=khluu/long_build`) -when manually triggering a build on Buildkite. This branch accomplishes two things: +To address this, manually trigger a build on Buildkite to accomplish two objectives: -1. Increase the timeout limit to 10 hours so that the build doesn't time out. -2. Allow the compiled artifacts to be written to the vLLM sccache S3 bucket -to warm it up so that future builds are faster. +1. Run the complete test suite against the PyTorch RC build by setting the environment variables: `RUN_ALL=1` and `NIGHTLY=1` +2. Populate the vLLM sccache S3 bucket with compiled artifacts, enabling faster subsequent builds

- Buildkite new build popup +Buildkite new build popup

## Update all the different vLLM platforms