xinyun/vllm
1
0
Fork 0
mirror of https://git.datalinker.icu/vllm-project/vllm.git synced 2025-12-18 06:45:01 +08:00
Code Issues Packages Projects Releases Wiki Activity
6,801 Commits 155 Branches 89 Tags
Commit Graph

51 Commits

Author SHA1 Message Date
Mark McLoughlin
0e98964e94
[V1][Metrics] Remove metrics that were deprecated in 0.8 (#18837) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-05-28 18:54:12 +00:00
Harry Mellor
a1fe24d961
Migrate docs from Sphinx to MkDocs (#18145) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
 2025-05-23 02:09:53 -07:00
Cyrus Leung
4213475ec7
[Metrics] Fix minor inconsistencies in bucket progression (#17262) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-04-27 16:19:39 +00:00
Flex Wang
18445edd0f
[Misc] Change buckets of histogram_iteration_tokens to [1, 8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096, 8096] to represent number of tokens (#17033) 
Signed-off-by: sfc-gh-zhwang <flex.wang@snowflake.com>
 2025-04-27 12:30:53 +00:00
Kay Yan
86fc2321ff
[Metrics] Add bucket for request_latency, time_to_first_token and time_per_output_token (#15202) 
Signed-off-by: Kay Yan <kay.yan@daocloud.io>
 2025-04-06 20:34:51 -07:00
Mark McLoughlin
98d7367b61
[Metrics] Hide deprecated metrics (#15458) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-04-02 07:37:19 -07:00
Mark McLoughlin
c8525f06fc
[V0][Metrics] Deprecate some questionable request time metrics (#14135) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-03-04 15:11:33 +00:00
Mark McLoughlin
2dfdfed8a0
[V0][Metrics] Deprecate some KV/prefix cache metrics (#14136) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-03-03 18:25:46 +00:00
Mark McLoughlin
c41d27156b
[V0][Metrics] Remove unimplemented vllm:tokens_total (#14134) 
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
 2025-03-03 17:50:22 +00:00
Mark McLoughlin
2cb8c1540e
[Metrics] Add --show-hidden-metrics-for-version CLI arg (#13295) 2025-02-22 00:20:45 -08:00
Mark McLoughlin
1cd981da4f
[V1][Metrics] Support vllm:cache_config_info (#13299) 2025-02-22 00:20:00 -08:00
Jewon Lee
bf3e05215c
[Misc] Fix typo at comments at metrics.py (#13024) 2025-02-11 08:20:37 -08:00
Russell Bryant
e489ad7a21
[Misc] Add SPDX-License-Identifier headers to python source files (#12628) 
- **Add SPDX license headers to python source files**
- **Check for SPDX headers using pre-commit**

commit 9d7ef44c3cfb72ca4c32e1c677d99259d10d4745
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:18:24 2025 -0500

    Add SPDX license headers to python source files
    
This commit adds SPDX license headers to python source files as
recommended to
the project by the Linux Foundation. These headers provide a concise way
that is
both human and machine readable for communicating license information
for each
source file. It helps avoid any ambiguity about the license of the code
and can
    also be easily used by tools to help manage license compliance.
    
The Linux Foundation runs license scans against the codebase to help
ensure
    we are in compliance with the licenses of the code we use, including
dependencies. Having these headers in place helps that tool do its job.
    
    More information can be found on the SPDX site:
    
    - https://spdx.dev/learn/handling-license-info/
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

commit 5a1cf1cb3b80759131c73f6a9dddebccac039dea
Author: Russell Bryant <rbryant@redhat.com>
Date:   Fri Jan 31 14:36:32 2025 -0500

    Check for SPDX headers using pre-commit
    
    Signed-off-by: Russell Bryant <rbryant@redhat.com>

---------

Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2025-02-02 11:58:18 -08:00
Cyrus Leung
df5dafaa5b
[Misc] Remove deprecated code (#12383) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-01-24 14:45:20 -05:00
youkaichao
6e650f56a1
[torch.compile] decouple compile sizes and cudagraph sizes (#12243) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-01-24 02:01:30 +08:00
youkaichao
ebf778061d
monitor metrics of tokens per step using cudagraph batchsizes (#11031) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2024-12-09 22:35:36 -08:00
tomeras91
7c32b6861e
[Frontend] correctly record prefill and decode time metrics (#10853) 
Signed-off-by: Tomer Asida <tomera@ai21.com>
 2024-12-03 19:13:31 +00:00
cduk
b7954776fd
[core] Avoid metrics log noise when idle - include speculative decodi… (#10809) 2024-12-02 01:49:48 +00:00
Russell Bryant
efa9084628
[Core] Avoid metrics log noise when idle (#8868) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2024-11-19 21:05:25 +00:00
Travis Johnson
272e31c0bd
[Bugfix] Guard for negative counter metrics to prevent crash (#10430) 
Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
 2024-11-19 04:57:10 +00:00
harrywu
874f551b36
[Metrics] add more metrics (#4464) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2024-11-12 00:17:38 +08:00
tomeras91
ac04a97a9f
[Frontend] Add max_tokens prometheus metric (#9881) 
Signed-off-by: Tomer Asida <tomera@ai21.com>
 2024-11-04 22:53:24 +00:00
Kunjan
0ad216f575
[MISC] Set label value to timestamp over 0, to keep track of recent history (#9777) 
Signed-off-by: Kunjan Patel <kunjanp@google.com>
 2024-10-29 19:52:19 +00:00
科英
74fc2d77ae
[Misc] Add metrics for request queue time, forward time, and execute time (#9659) 2024-10-29 10:32:56 -07:00
Kunjan
9bb10a7d27
[MISC] Add lora requests to metrics (#9477) 
Co-authored-by: Kunjan Patel <kunjanp_google_com@vllm.us-central1-a.c.kunjanp-gke-dev-2.internal>
 2024-10-18 20:50:18 +00:00
Russell Bryant
776dbd74f1
[CI/Build] mypy: Resolve some errors from checking vllm/engine (#9267) 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
 2024-10-16 22:55:59 +00:00
youkaichao
cbc2ef5529
[misc] hide best_of from engine (#9261) 
Co-authored-by: Brendan Wong <bjwpokemon@gmail.com>
 2024-10-10 21:30:44 -07:00
Cody Yu
3ac50b47d0
[MISC] Add prefix cache hit rate to metrics (#7606) 2024-08-19 11:52:07 -07:00
Robert Shaw
e3b318216d
[ Bugfix ] Fix Prometheus Metrics With zeromq Frontend (#7279) 
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
 2024-08-18 20:19:48 +00:00
Thomas Parnell
2f808e69ab
[Bugfix] StatLoggers: cache spec decode metrics when they get collected. (#6645) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2024-07-23 23:05:05 +00:00
Antoni Baum
5f0b9933e6
[Bugfix] Fix Ray Metrics API usage (#6354) 2024-07-17 19:40:10 +00:00
Cody Yu
160e1d8c99
[Misc] Log spec decode metrics (#6454) 2024-07-16 20:37:10 +00:00
Thomas Parnell
7508a3dc34
[Misc] Fix typos in spec. decode metrics logging. (#6470) 
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
 2024-07-16 13:55:15 +00:00
sroy745
80ca1e6a3a
[Speculative Decoding 2/2 ] Integrate typical acceptance sampler into Spec Decode Worker (#5348) 2024-07-01 00:33:05 -07:00
William Lin
906a19cdb0
[Misc] Extend vLLM Metrics logging API (#5925) 
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
 2024-06-29 10:36:06 +08:00
Cyrus Leung
0e9164b40a
[mypy] Enable type checking for test directory (#5017) 2024-06-15 04:45:31 +00:00
SangBin Cho
e7c46b9527
[Scheduler] Warning upon preemption and Swapping (#4647) 
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
 2024-05-13 23:50:44 +09:00
Robert Shaw
4dc8026d86
[Bugfix] Fix 307 Redirect for /metrics (#4523) 2024-05-01 09:14:13 -07:00
Ronen Schaffer
bf480c5302
Add more Prometheus metrics (#2764) 
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
 2024-04-28 15:59:33 -07:00
Roy
87f545ba6f
[Misc] Fix logger format typo (#4396) 2024-04-27 13:45:02 +08:00
SangBin Cho
a88081bf76
[CI] Disable non-lazy string operation on logging (#4326) 
Co-authored-by: Danny Guinther <dguinther@neuralmagic.com>
 2024-04-26 00:16:58 -07:00
Cade Daniel
62b8aebc6f
[Speculative decoding 7/9] Speculative decoding end-to-end correctness tests. (#3951) 2024-04-23 08:02:36 +00:00
zspo
ec8e3c695f
[Bugfix] fix_log_time_in_metrics (#4050) 2024-04-13 07:52:36 -07:00
Michael Feil
c2b4a1bce9
[Doc] Add typing hints / mypy types cleanup (#3816) 
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
 2024-04-11 17:17:21 -07:00
SangBin Cho
01bfb22b41
[CI] Try introducing isort. (#3495) 2024-03-25 07:59:47 -07:00
Zhuohan Li
2f8844ba08
Re-enable the 80 char line width limit (#3305) 2024-03-10 19:49:14 -07:00
Allen.Dou
29e70e3e88
allow user chose log level by --log-level instead of fixed 'info'. (#3109) 
Co-authored-by: zixiao <shunli.dsl@alibaba-inc.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
 2024-03-01 23:28:41 +00:00
Allen.Dou
9289e577ec
add cache_config's info to prometheus metrics. (#3100) 2024-02-29 06:15:18 +00:00
Harry Mellor
ef978fe411
Port metrics from aioprometheus to prometheus_client (#2730) 2024-02-25 11:54:00 -08:00
Robert Shaw
93b38bea5d
Refactor Prometheus and Add Request Level Metrics (#2316) 2024-01-31 14:58:07 -08:00