Roger Wang f2ebb6f541

[V1] Scatter and gather placeholders in the model runner (#16076) ...

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Jennifer Zhao <ai.jenniferzhao@gmail.com>

2025-04-08 10:43:41 +08:00

..

__init__.py

[V1]SupportsV0Only protocol for model definitions (#13959)

2025-02-27 20:02:15 -05:00

adapters.py

[Bugfix] Fix missing return value in load_weights method of adapters.py (#15542)

2025-03-31 06:56:42 -07:00

arctic.py

Clean up unused padding_idx variables across many model definitions (#13240)

2025-03-04 21:27:00 +00:00

aria.py

[VLM] Clean up Phi-4-MM ViT implementation (#14812)

2025-03-15 18:53:52 -07:00

aya_vision.py

[V1] Scatter and gather placeholders in the model runner (#16076)

2025-04-08 10:43:41 +08:00

baichuan.py

[Model] use AutoWeightsLoader for baichuan, gpt-neox, mpt (#15939)

2025-04-04 09:38:52 -07:00

bamba.py

[Model] use AutoWeightsLoader in model load_weights (#15770)

2025-04-02 07:47:31 -07:00

bart.py

[Quant] BartModel SupportsQuant (#14699)

2025-03-13 04:55:59 +00:00

bert.py

[SupportsQuant] Bert, Blip, Blip2, Bloom (#15573)

2025-04-03 08:23:19 -07:00

blip2.py

[SupportsQuant] Bert, Blip, Blip2, Bloom (#15573)

2025-04-03 08:23:19 -07:00

blip.py

[SupportsQuant] Bert, Blip, Blip2, Bloom (#15573)

2025-04-03 08:23:19 -07:00

bloom.py

[SupportsQuant] Bert, Blip, Blip2, Bloom (#15573)

2025-04-03 08:23:19 -07:00

chameleon.py

[V1] Scatter and gather placeholders in the model runner (#16076)

2025-04-08 10:43:41 +08:00

chatglm.py

[SupportsQuant] Chameleon, Chatglm, Commandr (#15952)

2025-04-03 08:25:22 -07:00

clip.py

[Bugfix] Fix Positive Feature Layers in Llava Models (#13514)

2025-02-19 08:50:07 +00:00

commandr.py

[SupportsQuant] Chameleon, Chatglm, Commandr (#15952)

2025-04-03 08:25:22 -07:00

constant_size_cache.py

[Model][MiniMaxText01] Support MiniMaxText01 model inference (#13454)

2025-04-01 16:23:55 -04:00

dbrx.py

[V1] EP/TP MoE + DP Attention (#13931)

2025-03-04 21:27:26 -08:00

deepseek_mtp.py

[Model][Speculative Decoding] Expand DeepSeek MTP code to support k > n_predict (#13626)

2025-02-27 15:28:08 -08:00

deepseek_v2.py

[Frontend][Bugfix] support prefill decode disaggregation on deepseek (#14824)

2025-03-20 00:00:33 -07:00

deepseek_vl2.py

fix: Comments to English for better dev experience (#15768)

2025-03-30 10:47:57 -07:00

deepseek.py

[Model] use AutoWeightsLoader for phi, gemma, deepseek (#16088)

2025-04-05 20:34:38 -07:00

eagle.py

[Feature] Enhance EAGLE Architecture with Proper RMS Norms (#14990)

2025-03-26 08:24:07 +00:00

exaone.py

[Model] use AutoWeightsLoader in model load_weights (#15770)

2025-04-02 07:47:31 -07:00

fairseq2_llama.py

[Misc] Add SPDX-License-Identifier headers to python source files (#12628)

2025-02-02 11:58:18 -08:00

falcon.py

[Model] use AutoWeightsLoader in model load_weights (#15770)

2025-04-02 07:47:31 -07:00

florence2.py

[VLM] Florence-2 supports online serving (#16164)

2025-04-07 04:04:02 -07:00

fuyu.py

[V1] Scatter and gather placeholders in the model runner (#16076)

2025-04-08 10:43:41 +08:00

gemma2.py

[Misc] Remove misleading message in gemma2 and gemma3 (#14850)

2025-03-14 21:35:12 -07:00

gemma3_mm.py

[V1] Scatter and gather placeholders in the model runner (#16076)

2025-04-08 10:43:41 +08:00

gemma3.py

[Model] Add module name prefixes to gemma3 (#15889)

2025-04-01 10:13:40 -07:00

gemma.py

[Model] use AutoWeightsLoader for phi, gemma, deepseek (#16088)

2025-04-05 20:34:38 -07:00

glm4v.py

[Bugfix] Loosen type check to avoid errors in V1 (#15021)

2025-03-18 12:54:40 +00:00

glm.py

[V1] V1 Enablement Oracle (#13726)

2025-03-14 22:02:20 -07:00

gpt2.py

Remove unused kwargs from model definitions (#13555)

2025-02-24 17:13:52 -08:00

gpt_bigcode.py

Remove unused kwargs from model definitions (#13555)

2025-02-24 17:13:52 -08:00

gpt_j.py

Remove unused kwargs from model definitions (#13555)

2025-02-24 17:13:52 -08:00

gpt_neox.py

[Model] use AutoWeightsLoader for baichuan, gpt-neox, mpt (#15939)

2025-04-04 09:38:52 -07:00

granite.py

Clean up unused padding_idx variables across many model definitions (#13240)

2025-03-04 21:27:00 +00:00

granitemoe.py

Clean up unused padding_idx variables across many model definitions (#13240)

2025-03-04 21:27:00 +00:00

granitemoeshared.py

[Model] Add support for GraniteMoeShared models (#13313)

2025-03-04 08:04:52 +08:00

gritlm.py

[V1]SupportsV0Only protocol for model definitions (#13959)

2025-02-27 20:02:15 -05:00

grok1.py

[Model] Support Grok1 (#13795)

2025-02-26 01:07:12 +00:00

h2ovl.py

[V1] Scatter and gather placeholders in the model runner (#16076)

2025-04-08 10:43:41 +08:00

idefics2_vision_model.py

[VLM] Clean up Phi-4-MM ViT implementation (#14812)

2025-03-15 18:53:52 -07:00

idefics3.py

[V1] Scatter and gather placeholders in the model runner (#16076)

2025-04-08 10:43:41 +08:00

interfaces_base.py

Remove unused kwargs from model definitions (#13555)

2025-02-24 17:13:52 -08:00

interfaces.py

[Model] Update support for NemotronNAS models (#15008)

2025-03-31 20:35:14 +08:00

intern_vit.py

[Misc] Add SPDX-License-Identifier headers to python source files (#12628)

2025-02-02 11:58:18 -08:00

internlm2_ve.py

Remove unused kwargs from model definitions (#13555)

2025-02-24 17:13:52 -08:00

internlm2.py

Clean up unused padding_idx variables across many model definitions (#13240)

2025-03-04 21:27:00 +00:00

internvl.py

[V1] Scatter and gather placeholders in the model runner (#16076)

2025-04-08 10:43:41 +08:00

jais.py

Remove unused kwargs from model definitions (#13555)

2025-02-24 17:13:52 -08:00

jamba.py

[MODEL] Add support for Zamba2 models (#13185)

2025-03-18 08:56:21 -07:00

llama4.py

Upstream Llama4 Support to Main (#16113)

2025-04-07 08:06:27 -07:00

llama.py

Upstream Llama4 Support to Main (#16113)

2025-04-07 08:06:27 -07:00

llava_next_video.py

[Bugfix] Check dimensions of multimodal embeddings in V1 (#15816)

2025-03-31 09:01:35 -07:00

llava_next.py

[VLM] Various cleanup and fixes (#14806)

2025-03-14 05:58:19 -07:00

llava_onevision.py

[V1] Override mm_counts for dummy data creation (#15703)

2025-03-30 03:20:42 -07:00

llava.py

[V1] Scatter and gather placeholders in the model runner (#16076)

2025-04-08 10:43:41 +08:00

mamba2.py

[V1]SupportsV0Only protocol for model definitions (#13959)

2025-02-27 20:02:15 -05:00

mamba_cache.py

[Model][MiniMaxText01] Support MiniMaxText01 model inference (#13454)

2025-04-01 16:23:55 -04:00

mamba.py

Clean up unused padding_idx variables across many model definitions (#13240)

2025-03-04 21:27:00 +00:00

medusa.py

[Misc] Add SPDX-License-Identifier headers to python source files (#12628)

2025-02-02 11:58:18 -08:00

minicpm3.py

Remove unused kwargs from model definitions (#13555)

2025-02-24 17:13:52 -08:00

minicpm.py

Clean up unused padding_idx variables across many model definitions (#13240)

2025-03-04 21:27:00 +00:00

minicpmo.py

[V1] Scatter and gather placeholders in the model runner (#16076)

2025-04-08 10:43:41 +08:00

minicpmv.py

[V1] Scatter and gather placeholders in the model runner (#16076)

2025-04-08 10:43:41 +08:00

minimax_cache.py

[Model][MiniMaxText01] Support MiniMaxText01 model inference (#13454)

2025-04-01 16:23:55 -04:00

minimax_text_01.py

[Model][MiniMaxText01] Support MiniMaxText01 model inference (#13454)

2025-04-01 16:23:55 -04:00

mistral3.py

[V1] Scatter and gather placeholders in the model runner (#16076)

2025-04-08 10:43:41 +08:00

mixtral_quant.py

[Bugfix][Model] Mixtral: use unused head_dim config argument (#14961)

2025-03-17 07:44:18 -07:00

mixtral.py

[Bugfix][Model] Mixtral: use unused head_dim config argument (#14961)

2025-03-17 07:44:18 -07:00

mllama4.py

[V1] Scatter and gather placeholders in the model runner (#16076)

2025-04-08 10:43:41 +08:00

mllama.py

[Bugfix][Model] fix mllama multi-image (#14883)

2025-03-31 22:53:37 -07:00

mlp_speculator.py

[Doc] Replace ibm-fms with ibm-ai-platform (#12709)

2025-02-04 07:05:04 +00:00

module_mapping.py

[Misc] Add SPDX-License-Identifier headers to python source files (#12628)

2025-02-02 11:58:18 -08:00

molmo.py

[V1] Scatter and gather placeholders in the model runner (#16076)

2025-04-08 10:43:41 +08:00

mpt.py

[Model] use AutoWeightsLoader for baichuan, gpt-neox, mpt (#15939)

2025-04-04 09:38:52 -07:00

nemotron_nas.py

[Model] Update support for NemotronNAS models (#15008)

2025-03-31 20:35:14 +08:00

nemotron.py

Apply torchfix (#15532)

2025-03-26 12:09:06 +00:00

nvlm_d.py

[V1] Scatter and gather placeholders in the model runner (#16076)

2025-04-08 10:43:41 +08:00

olmo2.py

[Bugfix] Fix interface for Olmo2 on V1 (#14976)

2025-03-17 11:26:38 -07:00

olmo.py

Remove unused kwargs from model definitions (#13555)

2025-02-24 17:13:52 -08:00

olmoe.py

[V1] EP/TP MoE + DP Attention (#13931)

2025-03-04 21:27:26 -08:00

opt.py

Clean up unused padding_idx variables across many model definitions (#13240)

2025-03-04 21:27:00 +00:00

orion.py

Clean up unused padding_idx variables across many model definitions (#13240)

2025-03-04 21:27:00 +00:00

paligemma.py

[V1] Scatter and gather placeholders in the model runner (#16076)

2025-04-08 10:43:41 +08:00

persimmon.py

Remove unused kwargs from model definitions (#13555)

2025-02-24 17:13:52 -08:00

phi3_small.py

Remove unused kwargs from model definitions (#13555)

2025-02-24 17:13:52 -08:00

phi3.py

[Misc] Add SPDX-License-Identifier headers to python source files (#12628)

2025-02-02 11:58:18 -08:00

phi3v.py

[V1] Scatter and gather placeholders in the model runner (#16076)

2025-04-08 10:43:41 +08:00

phi4mm_audio.py

[VLM] Add TP support for Phi-4-MM (#14453)

2025-03-08 05:57:14 -08:00

phi4mm_utils.py

Apply torchfix (#15532)

2025-03-26 12:09:06 +00:00

phi4mm.py

[Bugfix] Fix mm_hashes forgetting to be passed (#15668)

2025-03-28 05:51:05 +00:00

phi.py

[Model] use AutoWeightsLoader for phi, gemma, deepseek (#16088)

2025-04-05 20:34:38 -07:00

phimoe.py

[V1] EP/TP MoE + DP Attention (#13931)

2025-03-04 21:27:26 -08:00

pixtral.py

[V1] Scatter and gather placeholders in the model runner (#16076)

2025-04-08 10:43:41 +08:00

prithvi_geospatial_mae.py

[Bugfix] Fix mm_hashes forgetting to be passed (#15668)

2025-03-28 05:51:05 +00:00

qwen2_5_vl.py

[Misc] Add attention mask pre-computation optimization back to Qwen2.5-VL (#15273)

2025-03-21 10:32:33 +00:00

qwen2_audio.py

[V1] Scatter and gather placeholders in the model runner (#16076)

2025-04-08 10:43:41 +08:00

qwen2_moe.py

[V1] EP/TP MoE + DP Attention (#13931)

2025-03-04 21:27:26 -08:00

qwen2_rm.py

[V1]SupportsV0Only protocol for model definitions (#13959)

2025-02-27 20:02:15 -05:00

qwen2_vl.py

[V1] Override mm_counts for dummy data creation (#15703)

2025-03-30 03:20:42 -07:00

qwen2.py

[Model] Add Qwen3 and Qwen3MoE (#15289)

2025-04-07 04:06:41 -07:00

qwen3_moe.py

[Model] Add Qwen3 and Qwen3MoE (#15289)

2025-04-07 04:06:41 -07:00

qwen3.py

[Model] Add Qwen3 and Qwen3MoE (#15289)

2025-04-07 04:06:41 -07:00

qwen_vl.py

[V1] Scatter and gather placeholders in the model runner (#16076)

2025-04-08 10:43:41 +08:00

qwen.py

[Bugfix] Fix prompt format of GLM4V (#14539)

2025-03-13 11:37:17 +00:00

registry.py

[Bugfix] Re-enable support for ChatGLMForConditionalGeneration (#16187)

2025-04-07 23:15:58 +08:00

roberta.py

[New Model]: jinaai/jina-reranker-v2-base-multilingual (#15876)

2025-04-01 08:32:26 -07:00

siglip.py

Allow torchao quantization in SiglipMLP (#15575)

2025-03-26 22:45:51 -07:00

skyworkr1v.py

[V1] Scatter and gather placeholders in the model runner (#16076)

2025-04-08 10:43:41 +08:00

solar.py

Clean up unused padding_idx variables across many model definitions (#13240)

2025-03-04 21:27:00 +00:00

stablelm.py

[Model] use AutoWeightsLoader for stablelm,starcoder2,zamba2 (#16103)

2025-04-06 05:52:01 -07:00

starcoder2.py

[Model] use AutoWeightsLoader for stablelm,starcoder2,zamba2 (#16103)

2025-04-06 05:52:01 -07:00

telechat2.py

Upstream Llama4 Support to Main (#16113)

2025-04-07 08:06:27 -07:00

teleflm.py

Upstream Llama4 Support to Main (#16113)

2025-04-07 08:06:27 -07:00

transformers.py

Rename fallback model and refactor supported models section (#15829)

2025-03-31 22:49:41 -07:00

ultravox.py

[Bugfix] embed_is_patch for Idefics3 (#15696)

2025-03-28 08:27:52 -07:00

utils.py

[Bugfix] Fix Crashing When Loading Modules With Batchnorm Stats (#15813)

2025-03-31 13:23:53 +00:00

vision.py

[V1] Scatter and gather placeholders in the model runner (#16076)

2025-04-08 10:43:41 +08:00

whisper.py

[Bugfix] Make dummy encoder prompt padding alternative and add missing warnings (#16129)

2025-04-07 04:07:15 +00:00

zamba2.py

[Model] use AutoWeightsLoader for stablelm,starcoder2,zamba2 (#16103)

2025-04-06 05:52:01 -07:00