Guillaume Calmettes abbfb6134d

[Misc][OpenAI] deprecate max_tokens in favor of new max_completion_tokens field for chat completion endpoint (#9837)

2024-10-30 18:15:56 -07:00

..

fp8

…

production_monitoring

…

api_client.py

…

aqlm_example.py

…

cpu_offload.py

…

florence2_inference.py

[Model] Initialize Florence-2 language backbone support (#9555)

2024-10-23 10:42:47 +00:00

gguf_inference.py

…

gradio_openai_chatbot_webserver.py

…

gradio_webserver.py

…

llm_engine_example.py

[core] remove beam search from the core (#9105)

2024-10-07 05:47:04 +00:00

logging_configuration.md

…

lora_with_quantization_inference.py

[[Misc]Upgrade bitsandbytes to the latest version 0.44.0 (#8768)

2024-09-24 17:08:55 -07:00

multilora_inference.py

[core] remove beam search from the core (#9105)

2024-10-07 05:47:04 +00:00

offline_chat_with_tools.py

[Model] Add mistral function calling format to all models loaded with "mistral" format (#8515)

2024-09-17 17:50:37 +00:00

offline_inference_arctic.py

…

offline_inference_audio_language.py

[Model] Add Qwen2-Audio model support (#9248)

2024-10-23 17:54:22 +00:00

offline_inference_chat.py

[Frontend] Batch inference for llm.chat() API (#8648)

2024-09-24 09:44:11 -07:00

offline_inference_distributed.py

…

offline_inference_embedding.py

…

offline_inference_encoder_decoder.py

…

offline_inference_mlpspeculator.py

[Core] Deprecating block manager v1 and make block manager v2 default (#8704)

2024-10-17 11:38:15 -05:00

offline_inference_neuron_int8_quantization.py

…

offline_inference_neuron.py

…

offline_inference_openai.md

[Misc][OpenAI] deprecate max_tokens in favor of new max_completion_tokens field for chat completion endpoint (#9837)

2024-10-30 18:15:56 -07:00

offline_inference_pixtral.py

…

offline_inference_tpu.py

…

offline_inference_vision_language_embedding.py

[Model] Support E5-V (#9576)

2024-10-23 11:35:29 +08:00

offline_inference_vision_language_multi_image.py

[Model] Support E5-V (#9576)

2024-10-23 11:35:29 +08:00

offline_inference_vision_language.py

[Model] Add min_pixels / max_pixels to Qwen2VL as mm_processor_kwargs (#9612)

2024-10-23 14:05:18 +00:00

offline_inference_with_prefix.py

[MISC] Consolidate cleanup() and refactor offline_inference_with_prefix.py (#9510)

2024-10-18 14:30:55 -07:00

offline_inference_with_profiler.py

[bugfix] torch profiler bug for single gpu with GPUExecutor (#8354)

2024-09-12 21:30:00 -07:00

offline_inference.py

…

offline_profile.py

[misc] CUDA Time Layerwise Profiler (#8337)

2024-10-17 10:36:09 -04:00

openai_api_client_for_multimodal.py

[Misc][OpenAI] deprecate max_tokens in favor of new max_completion_tokens field for chat completion endpoint (#9837)

2024-10-30 18:15:56 -07:00

openai_chat_completion_client_with_tools.py

…

openai_chat_completion_client.py

…

openai_completion_client.py

…

openai_embedding_client.py

…

openai_example_batch.jsonl

[Misc][OpenAI] deprecate max_tokens in favor of new max_completion_tokens field for chat completion endpoint (#9837)

2024-10-30 18:15:56 -07:00

run_cluster.sh

…

save_sharded_state.py

…

template_alpaca.jinja

…

template_baichuan.jinja

…

template_blip2.jinja

…

template_chatglm2.jinja

…

template_chatglm.jinja

…

template_chatml.jinja

…

template_falcon_180b.jinja

…

template_falcon.jinja

…

template_inkbot.jinja

…

template_llava.jinja

…

tensorize_vllm_model.py

…

tool_chat_template_granite_20b_fc.jinja

[Model] tool calling support for ibm-granite/granite-20b-functioncalling (#8339)

2024-10-29 15:07:37 -07:00

tool_chat_template_hermes.jinja

…

tool_chat_template_internlm2_tool.jinja

[Frontend][Feature] support tool calling for internlm/internlm2_5-7b-chat model (#8405)

2024-10-04 10:36:39 +08:00

tool_chat_template_llama3.1_json.jinja

[Feature] Add support for Llama 3.1 and 3.2 tool use (#8343)

2024-09-26 17:01:42 -07:00

tool_chat_template_llama3.2_json.jinja

[Feature] Add support for Llama 3.1 and 3.2 tool use (#8343)

2024-09-26 17:01:42 -07:00

tool_chat_template_mistral_parallel.jinja

[Bugfix] example template should not add parallel_tool_prompt if tools is none (#9007)

2024-10-03 03:04:17 +00:00

tool_chat_template_mistral.jinja

…