Patrick von Platen a54ed80249

[Model] Add mistral function calling format to all models loaded with "mistral" format (#8515) ...

Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>

2024-09-17 17:50:37 +00:00

..

fp8

[Hardware][NV] Add support for ModelOpt static scaling checkpoints. (#6112)

2024-09-11 00:38:40 -04:00

production_monitoring

…

api_client.py

…

aqlm_example.py

…

cpu_offload.py

…

gguf_inference.py

…

gradio_openai_chatbot_webserver.py

…

gradio_webserver.py

…

llm_engine_example.py

…

logging_configuration.md

…

lora_with_quantization_inference.py

…

multilora_inference.py

…

offline_chat_with_tools.py

[Model] Add mistral function calling format to all models loaded with "mistral" format (#8515)

2024-09-17 17:50:37 +00:00

offline_inference_arctic.py

…

offline_inference_audio_language.py

[Model] Add Ultravox support for multiple audio chunks (#7963)

2024-09-04 04:38:21 +00:00

offline_inference_chat.py

…

offline_inference_distributed.py

…

offline_inference_embedding.py

…

offline_inference_encoder_decoder.py

…

offline_inference_mlpspeculator.py

…

offline_inference_neuron_int8_quantization.py

[Neuron] Adding support for adding/ overriding neuron configuration a… (#8062)

2024-09-04 16:33:43 -07:00

offline_inference_neuron.py

[Neuron] Adding support for context-lenght, token-gen buckets. (#7885)

2024-08-29 13:58:14 -07:00

offline_inference_openai.md

…

offline_inference_pixtral.py

[Misc] Update Pixtral example (#8431)

2024-09-12 17:31:18 -07:00

offline_inference_tpu.py

…

offline_inference_vision_language_multi_image.py

[Model] Support multiple images for qwen-vl (#8247)

2024-09-12 10:10:54 -07:00

offline_inference_vision_language.py

[Model][VLM] Add Qwen2-VL model support (#7905)

2024-09-11 09:31:19 -07:00

offline_inference_with_prefix.py

…

offline_inference_with_profiler.py

[bugfix] torch profiler bug for single gpu with GPUExecutor (#8354)

2024-09-12 21:30:00 -07:00

offline_inference.py

…

openai_audio_api_client.py

…

openai_chat_completion_client_with_tools.py

[Feature] OpenAI-Compatible Tools API + Streaming for Hermes & Mistral models (#5649)

2024-09-04 13:18:13 -07:00

openai_chat_completion_client.py

…

openai_completion_client.py

…

openai_embedding_client.py

…

openai_example_batch.jsonl

…

openai_vision_api_client.py

[Doc] Indicate more information about supported modalities (#8181)

2024-09-05 10:51:53 +00:00

run_cluster.sh

…

save_sharded_state.py

…

template_alpaca.jinja

…

template_baichuan.jinja

…

template_blip2.jinja

…

template_chatglm2.jinja

…

template_chatglm.jinja

…

template_chatml.jinja

…

template_falcon_180b.jinja

…

template_falcon.jinja

…

template_inkbot.jinja

…

template_llava.jinja

…

tensorize_vllm_model.py

…

tool_chat_template_hermes.jinja

[Bugfix] Fix Hermes tool call chat template bug (#8256)

2024-09-07 10:49:01 +08:00

tool_chat_template_mistral_parallel.jinja

[Feature] OpenAI-Compatible Tools API + Streaming for Hermes & Mistral models (#5649)

2024-09-04 13:18:13 -07:00

tool_chat_template_mistral.jinja

[Feature] OpenAI-Compatible Tools API + Streaming for Hermes & Mistral models (#5649)

2024-09-04 13:18:13 -07:00