OLMoE - GuideLLM - vLLM 0.6.0
vLLM 0.6.0 support now tool_choice auto and a specific template to run Mistral functions call exaclty the same way as La plateforme. You only need to use theses args :
--enable-auto-tool-choice
--tool-call-parser mistral
--chat-template example/tool_chat_template_mistral.jinja
Great!
Good to know
Thanks for sharing and looking forward to your review of GuideLLM!
vLLM 0.6.0 support now tool_choice auto and a specific template to run Mistral functions call exaclty the same way as La plateforme. You only need to use theses args :
--enable-auto-tool-choice
--tool-call-parser mistral
--chat-template example/tool_chat_template_mistral.jinja
Great!
Good to know
Thanks for sharing and looking forward to your review of GuideLLM!