#vllm

vLLM + k8s on Bare Metal

Ollama on Ubuntu was an easy enough install, with the addition of OpenWebUI – we had an easy to use in-house tool – but we stopped short of voice chat, due to lack of TLS. The next project is to use a production grade tool – vllm on kubernetes.