LLMs on homelab Kubernetes: llm-d, vLLM, HAMi and Kaito
GPU sharing, vLLM and cache-aware routing on hardware you own, and how much of the production AI inference stack you can learn at home.
Mohamed Belgaied Hassine
I'm a Consulting Architect at Mirantis
I write about Kubernetes, virtualization and local AI Inference, and the parts of running infrastructure on your own hardware.
New here? Start with the Harvester guide.
Latest
macOS Local Network Privacy: why your CLI gets EHOSTUNREACH and how to fix it
Why a CLI or a background service cannot reach your LAN on macOS, what EHOSTUNREACH really means, and the signed .app wrapper that fixes it.
Rancher Turtles with Harvester: Cluster API in practice
Deploy Kubernetes clusters on Harvester VMs with Rancher Turtles and Cluster API, from installing the provider to importing the cluster.
Harvester networking: attaching a VM to a bridged network
Give a Harvester VM an address on your LAN: configure a host network interface, add a VLAN, and attach the VM with bridged networking.
Start here
- Harvester HCI: what it is and how to install it — start here if you have never run Harvester
- Harvester networking: attaching a VM to a bridged network — the step that makes a VM reachable from your LAN
- Rancher Turtles with Harvester — let Cluster API provision clusters on Harvester VMs
- macOS Local Network Privacy — why your CLI gets EHOSTUNREACH on the LAN, and the fix
Topics
About
I work on Kubernetes and virtualization at Mirantis, and I run most of it at home before writing about it. More about me.