david

(Post-WIP) Self-hosted distributed local LLMs load balanced proxy endpoint

* People * David * Idea * Single OpenAI-compatible LLM endpoint that can be reached over the internet, secured with an API key, bridges hardware options across 3+ GPU machines (Apple, Nvidia, etc) * Details * LiteLLM * LM Studio * TabbyAPI * Caddy * Cloudflare Tunnels+DNS