People David Idea I tested both vllm-metal and vllm-mlx on the M3 Ultra Mac Studio to see which one gives better multi-user LLM throughput on a high-memory Apple Silicon machine. Details * Both projects showed up in late 2025 - vllm on Metal/MLX is very exciting because it should let
People David Idea Testing Nvidia's v1 LLM Router blueprint as a third approach to intelligent query routing - this time using policy-based task classification instead of semantic similarity or neural network intent matching. Details * Nvidia's v1 LLM Router blueprint (main branch) takes a three-step approach: apply
People: David Idea: Tested UIUC's LLMRouter framework as an alternative to LiteLLM's semantic routing - this one trains an actual neural network for intent classification and can run on hardware as small as a Raspberry Pi with 4GB of ram. Details: * Wanted to compare this against
People: David Idea: Testing semantic routing as a way to automatically send LLM requests to different models based on what the user is asking—part of a bigger vision for smart edge-to-hub-to-cloud routing on our Maui cluster. Details: * Our LiteLLM proxy already bundles multiple machines into one API endpoint, so
People: David Idea: Testing KIRA, Krafton's open-source project that lets you run a full Claude Code instance through Slack, on our local PMF hardware. Details: * Nicole mentioned wanting more Slack bot and automation options for the team * Found Krafton's KIRA project on GitHub which does exactly