Local Inference
Curated collection of thoughts and builds centered around Local Inference.
Daily Briefings 14
- Salesforce-NVIDIA reasoning model targets enterprise tasks; Apple ships on-device Siri with Gemini
- Reducto ships r-1 document parser at 1¢/page; OpenBMB releases MiniCPM5-2B small language model
- Perplexity open-sources Lily inference engine; Qwen releases local search layer
- Google's Gemini cuts video token usage 88%; Anthropic releases Fable 5.1 with 45% cost reduction
- FreeToken runs 753B MoE models on single GPU; agentic token consumption surges 14x
- NVIDIA Releases Nemotron 3.5 Lightning MoE and LTX-2.5 Open Video Model
- Liquid AI Releases On-Device Agentic Model; Microsoft Open-Sources Unit-Test Agent
- Meta launches Muse Code agent for large codebases; Mistral's 3B Shieldstral matches larger safety models
- Token Saver MCP cuts PDF costs 90-99%; Moonshot open-sources MoonEP for MoE training
- NVIDIA Releases Cosmos 3 Edge; Google's Frozen v2 Chip Targets 6-10x TPU Efficiency Gains
- Feyn Labs ships SQRL text-to-SQL family; Moonshot maxes GPU capacity on Kimi K3 in 48 hours
- Thinking Machines releases Inkling open model; PrismML compresses 27B reasoning model to iPhone
- PrismML ships 1-bit and ternary Qwen3.6-27B builds; Mistral's Robostral Navigate runs on one RGB camera
- Moebius 0.2B ported to run in-browser via Claude Code; research recasts prompt injection as role confusion