# You Don't need to use Cloud AI! Switchyard and Nemotron 3.5 Lightning

Level1Techs (Wendell) argues enterprises are overspending on frontier cloud AI and should adopt observable model routing plus customizable local models — centered on NVIDIA NeMo Switchyard and Nemotron 3.5 Lightning. Companion forum write-up expands the research trail and economics.

Tags: youtube-inbox, source, nvidia, switchyard, nemotron, local-ai, model-routing, data-flywheel, level1techs

## Summary

Most ordinary corporate and coding tasks do not need frontier-model spend ($12–$25/M tokens in the speaker's framing). NVIDIA's **NeMo Switchyard** (open-source router/proxy) plus **Nemotron 3.5 Lightning** (≈30B MoE / ≈3B active, LoRA/SFT/RL recipes) make a practical "data flywheel" reachable for IT: route → observe → evaluate → specialize → cheaper local inference → fewer frontier calls. Observability and institutional learning matter as much as token savings.

## Key points

- **Problem**: Blind cloud AI spend + "turn it loose" tooling that removes human supervision; Codex CLI called out for reduced visibility into agent reasoning/traces.
- **Switchyard**: Supervision/routing layer that sends requests to specialized local, customized, or stronger models; supports protocol translation (OpenAI Chat/Responses, Anthropic Messages), metrics, classifier/stage/escalation routing; can front Claude Code / Codex.
- **Nemotron 3.5 Lightning**: ~30B MoE with ~3B active params; pitched for high-volume agentic work and easy customization (LoRA, SFT, RL recipes/data under OpenMDW-1.1); NIM can hot-load multiple LoRA adapters on one base model.
- **Prior art in stack**: Nemotron Orchestrator 8B / ToolOrchestra — NVIDIA reported orchestrated system 37.1% vs GPT-5 35.1% on Humanity's Last Exam at ~30% cost and 2.5× speed (2025 numbers cited in video).
- **Three advantages of the structure**: (1) cost — don't spend Opus money on Quinn-class tasks; (2) observability — know which component did what; (3) organizational learning — routing/escalation/corrections become a secondary dataset.
- **Demo hardware**: HP Z8 Fury with dual RTX PRO 6000 (96GB each; platform up to 384GB GPU memory); small + large local models + RAG/vector DB; frontier still reachable via router as exception path.
- **Flywheel**: Instrument production traffic → eval/fine-tune datasets → customize smaller models → promote winners; older NVIDIA Data Flywheel Blueprint deprecated for new prod but still conceptual.
- **Qwen 3.6 angle**: Disruption is customization docs/ecosystem (MS-SWIFT, LoRA/QLoRA/SFT/GRPO/DPO/PPO), not just benchmark scores.
- **LoRA ops story**: Accounting / code-review / support / "Gary's dark knowledge" adapters on one resident base model.
- **Thesis outcomes**: Frontier inference becomes the exception; company accumulates its own intelligence instead of only renting it.
- **Caveats from forum discussion**: Capex/TCO vs marginal $/token; utilization required for payback; router quality gaps (LLMRouterBench); observability alone does not require dual Blackwells; silent "enstupification" risk if routing is opaque.

## Quotes / memorable lines

- "Your company is burning money… setting money on fire on your AI tooling."
- Frontier model "cosplaying as an omniscient intelligence."
- "You can't fix what you don't measure."
- Days of Opus "helping you remember the order that the arguments should go in" should be over.

## Sources

- YouTube: https://www.youtube.com/watch?v=-IGB6Avxwgo
- Raw ingest: `raw/youtube-inbox/-IGB6Avxwgo/` (metadata.json, transcript.txt, notes.md, source.url)
- Forum write-up: https://forum.level1techs.com/t/stop-wasting-cloud-tokens-local-ai-ftw/253759
- NVIDIA blog: https://blogs.nvidia.com/blog/nemotron-lightning-switchyard-rtx-dgx/
- Switchyard: https://github.com/nvidia-nemo/switchyard

## Related pages

- [[youtube-local-router-nemotron-lightning]] (Prompt Engineering: local Switchyard + Lightning + Kimi K3)