Setting Up a Local LLM Lab

Cloud APIs are convenient until you need to probe for jailbreaks, strip the content policy, or run a thousand adversarial inputs without a bill. Running a model locally — on your own hardware, fully air-gapped, no rate limits — changes what you can do in an afternoon. This guide covers Ollama for the fast path, llama.cpp for control, vLLM for throughput, and how to point security tools like garak and PyRIT at the local OpenAI-compatible endpoint without changing a line of their code.