nginx

Tag · 2 stories

Oct 3, 2026AI Infra

Lock Down a Self-Hosted LLM Server: Ollama and vLLM Defaults

Ollama has no authentication for local requests, and vLLM’s --api-key covers only some paths. What an exposed inference server gives away, and the setup that closes it: loopback binding, an allowlisting proxy and closed ports.

ollama
vllm
llm-security

Sep 14, 2026AI Infra

Stream LLM Responses Through a Reverse Proxy Without Buffering

Tokens stream on localhost, then arrive late and in clumps behind a production proxy. How nginx, HAProxy, Caddy and Cloudflare can hold them, how to survive idle timeouts, and a probe that measures per-event latency and fails CI when events are held.

sse
llm-streaming
nginx

Stories tagged nginx

Lock Down a Self-Hosted LLM Server: Ollama and vLLM Defaults

Stream LLM Responses Through a Reverse Proxy Without Buffering

The Stackonomist Brief