Provider-agnostic LLM gateway

The problem

Depending on a single LLM provider means their outages and rate limits become yours.

Approach

A self-hosted LiteLLM gateway routes across OpenAI, Anthropic, AWS Bedrock, and Vertex — load-balanced with same-model failover on rate-limit and a two-layer retry + backoff.