Nestjs · openai · google/genai · anthropic-ai/sdk
Backend · regular
A NestJS backend gateway for apps and IDEs. Clients call the native API or OpenAI/Anthropic facades, while routing to Anthropic, Gemini, OpenAI or compatible backends is driven by YAML model aliases.
AI Provider Gateway unifies access to multiple LLM providers behind one stable API, hiding SDK, auth, error and streaming differences while adding configuration, resilience and observability.
AI Provider Gateway is an HTTP microservice for LLM integrations. It exposes a native chat contract, SSE streaming, model catalog, and OpenAI/Anthropic-compatible facades for tools such as Cursor or Claude Code. Clients use gateway keys and model aliases, while the actual runtime provider is selected from YAML configuration. The system handles validation, retries, timeouts, fallback, smart rate limiting, exact/semantic response caching, requestId propagation, health/readiness, metrics, logs and security tests.
The project brings multiple LLM vendors behind one gateway layer. The main challenges were working with different provider SDKs and contracts, designing readable configuration files for provider and model instances, and optimizing cache layers, including exact and semantic caching. It was also a practical step in learning NestJS at an application architecture level.
Gallery
