polygate logopolygate

One function.
Any LLM provider.

A tiny, dependency-light client that gives Anthropic, OpenAI, Gemini, and Moonshot one consistent request and response shape.

GitHub
from polygate import chat
 
response = chat(
    provider="anthropic",  # or "openai", "gemini", "moonshot"
    model="claude-sonnet-4-6",
    messages=[{"role": "user", "content": "Say hi in one word."}],
)
 
print(response.content)   # "Hi!"
print(response.usage)     # Usage(prompt_tokens=..., ...)

Swap providers by changing one string

Every adapter translates the same message list into whatever shape the provider actually expects. API keys come from the argument or fall back to the environment.

Anthropic

Claude models
"anthropic""claude"

ANTHROPIC_API_KEY

OpenAI

GPT models
"openai""gpt"

OPENAI_API_KEY

Google Gemini

Gemini models
"gemini""google"

GEMINI_API_KEY

Moonshot

Kimi models
"moonshot""kimi"

MOONSHOT_API_KEY

Adding a provider is one adapter file and one registry line. Contribute one

Every call returns the same shape

No more learning four response formats. Anything a provider returns that polygate does not normalize is still there, untouched, under raw.

Extra keyword arguments pass straight through to the provider, so provider-specific options like tools, top_p, or stop sequences keep working without polygate needing to know about them.

content
the assistant's reply text
role
always "assistant"
model
the model that was actually used
provider
which provider served the request
usage
prompt, completion, and total token counts
raw
the untouched original provider response

Rotate keys, ride out rate limits

A rate limit is per key, not per host. Sleeping through a 429 while another key sits idle wastes the whole backoff window, so polygate rotates first and only backs off once it runs out of keys.

A key the provider rejects outright is dropped for the rest of the process rather than retried forever. Deterministic failures are never retried at all: a 400, 404, or 422 cannot succeed on a second attempt, so sending it again just bills you twice.

Both features are off unless you ask for them. A single api_key with no retry behaves exactly as it always has.

from polygate import chat, Retry
 
response = chat(
    provider="anthropic",
    model="claude-sonnet-4-6",
    messages=[{"role": "user", "content": "Say hi."}],
 
    # One key, a list, or a reusable KeyPool.
    api_key=["sk-1", "sk-2", "sk-3"],
 
    # Omit retry= for the old behavior: one request, no retries.
    retry=Retry(max_attempts=4),
)

Small on purpose

polygate is not trying to out-feature LiteLLM, Portkey, or Helicone. It is the smallest layer that normalizes requests and responses across providers, with each adapter readable in under 100 lines. Key rotation and backoff are opt-in and off by default; caching and routing are still yours to build on top, with whatever opinions your project actually needs.

What it does

  • One chat function across all providers
  • One message format, one response shape
  • API keys from arguments or environment
  • Passthrough for provider-specific options

What it leaves to you

  • Retries and backoff
  • Caching and cost tracking
  • Routing and fallback logic
  • Framework integrations