freeport

The open-source LLM gateway you run on your own infrastructure. OpenAI-compatible, compliance-grade, single container.

$ git clone github.com/ReallyArtificial/freeport && docker compose up

Run it yourself

Your prompts never leave your infrastructure.

Most gateways are someone else's cloud. freeport is a single container you deploy where your data already lives — VPC, on-prem, or air-gapped.

Compliance-grade by default

PII redaction, audit logging, encrypted provider keys and hard budget caps — the controls teams under GDPR, HIPAA or the EU AI Act actually need.

No vendor in the path

Requests go straight from your gateway to the model provider. No middleman sees your traffic, and there's no per-request SaaS tax.

Drop-in OpenAI API

Point any OpenAI SDK at freeport and keep your existing code. Routing, fallback and caching happen behind the same interface.

A complete gateway

Everything between your app and the model.

Each capability below is implemented and shipping in the open-source codebase.

Multi-provider routing

Route across OpenAI, Anthropic and Google Gemini through one unified, OpenAI-compatible API.

Fallback & circuit breaker

Automatic failover across providers with a circuit breaker, so one outage doesn't take you down.

Semantic cache

Similar prompts return cached answers using local embeddings (all-MiniLM-L6-v2) — no external calls.

Cost & budget caps

Per-project spend tracking with daily and monthly limits, plus an emergency kill switch.

PII & content guardrails

Detect SSNs, cards, emails and phone numbers, filter content and cap tokens. Pluggable for custom rules.

Audit logging

Every request and response is logged with usage analytics and configurable retention.

Encrypted keys

Provider API keys are stored encrypted with AES-256-GCM in the local database.

Rate limiting

Token-bucket rate limiter with per-key request limits to protect your providers.

Prompt versioning & A/B

Manage and version prompts externally, then split traffic between variants to compare them.

Prometheus metrics

Prometheus-compatible counters and histograms for latency, cost and request volume.

Admin UI

A built-in Preact dashboard to add providers, set budgets, inspect logs and manage prompts.

OpenAI-compatible

Chat, completions, embeddings and model listing — streaming included. Any OpenAI SDK just works.

Quickstart

Up and running in under a minute.

Two ways to start. The gateway and admin UI come up together on :4000.

Run with Docker

$ git clone https://github.com/ReallyArtificial/freeport
$ cd freeport
$ docker compose up

Run from source

$ npm install
$ npm run build:ui
$ npm run dev

Point your OpenAI SDK at it

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:4000/v1",
    api_key="any-key",
)

resp = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello!"}],
)

No config files required to start. Add provider keys through the admin UI at /ui/, environment variables, or a YAML config — your choice.

Project status

Honest about where we are.

freeport is the gateway described on this page. It's open-source under the MIT license at version 1.0.0. Every feature listed above is implemented in src/, not roadmap. Built on Fastify v5 and better-sqlite3, it runs on Node 20+ and ships as a single Docker container.

We also maintain mcp-jest, a testing tool for MCP servers. No usage metrics, benchmarks or testimonials are shown here because we'd rather earn them than fake them.