Self-hosted · BYOK · Written in Rustv1.0.0-rc.3
Arandu is the AI gateway that runs entirely on your infrastructure. Use any model you want with your own keys — with a spending limit enforced before you spend, a firewall that redacts sensitive data before it leaves, and an audit trail nobody can erase.
We never see your data. We couldn’t even if we wanted to: Arandu runs on your server, not ours.
The problem is nobody knows how much you’re spending, what data is leaving, or how to prove any of it to an auditor. Arandu closes all three.
A loop, an oversized prompt, someone else’s script. Every wrapper tells you after the money is gone.
Arandu rejects the request before it ever calls the provider.
It estimates the cost before spending it and stops if it breaks the cap. Budgets per tier, API key, user, model and provider. Idempotent accounting: a retry never bills you twice. And local models are exempt — running on your own hardware costs you nothing.
This is BYOK’s real hole, and almost nobody says it out loud: if you use a cloud model, the data does leave — to your provider.
Arandu redacts it first — even while the model is still writing.
It scans input, output and the stream itself. In redact mode the literal value never gets through; in block mode it stops before the provider. It holds a 4,096-character window so a value split across two chunks can’t slip through the crack. The event is logged; the content never is.
No traceability, no answer for the risk committee. And storing prompts to get it just trades one problem for a worse one.
An audit log made immutable by the database itself.
Not “we promise not to delete it”: a Postgres trigger makes it unalterable. Not even a DBA with direct access can touch it. It records who, which model, what it cost and why it routed there — without storing a single prompt.
This is the part others hide. We’d rather you audit it.
Point Arandu at your own Ollama, vLLM or LM Studio. The prompt never leaves your network — not to a provider, not to us. And it doesn’t touch your budget: it’s your hardware.
Bring your OpenAI key, or any OpenAI-compatible provider. The prompt travels from your server to the provider you chose, on your contract at your price — after clearing the firewall. We are not in that path, and we never resell tokens.
Only the license heartbeat: license id, instance id, version and timestamp. Not a prompt, not a document, not a credential. That is the one and only outbound connection.
None of this is roadmap. Everything below is in the code, with tests that exercise it.
You pay for people working at the same time, not for accounts created. Onboard 50, pay for 10. Unlimited accounts on every plan.
Arandu never resells tokens. The AI cost stays between you and your provider, at your volume discount. Credentials are encrypted with AES-256-GCM under a master key that lives outside the database.
The same adapter serves OpenAI and your on-prem Ollama, at full parity: same routing, same budget, same audit trail, same firewall.
Okta, Entra or Google Workspace. No SSO tax: we don’t charge extra for the secure front door. Tested against a real identity server signing real tokens, not a mock.
Circuit breaker per model: if your provider dies at 3am, Arandu stops sending it traffic and probes carefully until it recovers. Nobody has to wake up.
Idempotency-Key at payment-gateway level, streaming included. A retry returns the same response, not a second invoice.
Vector indexing on Qdrant. Embeddings can also run against your local runtime: the promise that your documents never leave your network holds at indexing time too, not just at chat time.
Several recovery paths, including a command-line one for the owner. No hand surgery on the database.
If you’re handing your AI keys to a process, you have a right to know how it’s built. Every point here is verifiable.
Roughly 100,000 lines with zero unsafe. The process holding your OpenAI keys cannot suffer a buffer overflow. That’s guaranteed by the compiler, not by a code review.
unwrap, panic and expect are forbidden by lint at runtime. It runs on your infra: there’s no on-call engineer of ours to restart it at 3am. It has to not fall over.
We prefer Podman: no daemon and no root means less attack surface, not just less weight. If your company already runs Docker, it works just the same — the installer detects which one you have.
Every OCI image is signed with cosign and we publish the public key. You don’t have to take our word for it — verify the signature before running anything. Each release ships a CycloneDX SBOM, ready for your client’s security questionnaire.
The domain knows nothing about infrastructure. Changing provider, vector store or cache means implementing an interface, not rebuilding the product. Anti lock-in you can inspect, not just be promised.
Measured: 16 ms to first token, about 30 ms of overhead. Auditing, budgeting and filtering cost you no speed.
Saying it saves us both time. If you need anything on this list, we’re not for you yet.
We’re finishing the release audit before opening downloads. If you’re a company that wants to evaluate it, or you want to see the product from the inside, write to us — you’ll get an answer from the people who built it.