A self-hosted AI agent runs on hardware you control instead of a vendor's cloud. You operate four parts yourself: a model runtime that serves an open-weight model, an orchestrator that runs the agent loop, the tools it may call, and a memory store. The payoff is data control; the cost is that security becomes your job.
| Part | What it does | Open-source examples |
|---|---|---|
| Model runtime | Loads a model and answers requests over a local API | vLLM, LocalAI |
| Orchestrator | Runs the loop: ask the model, call a tool, feed back the result | LangGraph, CrewAI, smolagents, n8n |
| Tools | Functions or MCP servers the agent may call | Filesystem MCP server, your own scripts |
| Memory | Stores conversations, documents and state | A database you already run |
Updated
Self-hosting appeals to businesses that cannot send customer records to a third party, or that want fixed costs instead of per-request bills. It is real engineering work, though: you patch the servers, size the hardware and decide what the agent may touch. This page walks through each part in plain words and ends with a checklist you can tick through before anything goes live.
Tick what is already true. Your ticks stay in this browser tab only.
A sketch of the network idea, not a full deployment. Only the orchestrator is published to the host; the runtime has no published port.
The runtime sits only on an internal network with no route out; the agent joins it and the default network. Replace the image names with the runtime and agent you chose.
services:
runtime:
image: your-model-runtime:pinned-version
networks: [private]
# no "ports:" here, so nothing outside can reach it
agent:
image: your-agent:pinned-version
environment:
MODEL_URL: http://runtime:8000
ports:
- "127.0.0.1:3000:3000"
networks: [private, default]
networks:
private:
internal: trueThe runtime only answers "given this text, what comes next". It has no idea what an order or a booking is. The orchestrator turns those answers into actions by reading the model's request for a tool, running it, and passing back the result.
Keeping them apart means you can swap the model without rewriting the agent, and you can lock the runtime away on a private network while the orchestrator handles the outside world.
OWASP names excessive agency as a core risk: an agent with too many functions, too many permissions or too much autonomy. In a self-hosted setup you set all three, so set them low.
The MCP project's security guidance adds a point specific to local tool servers: they run on your machine with your access, so only install ones from sources you trust and review what each one can do.
It can be, because prompts and data stay on machines you control. That only holds if the runtime, logs and backups are also kept private, and if no tool quietly sends data to an outside service.
It depends heavily on the model size and how many people use it at once, so we will not quote a number. As one data point, vLLM's install guide lists Linux and a GPU with compute capability 7.5 or higher, while LocalAI says no GPU is required. Test on rented hardware first.
Some workflow tools can be self-hosted. n8n, for example, offers self-hosting under its own Sustainable Use License, a fair-code licence with limits that differ from MIT or Apache, so read it before commercial use.
Prompt injection: text the agent reads, such as an email or web page, that tricks it into acting against you. OWASP lists it first in its risks for language model apps. Limiting tools and requiring human approval for high-impact actions are its main defences.
Related: How to build an AI agent, What is an MCP server?, Open-source agent frameworks, Filesystem MCP server, AI agent test checklist.
Start for free or see all free tools. We.Inc is free to start, with no card needed. AI building is on paid plans from $20 a month.
We.Inc is an AI-powered website builder you can resell under your own brand. Launch a branded client dashboard, bill on Stripe Connect, and deliver AI-generated websites in minutes. White-label plans start at $99 a month for 25 client sites, with a 7-day free trial and no per-site fees.