I built a thing: memserv
I had a long weekend and a small itch to scratch, so I built memserv — a self-hosted memory server for AI agents.
The short version: I wanted one place where my various Claude clients and home-grown agents could store and recall things across sessions, instead of every tool keeping its own little island of context. memserv is that place. It wraps mem0 and a Qdrant vector database behind a single service, and — this is the part I like — it exposes that shared memory two different ways at once.
Two protocols, one memory store
- REST (
/api/v1/memories…) for scripts and custom agents that just want to make HTTP calls. - Streamable HTTP MCP (
/mcp/) so Claude Code, Claude Desktop, and Claude.ai on the web can talk to it natively as a Model Context Protocol server.
Both front ends read and write the same memories. So a script can stash something via REST, and the next time I’m chatting in Claude Desktop, it’s already there. It’s a single-user system by design — there’s one MEM0_DEFAULT_USER_ID and that user is me — which kept the whole thing pleasantly simple.
The build
A boring-in-the-good-way stack, which was the point:
- FastAPI with FastMCP mounted into it, so REST and MCP live in one app
- Qdrant for vector storage
- Python throughout
- Docker Compose that bundles Qdrant and the app together for local runs
- CapRover for deployment, auto-deploying on pushes to
main - A companion
mem0-backupapp that takes nightly Qdrant snapshots to S3
Auth comes in two phases: a static bearer token to get going, and OAuth 2.1 + PKCE with Dynamic Client Registration for the grown-up version. Starting with a bearer token meant I could be talking to the thing in an afternoon and worry about the proper OAuth dance later.
The fiddliest part, predictably, was the MCP transport — getting Streamable HTTP to behave across the different Claude clients is exactly the kind of thing that looks like a one-liner until it very much isn’t. But it’s running, it remembers things, and my agents finally share a brain.
If you want to stand up your own, the README walks through both the Docker Compose and CapRover paths. As always, I’d love to hear if it’s useful — or if I’ve gotten anything wrong.