SearXNG: The Underrated Search Infrastructure for Agents

When Search Becomes Part of the System

In the first wave of agent stacks, web search defaulted to a commercial provider: Tavily, Serper, Exa, Brave Search API, Perplexity. A cent per query, an API key per agent, a rate limit per customer that you don't control.

That works until search stops being a function of your agent and becomes the most-called function. The cent per query turns into a line in your cost of goods, and the rate limit turns into a single point of failure.

SearXNG is the answer that has been around for years and is being rediscovered in agent setups right now. 31.8k GitHub stars, AGPL-3.0, a self-hosted metasearch engine that bundles 200+ search engines under one JSON API. No tracking, no profiles, no vendor lock-in.

What You Actually Get

• One endpoint, many engines. Google, Bing, DuckDuckGo, Brave, Wikipedia, GitHub, Stack Overflow, arXiv, Reddit, YouTube, Mastodon – combinable per query, filterable per category.
• JSON output. format=json on /search is enough. An agent tool that parses it takes ten minutes to write.
• Categories, not engines. categories=science triggers the right sources, so your agent doesn't have to learn an engine-selection heuristic.
• No API keys, no quotas. You run the instance, you set the limits. That's a feature and a responsibility.
• Privacy by design. No logs by default, no profiles, no cookies. For agents asking questions on behalf of customers, that's substantive, not just ideological.

Setup is a Docker Compose file. Caddy or Traefik in front, Redis beside it, done.

Why It's Landing in Harnesses

An agent that searches ten times per conversation is normal in 2026. Three shifts make SearXNG interesting for that:

1. Custom harnesses are going mainstream. Anyone running a DeerFlow- or Claude-Code-style agent owns the tool layer. A search tool you start with docker compose up fits better than another SaaS contract.
2. Cost per query matters. A hundred agent sessions a day with ten searches per session at 0.5¢ per query gets you to about 150 EUR a month – just for search. SearXNG on a 20 EUR VM does the same job.
3. Data residency. EU customers ask where the search queries end up. "On our Hetzner VM in Falkenstein" beats "with a US provider under a DPA".

The Honest Limits

SearXNG isn't a better Tavily for every use case.

• AGPL-3.0. Running it standalone is fine. Embedding SearXNG code into your product and shipping the product is a license question your legal team should read. For our clients: SearXNG runs as a service behind the API, not as a library in the codebase – which makes AGPL a non-issue in most setups.
• Upstream rate limits. Google blocks scraping aggressively. A SearXNG instance pushing 1,000 queries an hour through Google without a Tor or proxy pool will see CAPTCHAs after a few hours. Brave Search as an engine is much friendlier; many teams run SearXNG with an engine whitelist instead of all 200.
• No "answer", only links. SearXNG is metasearch, not an answer generator. If you want Tavily's raw_content convenience, you'll need Crawl4AI or Firecrawl alongside it – standard in the stacks that adopt SearXNG anyway.
• Ops is part of the deal. A service that goes down falls back on you. Health checks, Redis, updates, engine configuration – manageable, not zero.

How We'd Build It

For an internal agent setup with moderate volume:

That lets you swap every existing Tavily or Serper call in your agent without a prompt change. That's the pragmatic win: no re-engineering, just a tool swap.

When SearXNG Is Not the Right Call

• Volume under 10,000 queries a month – don't run a service, pay Tavily 5 EUR.
• You need a hard SLA on search but don't want an on-call setup – a commercial API is the more honest choice.
• Your agent runs on-device at the end customer – there's no central instance that can host it.

For everything in between – which is most of the custom agent work we're building right now – SearXNG belongs on the shortlist at minimum.

→ DeerFlow 2.0: ByteDance's super-agent harness → Agent runtime comparison: Claude, LangGraph, CrewAI → Agent sandboxing approaches compared