agent.json: Why Your Site Needs a robots.txt for AI Agents
AI agents are browsing the web at scale now — checkout flows, search bars, dashboards. Without guidance, every agent screenshots your pages, dumps the DOM into an LLM, and guesses what to click.
We built agent.json as an open standard that lets websites declare their capabilities to AI agents. Same idea as robots.txt, but instead of "don't crawl this," it says "here's how to search, here's the add-to-cart button, here's the API shortcut."
How It Works
Place a JSON file at /.well-known/agent.json on your domain. Inside, declare capabilities:
{
"capabilities": [
{
"name": "search_products",
"description": "Search the product catalog",
"selector": "input#search-bar",
"entry_url": "/search"
}
]
}
Agents read this file first, get a structured execution plan, and interact with your site predictably.
Why This Matters
Browser automation is shifting from scripted Selenium/Playwright to AI-driven agents. The current approach — screenshot everything, send to LLM, guess what to click — burns tokens and produces unreliable results.
With agent.json, you control how agents interact with your site. Predictable behavior, reduced server load, and explicit capability exposure.
The Broader System: AIR SDK
agent.json is one piece of the Agent Internet Runtime (AIR). Even without it, the AIR SDK learns from agent interactions and builds verified execution plans.
The three-step pattern:
- Browse — Check what's known about a domain
- Execute — Get a pre-verified plan
- Report — Feed back results
Over 2,225 domains indexed. The SDK works with any agent framework.
Check out the open spec. What capabilities would you expose to AI agents?