Make your site crawlable by AI agents in minutes.
npx agre-cli
That's it. No install, no config, no account. Works on any machine with Node.js 20+.
npx agre-cli --site-url https://example.com
npx agre-cli generate all --site-url https://example.comPrefer npx for one-off usage, or install globally:
npm i -g agre-cli
agre-cli --helpRun npx agre-cli in any web project root and it:
- Detects your framework — Next.js, Nuxt, Astro, SvelteKit, Remix, Vite, plain HTML
- Scans all routes and extracts metadata (title, description, content type)
- Prints a scored checklist of AI-crawlability issues (0–100, A–F grade)
- Generates
llms.txtandllms-full.txtin the correct public directory - Analyzes
robots.txtand patches it to allow all known AI bots - Detects CSR-only pages invisible to AI crawlers
- Generates Schema.org JSON-LD snippets for key page types
- Shows an agent preview — what an AI bot actually sees from a route file
┌─────────────────────────────────────────────────┐
│ │
│ agre-cli v1.0.0 │
│ Make your site crawlable by AI agents │
│ │
└─────────────────────────────────────────────────┘
◆ Detected: Next.js (App Router) — next.config.ts
◆ Found 23 routes
CRITICAL ──────────────────────────────────────────
✗ llms.txt missing Not found in public/
✗ llms-full.txt missing Not found in public/
HIGH ──────────────────────────────────────────────
✗ ClaudeBot allowed ClaudeBot: Disallow: /
✓ GPTBot allowed GPTBot: Allow: /
MEDIUM ────────────────────────────────────────────
◐ Schema.org — 0/23 pages No JSON-LD found
✓ sitemap.xml exists
✓ Meta descriptions — 19/23 pages
Score: 44/100 Grade: D
npx agre-cli # full interactive scan (default)
npx agre-cli scan # scan only, print checklist, no writes
npx agre-cli generate llms # generate llms.txt
npx agre-cli generate llms-full # generate llms-full.txt
npx agre-cli generate robots # patch robots.txt to allow AI bots
npx agre-cli generate schema # generate Schema.org JSON-LD snippets
npx agre-cli generate all # generate everything
npx agre-cli preview [file] # show what an AI bot sees from a route
npx agre-cli init # create agre-cli.config.ts--cwd <path> Working directory (default: cwd)
--site-url <url> Production URL (e.g. https://myapp.com)
--out <dir> Output directory override
--yes, -y Skip confirmation prompts
--dry-run Show output without writing files
--json Output as JSON (for piping/scripting)
--ci CI mode: non-interactive, exit 1 on critical failures
--quiet, -q Suppress non-error output
--verbose Debug-level output
--config <path> Path to agre-cli.config.tsCreate an agre-cli.config.ts in your project root (or run agre-cli init):
import type { agreConfig } from 'agre-cli';
const config: agreConfig = {
siteUrl: 'https://myapp.com',
siteName: 'MyApp',
siteDescription: 'The fastest way to ship AI-ready web apps.',
excludeRoutes: ['/internal/**', '/preview/**'],
includeApiRoutes: false,
includeDynamicRoutes: false,
tokenBudget: 80000,
};
export default config;| Framework | Detection | Public Dir | Routes |
|---|---|---|---|
| Next.js (App Router) | next.config.* + app/ |
public/ |
app/**/page.{tsx,ts,jsx,js} |
| Next.js (Pages Router) | next.config.* + pages/ |
public/ |
pages/**/*.{tsx,ts,jsx,js} |
| Nuxt 3 | nuxt.config.* |
public/ |
pages/**/*.vue |
| Astro | astro.config.* |
public/ |
src/pages/**/*.{astro,md,mdx} |
| SvelteKit | svelte.config.* |
static/ |
src/routes/**/+page.svelte |
| Remix | remix.config.* |
public/ |
app/routes/**/*.{tsx,ts,jsx,js} |
| Vite SPA | vite.config.* |
public/ |
index.html |
| Plain HTML | index.html in root |
root | **/*.html |
- name: Check AI crawlability
run: npx agre-cli scan --ci --site-url https://myapp.comExit code 1 if any critical checklist items fail.
# If this project is published as a GitHub Action:
- uses: <owner>/<repo>@v1
with:
site-url: https://myapp.com
generate: all
fail-on-critical: truenpx agre-cli scan --json | jq '.checklist.score'| ID | Severity | What it checks |
|---|---|---|
llms-txt-exists |
Critical | llms.txt exists in public dir |
llms-full-txt-exists |
High | llms-full.txt exists |
llms-txt-valid |
High | llms.txt has title, sections, links |
robots-allows-gptbot |
High | GPTBot not blocked |
robots-allows-claudebot |
High | ClaudeBot not blocked |
robots-allows-perplexity |
Medium | PerplexityBot not blocked |
no-csr-only-pages |
High | No client-side-only pages |
sitemap-exists |
Medium | sitemap.xml found |
homepage-has-schema |
Medium | JSON-LD on homepage |
meta-descriptions |
Medium | ≥80% pages have descriptions |
import {
detect,
scanRoutes,
runChecklist,
generateLlmsTxt,
analyzeRobots,
generateSchema,
previewAgent,
} from 'agre-cli';
const framework = await detect(process.cwd());
const routes = await scanRoutes(process.cwd(), framework);
const result = await runChecklist(process.cwd(), framework, routes);
console.log(`Score: ${result.score}/100 (${result.grade})`);pnpm install
pnpm lint # Run ESLint
pnpm build # Build with tsup
pnpm test # Run vitest
pnpm test:coverage # Run tests with coverage
pnpm test:watch # Watch mode
pnpm typecheck # TypeScript type checking
pnpm dev # Build in watch modeEach checklist item has a severity weight:
- Critical: 35 points
- High: 25 points
- Medium: 15 points
- Low: 5 points
Pass = full points, Warn = half, Fail = 0, Skip = excluded.
| Grade | Score |
|---|---|
| A | 90–100 |
| B | 75–89 |
| C | 55–74 |
| D | 35–54 |
| F | 0–34 |
- Does not deploy or host anything
- Does not make HTTP requests (fully offline static analysis)
- Does not run a headless browser
- Does not require an account or API key
- Does not modify source code — generates new files and snippets only
MIT © agre-cli