Getting started

What is crawlie?

crawlie is a fast, free, open-source technical-SEO and GEO (Generative Engine Optimization — AI-search readiness) crawler. It runs locally, ships a CLI and an MCP server, and gives you plain-English fixes for every issue it finds.

One engine, many surfaces: a shared Rust core powers the CLI, the MCP server (for agents), and a Tauri desktop app.

Install

The CLI and MCP server ship together on npm. The right native binary installs automatically — nothing to download or unblock.

npm i -g crawlie

This puts two commands on your PATH:

• crawlie — the CLI
• crawlie-mcp — the MCP server for agents

No install needed at all? Run it on demand with npx:

npx -y crawlie crawl https://your-site.com

From source

Needs Rust. The desktop app additionally needs pnpm + Node.

git clone https://github.com/spronta/crawlie
cd crawlie
cargo build --release          # → target/release/crawlie and crawlie-mcp
cargo install --path crates/crawlie-cli
cargo install --path crates/crawlie-mcp

Your first crawl

Crawl a whole site and print a readable report:

crawlie crawl https://your-site.com --format pretty

Every crawl returns two scores: a Health score (technical SEO) and a GEO score (AI-search readiness). See Scoring.

Audit a single page, or a specific list:

crawlie audit https://your-site.com/pricing
crawlie audit https://your-site.com/a https://your-site.com/b

Learn why any finding matters and how to fix it:

crawlie explain geo-not-answerable

Where to go next

• CLI reference — every command, flag, and output format.
• MCP server — let an AI agent run and act on audits.
• Skills & plugin — one-step Claude Code setup.
• What it checks — the full rule catalogue.