crawlie
← Changelog

Release

crawlie v0.3.0

The biggest release yet: crawlie gains a deterministic content-quality engine, self-updating apps, and its own changelog.

Added

  • .crawlie rule packs — a deterministic, editable, agent-writable way to catch AI slop and off-brand copy. Rules are literal phrases, regexes, and text statistics (sentence burstiness, em-dash density, filler ratio, lexical diversity, repeated phrasing) — no model at runtime, so results are reproducible on every run. The new crawlie-rules engine is pure Rust and compiles to WebAssembly, so the same pack runs on your laptop, in CI, and in the cloud.
  • crawlie slop — score a whole site, a file, or stdin against a rule pack. You get an explainable ledger of exactly which rules fired and the evidence (--format json for agents), and you can gate your build with --fail-on-score.
  • Installable rule packs — crawlie init scaffolds an editable pack in your repo; crawlie pack add / list / new / which / remove manage packs across three layers (repo .crawlie/, global ~/.crawlie/packs, and built-in). Ships with a tunable slop-default pack you can copy and make your own.
  • CLI self-update — crawlie update checks for a newer release and installs it in place (--check to only look, -y to skip the prompt). A quiet nudge appears when you’re behind.
  • Desktop auto-update — the app now prompts when a new version is available and updates in-app in one click (download → install → restart). A new Settings area lets you toggle check on launch and fully automatic updates.

Changed

  • New app icon — the desktop app now wears the crawl-graph brand mark, matching the site and favicon.
  • Changelog moved to its own page — release notes now live at crawlie.dev/changelog with a dedicated, search-friendly page per release, plus an email subscribe option so you hear about new versions.

Fixed

  • Accurate compression auditing — crawlie now advertises Accept-Encoding and decodes gzip / Brotli / deflate itself, so the “no compression” check reflects what a server actually sends instead of being masked by the HTTP client’s transparent decompression.
  • Page body text is now exposed to content checks (previously only aggregate word counts were available).