Every Crab release, dated and categorised. New features, fixes, performance work, and UI polish.
Crab no longer opens itself when you start an AI assistant
If you connected Crab to Claude Code, Claude Desktop, Cursor or VS Code, Crab opened every time you started a session, even if you never asked about Crab. Now it starts only when the assistant actually uses one of Crab’s tools, such as starting a crawl or reading an audit.
Update notifications work again — please download this one by hand
Crab never told you about new versions: a build setting was missing, so every release since 1.1.0 skipped its update check. From 1.6.2, Crab checks at launch and every 30 minutes and installs updates in one click. If you’re on 1.6.1 or earlier, download this version once — future updates then arrive inside the app.
One click to move Crab into Applications, and updates that show up without a restart
Opened Crab straight from the downloaded disk image? It now offers to move itself into your Applications folder in one click, ejects the disk image and reopens. Running from the disk image meant Crab stopped working once it was ejected, couldn’t install updates, and gave AI assistants a path that broke.
Connect AI warns when Crab isn’t in Applications, instead of showing a setup that points at the disk image.
Crab checks for updates every 30 minutes while it’s open, not just at launch.
Use Crab from Claude, Cursor and VS Code — plus URL exclusions and one-click updates
Crab now has a built-in MCP server, so AI assistants can run crawls and read your audits. Ask “crawl example.com and list every broken link, grouped by the page I need to edit” or “what got worse since last week’s crawl?” Click Connect AI in the status bar for one-step setup with Claude Code, Claude Desktop, Codex, Gemini CLI, Cursor and VS Code.
When an issue affects hundreds of pages, the assistant gets a summary by folder and status code, with the full list saved to a file — a big site never floods the conversation.
Exclude URLs from a crawl. Under Advanced settings, add patterns like */tag/* or *?variant=* and matching URLs are never fetched. Patterns are saved with the project, so recrawls and scheduled crawls keep them.
Updates install from inside the app: Crab downloads the new version with a progress bar and, on macOS, installs and relaunches in one click after checking it’s signed by the same developer. This kicks in from the next update after 1.6.0.
The project Crawl dialog now matches New Crawl, with the project’s scope and exclusions pre-filled. Depth and Exclude URLs sit under a collapsible Advanced settings section.
The homepage was counted twice when a site linked to it both with and without a trailing slash, showing up as a duplicate of itself and lowering the health score. Re-crawl to clear it from existing results.
“Sitemap URLs Not Found in Crawl” no longer lists pages that were crawled.
Recrawling a Subfolder or Exact URL project no longer silently switches it to a whole-subdomain crawl, and the project list no longer shows stale “idle” or “running” status around a crawl.
macOS builds are signed and notarized by Apple — no more Terminal commands to install
Crab for macOS is now code-signed and notarized by Apple. Download the DMG, drag it to Applications, and open it — that’s the whole install. No security warning, and no more running `xattr -cr` in Terminal just to get started.
The bundled headless Chrome that powers crawling and PDF export is signed under the same identity, so macOS no longer blocks it on first launch under the hardened runtime.
Export any issue to CSV, and an audit report that's actually readable — now as PDF
Every issue on the Issues view exports on its own. Click the download icon on any row for a CSV of exactly the pages behind that count — no more switching to the Pages view and rebuilding the filter by hand.
Export the whole issues list in one file: every active issue together with the URLs behind each count.
The SEO audit report now exports as PDF, rendered by the Chrome that already ships with Crab — no extra download and no meaningful increase in app size.
The audit report was silently truncating. Issue text containing angle brackets (“Pages with more than one <title> element”) was written unescaped, which opened a real <title> element and swallowed the rest of the document — you saw one category out of ten. Reports generated before this version were missing most of their content and are worth regenerating.
The audit report lists the affected URLs under each issue, alongside a “Why it matters” explanation that was in the data but never shown.
Stopping a crawl could race: the JavaScript renderer and browser pool could be torn down while workers were still using them.
Unsaved schedule edits were discarded when a background refresh fired on window focus.
The site structure tree kept resetting your expand and collapse state on every refresh, and the site graph snapped back to origin mid-pan.
Reliable crawling of WAF-protected sites — smarter block detection and a one-click User-Agent switch
User-Agent block detection with a one-click fix. Many site firewalls (e.g. SiteGround) block Crab's Chrome user-agent outright. Crab now detects this after a crawl and shows a banner offering to switch to Safari and re-crawl in one click.
Sites on SiteGround and similar WAFs that returned 0 pages now crawl correctly — block detection extended to the X-Proxy-Cache-Info and Host-Header response headers, plus captcha / "Robot Challenge" interstitials that were previously stored as empty pages.
Crawls no longer trigger their own IP rate-limits. Crab stopped running its full WAF-bypass retry on every static asset (fonts, images, CSS, JS), which had been hammering protected origins into blocking the whole crawl.
JavaScript, CSS, font and image files were being listed as HTML pages — classification now trusts the URL extension over a mislabeled Content-Type.
Pages whose content fingerprint had its high bit set (roughly half of all values) were silently dropped by SQLite and never appeared in results.
Country-code domains like example.co.uk were scoped to co.uk, so every page looked external and the crawl stopped after one page. Now resolved via public-suffix eTLD+1 (.co.uk, .com.au, .co.nz, …).
Crawls could finish early — the completion check could close the queue before all sitemap URLs were enqueued, ending with fewer pages than expected.
429 Too Many Requests responses are now retried — Crab respects Retry-After (or waits 10s) and re-queues up to 3× instead of dropping the page.
Scheduling rejected standard cron expressions using ranges (1-5) or lists (8,12,18); the full cron syntax is now supported.
User-Agent presets updated to current browser versions (Chrome 131, Firefox 128, Safari 17.6).
Compare Crawls, big-site reliability, and the version label finally tells the truth
New Compare tab — diff any two crawls of the same site and see what's fixed and what's new since last time. Site Health, total issues, errors/warnings/notices and per-issue deltas with Fixed/New pills.
Cross-project comparison — if you've crawled the same domain multiple times across different projects, all of those snapshots show up in the Compare dropdowns, sorted by date.
Big sites no longer fail Issues with "Failed to load issues report" — the 15s endpoint timeout was killing the audit on 10k+ page sites. Bumped to 120s, results are now cached per crawl, and tab-switches don't re-run the audit.
Near-duplicate detection rewritten for big sites — the O(N²) self-join was scanning tens of millions of page pairs on whydonate-sized sites. Added a partial simhash index and a GROUP BY approximation above 5,000 pages.
App footer version now reads the actual build version instead of the v0.1.0 placeholder that had been there since launch.
Re-enable redirect prompt buttons
The redirect prompt modal was leaving its buttons disabled because isStarting got stuck true after the first click.
Preflight on the home-page New Crawl flow
The seed-URL preflight check (host-changing redirects, scheme upgrades) now runs from the home-page New Crawl entry point, not just from inside an existing project.
Detect host-changing redirects on the seed URL
When the seed URL 301s to a different host (e.g. http://crab.so → https://www.crab.so), Crab now surfaces the chain and asks before crawling under the wrong domain.
Eliminate "Loading projects forever" on Windows 11
Port walking — when 8080 is held by McAfee / IIS / Hyper-V the backend now walks to the next free port, and the frontend asks for the actual port via Wails instead of hard-assuming 8080.
Pinned API host to 127.0.0.1 to dodge the IPv4/IPv6 localhost resolution mismatch that some Win11 boxes were hitting.
Robust CSV export with friendly errors
CSV writes are now atomic (temp file + rename) so a half-written file is never left behind, and lock / permission failures get a readable error instead of a stack trace.
Honour <base href> for relative URL resolution
Pages with a <base href> tag now resolve relative links correctly per HTML spec §4.2.3 instead of treating the request URL as the base.
Comprehensive SEO audit engine — 62 issue checks
A full Issues dashboard with 62 issue types across HTML Tags, Content, Links, Images, Indexability, Structured Data, Performance, Headings, Security, and URL Structure — each with an actionable fix guide and impact explanation.
Weighted Site Health score that deducts based on severity × priority × percentage of pages affected, instead of binary pass/fail.
PostHog analytics, table sorting fixes, external-links restructure
PostHog product analytics wired into the desktop app.
Fixed several table sorting edge cases (numeric columns, nulls, locale).
External links restructured into a dedicated tab with source-page drill-in.
Cloudflare bypass with uTLS fingerprinting + adaptive rate limiting
uTLS fingerprinting so Crab presents a real Chrome TLS handshake to Cloudflare-protected sites; falls back to bundled Chrome when uTLS isn't enough; adaptive rate limiting backs off on 429s and gradually re-ramps.
Screaming Frog-style Images tab + External URLs tab + schema-type fixes
Dedicated Images tab with size, alt-text, and missing-alt filters.
External URLs tab grouped by target with source-page drill-in.
Several JSON-LD schema parsing bugs that were marking valid schema as invalid.
JavaScript rendering with bundled Chrome
JS rendering powered by a bundled Chrome headless shell — modes: off, auto (render only when needed), forced (always render). No need to install Chrome separately.
Sentry error tracking
Sentry wired into both the Go backend and React frontend with session replay on errors, so we actually see crashes when they happen.
Native-style pagination and layouts
Pagination, table rows, and toolbar layouts re-styled to feel native on each platform instead of webby.
Fix "failed to fetch" on Windows
CORS preflight was being rejected on Windows because the Wails asset server set a different Origin than the API expected.
Crab 1.0 — initial release
Crab launches as a free, local-only SEO crawler for macOS, Windows and Linux. No URL limits, no subscriptions, no data leaving your machine.
Found a bug or want to request a feature?