Screaming Frog SEO Spider is a desktop website crawler from Screaming Frog Ltd built for technical SEO audits — finding broken links, analyzing meta data, spotting duplicate content, rendering JavaScript, validating structured data, and generating XML sitemaps. It's aimed at SEO professionals and technical auditors who want a fast, locally run crawler rather than a cloud-based all-in-one suite, and it's free to use on sites up to 500 URLs.
Who builds it
Screaming Frog SEO Spider is developed by Screaming Frog Ltd, a UK-based company that has built its reputation almost entirely around this one crawler. Rather than expanding into a broad SEO suite, Screaming Frog focuses on being the desktop crawling tool that other SEO software and agencies rely on for deep, controllable, page-by-page site audits.
Core features
- Broken link, error & redirect detection, so you can find and fix crawl issues before they hurt rankings or user experience.
- Page title and meta data analysis, covering titles, descriptions, and other on-page elements at scale across a crawl.
- Meta robots and directive review, to confirm indexation and crawl instructions are set the way you intend.
- hreflang attribute auditing, useful for sites managing multiple language or regional versions.
- Exact duplicate page discovery, flagging content that risks cannibalizing itself in search.
- XML sitemap generation and site visualizations, turning a crawl into deliverables you can hand to a client or dev team.
- The vendor's own description also highlights JavaScript rendering and structured data validation as part of the crawler's technical-SEO toolkit.
Pricing
- Free version: free to use indefinitely, limited to crawling up to 500 URLs, and already includes broken-link detection, meta data analysis, hreflang auditing, duplicate discovery, and XML sitemap generation.
- Paid licence — from €245 per year: removes the 500-URL cap (the practical limit then depends on your machine's allocated memory and storage) and unlocks the software's advanced features.
- Pricing is published per year, not per month, reflecting Screaming Frog's licence-based, not subscription-tier, model.
Who it's for
Screaming Frog SEO Spider is built for technical SEOs, agencies, and in-house teams that need a controllable, desktop-run crawler for structured, repeatable site audits — the kind of tool you point at a site (your own or a client's) and interrogate directly, rather than a platform with its own keyword or backlink database. Its free version is generous enough for small sites or spot-checks, while the €245/year licence is aimed at anyone auditing larger sites regularly. It pairs naturally with a separate keyword-research or backlink tool, since Screaming Frog's own data, per our data set, doesn't extend to keyword or backlink indexes — it crawls the site you give it, which keeps the licence cost low relative to full SEO suites while leaving database-driven research to dedicated tools.