Screaming Frog SEO Spider 1.8

Screaming Frog SEO Spider 1.8 is developed to be a helpful and beneficial desktop program which spiders websites’ links, images, CSS, script and apps from an SEO perspective

It fetches key onsite page elements for SEO, presents them in tabs by type and allows you to filter for common SEO issues, or slice and dice the data how you see fit by exporting into Excel. You can view, analyse and filter the crawl data as it’s gathered and updated continuously in the program’s user interface.

The Screaming Frog SEO Spider allows you to quickly analyse, audit and review a site from an onsite SEO perspective. It’s particulary good for analysing medium to large sites where manually checking every page would be extremely labour intensive (or impossible!) and where you can easily miss a redirect, meta refresh or duplicate page issue.

Major Features:

1. Errors – Client & server errors (No responses, 4XX, 5XX)
2. Redirects – (3XX, permanent or temporary)
3. External Links – All followed links and their subsequent status codes
4. URI Issues – Non ASCII characters, underscores, uppercase characters, dynamic uris, long over 115 characters
5. Duplicate Pages – Hash value / MD5checksums lookup for pages with duplicate content
6. Page Title – Missing, duplicate, over 70 characters, same as h1, multiple
7. Meta Description – Missing, duplicate, over 156 characters, multiple
8. Meta Keywords – Mainly for reference as it’s only (barely) used by Yahoo.
9. H1 – Missing, duplicate, over 70 characters, multiple
10. H2 – Missing, duplicate, over 70 characters, multiple
11. Meta Robots – Index, noindex, follow, nofollow, noarchive, nosnippet, noodp, noydir etc
12. Meta Refresh – Including target page and time delay
13. Canonical link element & Canonical HTTP headers
14. X-Robots-Tag
15. File Size
16. Page Depth Level
17. Inlinks – All pages linking to a URI
18. Outlinks – All pages a URI links out to
19. Anchor Text – All link text. Alt text from images with links

Enhancements:

• The following new features are included in the update:
• • Crawl Speed Control – The ability to control the speed of the crawl is something they have wanted to release for a long time. they have had some requests for throttle control due to the SEO spider crawling too fast for some older servers. Hence, they have introduced a ‘speed’ feature which allows you to limit the number of threads or the number of URI requests per second. This feature also allows you to significantly increase the speed of the SEO spider at the same time, which theyjust ask everyone uses responsibly :-).
  • XML Sitemaps – they have included the ability to create XML sitemaps in the SEO spider under the top level menu ‘export’ option. Currently this feature is only for HTML pages, so it does not include images, videos etc (which they will be introducing at a later date). theyconform to the standards outlined in sitemaps.org protocol. If you have over 49,999 urls the SEO spider will automatically create additional sitemap files and create a sitemap index file referencing the sitemap locations.
  • Exclude Is Now Regex – Much like the ‘include’ feature, the ‘exclude’ option in the SEO spider is now regex. In previously versions it was robots.txt syntax, but regex provides more control and better balance.
  • Bulk Export Missing Alt Text URI – they have included the ability to bulk export all images with missing alt text and the URI that reference them.
  • List Mode Accepts XML Sitemap Files – they added the ability in ‘list’ mode to be able to upload and crawl .xml files. Hence, if you already have an XML sitemap and wish to audit it, simply switch to ‘list’ mode, change the filetype to ‘.xml’ and upload it. they will discover all the URLs automatically and crawl them.
• They also fixed up a couple of bugs
• • Level – They did not always find the shortest path. Now they do.
  • Https – There was a bug crawling secure pages in version 1.70. This has been fixed!

Requirements:

• Mac OS X 10.0 or later