Screaming Frog excels at comprehensive technical SEO audits because it crawls your site like a search engine would. You'll identify critical issues affecting indexation, discover optimization opportunities, and build actionable fix lists with concrete priorities. The tool's strength lies in its ability to surface problems that Google Search Console might miss and provide the granular data needed to justify technical SEO investments.
This guide covers the full audit workflow from initial crawl setup through generating client-ready reports. You won't get generic advice about "improving SEO" — instead, you'll learn the specific configurations, filters, and data exports that separate surface-level audits from thorough technical reviews.
What You'll Need
You'll need Screaming Frog SEO Spider installed (free version works for sites under 500 URLs), access to your site's analytics and Search Console data, and familiarity with basic HTML concepts. For larger sites, the paid version ($259/year) removes crawl limits and adds advanced features like JavaScript rendering and Google Analytics integration.
Step 1: Configure Crawl Settings
Time: 15 minutes | Tool: Screaming Frog Open Screaming Frog and access Configuration > Spider settings before starting your crawl. Set the crawl speed to "Slower" under the Limits tab — this prevents server overload and reduces the chance of getting blocked. Increase the timeout to 30 seconds for slower-loading pages. Under the User-Agent tab, switch from the default Screaming Frog user-agent to Googlebot if you want to see exactly what Google's crawler encounters. This reveals JavaScript-dependent content issues that might not surface with the default crawler. Navigate to Configuration > Include and add your target subdirectories if you're auditing specific site sections. For e-commerce sites, prioritize /category/ and /product/ paths. Exclude admin areas, search result pages, and pagination parameters by adding them to Configuration > Exclude unless they're specifically part of your audit scope.
Step 2: Run Initial Crawl and Identify Scope
Time: 30-90 minutes | Tool: Screaming Frog Enter your target URL in the address bar and click Start. Watch the crawl progress in the bottom status bar — you'll see URLs Found, URLs Crawled, and any errors in real-time. Large sites can take hours to crawl completely. Stop the crawl once you've captured your core pages or hit your analysis threshold. Click on the "Response Codes" tab to get your first view of site health. Sort by Status Code to group 4xx and 5xx errors together — these represent your highest-priority technical issues. Export the response codes data by clicking "Export" and selecting "Response Codes > All Response Codes." You'll use this data to quantify error rates and build fix priority lists. Sites with more than 5% error rates typically have serious crawlability problems that affect organic performance.
Step 3: Analyze Title Tags and Meta Descriptions
Time: 20 minutes | Tool: Screaming Frog Click the "Page Titles" tab to review title tag issues. The interface shows Missing, Duplicate, Too Long, and Too Short tabs — each represents a different optimization opportunity. Duplicate titles often indicate template issues or lack of dynamic content management. Sort pages by URL to identify patterns in title problems. E-commerce category pages frequently share identical titles, while blog posts might have templated formats that create artificial duplication. Export the "Duplicate Titles" data to prioritize pages by organic traffic potential. Switch to the "Meta Description" tab and repeat the analysis. Missing meta descriptions won't hurt rankings directly, but they reduce click-through rates when Google generates automatic snippets. Duplicate meta descriptions across high-value pages represent lost differentiation opportunities in search results.
Step 4: Review Internal Link Structure
Time: 25 minutes | Tool: Screaming Frog Navigate to the "Internal" tab under the main tabs. This view shows your site's internal linking structure with metrics for each page's internal link count. Sort by "Inlinks" to identify orphaned pages (zero incoming internal links) and over-optimized hub pages. Click on any URL in the main window, then check the "Inlinks" tab in the lower panel to see which pages link to it and with what anchor text. Pages with strong organic potential but few internal links represent quick optimization wins. Export the "All Inlinks" data to build internal linking strategies. Look for high-authority pages (those with many inlinks) that could pass link equity to important conversion pages through strategic internal linking additions.
Step 5: Audit Image Optimization
Time: 15 minutes | Tool: Screaming Frog Click the "Images" tab to review image-related SEO issues. The interface shows tabs for Missing Alt Text, Alt Text Too Long, and Large Images. Missing alt text hurts accessibility and wastes opportunities to include relevant keywords. Sort the "Missing Alt Text" results by image file size to prioritize larger, more prominent images. Product images and hero graphics without alt text should be fixed first since they're more likely to appear in image search results. Use the "Large Images" tab to identify optimization opportunities. Images over 100KB can slow page load times significantly. Export this data to provide your development team with a prioritized list of images to compress or optimize.
Step 6: Check URL Structure and Redirects
Time: 30 minutes | Tool: Screaming Frog Return to the main "Internal" tab and examine URL structures by sorting alphabetically. Look for URL patterns that indicate technical problems — excessive parameters, session IDs, or inconsistent subdirectory structures. Click on the "Response Codes" tab and filter by 3xx redirects to map your site's redirect chains. Multiple redirects in sequence waste crawl budget and can cause indexation delays. Export redirect data to identify chains longer than one hop. Check the "Redirect Chains" report under the "Reports" menu. This automatically identifies URLs with multiple redirects and calculates the total chain length. Chains longer than five redirects often indicate migration problems or poor redirect management.
Step 7: Generate Audit Report and Prioritize Fixes
Time: 20 minutes | Tool: Screaming Frog Use the "Reports" menu to access Screaming Frog's built-in audit reports. The "Crawl Overview" provides summary statistics perfect for executive presentations, while the "Redirect & Canonical Issues" report identifies indexation problems. Export critical data sets for further analysis: Response Codes (for error tracking), Page Titles (for content optimization), and URL Structure (for technical cleanup). Import this data into spreadsheets to add traffic data from Google Analytics and create business-impact prioritization. Create your fix priority matrix by combining Screaming Frog error data with organic traffic values. Critical errors on high-traffic pages get immediate attention, while minor issues on low-value pages can wait for future optimization cycles.
Pro Tips
Enable JavaScript rendering in Configuration > Spider > Rendering if you're auditing a React or Angular site. This reveals content and links that wouldn't appear in a standard crawl, but increases crawl time significantly. Connect Google Analytics and Search Console accounts through Configuration > API Access to enrich Screaming Frog data with performance metrics. This integration automatically adds organic traffic and click data to your crawl results.
Common Pitfalls
Don't crawl during peak traffic hours — the additional server load can trigger rate limiting or temporary blocks. Run large site audits during off-peak hours to avoid impacting user experience. Avoid stopping crawls prematurely on large sites. Partial crawls miss deep pages where technical issues often concentrate, leading to incomplete audit findings that underestimate optimization opportunities.
Expected Results
You'll finish with comprehensive technical SEO data covering crawlability, indexability, and on-page optimization opportunities. Export files will contain 500-5000+ URLs with specific issues identified, error counts quantified, and redirect chains mapped for immediate technical remediation.
Quick Facts
About Screaming Frog
Industry-standard website crawler for technical SEO auditing
View tool review →Screaming Frog Alternatives
6 alternatives compared
View alternatives →