Website Content Crawler

"SEO Audit & Content Extraction" by John Rippy | johnrippy.link

🏆 2025 Zapier Automation Hero of the Year — Project Phoenix: A 95-step AI sales pipeline cutting development time by 50%. Read more →

---

Crawl websites for SEO audits. Extracts HTML, title, meta tags, headings, links, and text content from each page.

Features

Automatic sitemap detection and parsing
Extracts metadata (title, description, OG tags)
Heading structure (H1, H2, H3)
Internal and external link analysis
Image extraction with alt text
Word count and load time metrics

Input

{
"startUrls": [{ "url": "https://example.com" }],
"maxCrawlPages": 25,
"maxCrawlDepth": 2,
"crawlSitemap": true
}

Output

Each page returns:

{
"url": "https://example.com/page",
"title": "Page Title",
"html": "<html>...</html>",
"text": "Page text content...",
"statusCode": 200,
"loadTimeMs": 1234,
"metadata": {
"description": "Meta description",
"ogTitle": "Open Graph Title",
"canonical": "https://example.com/page"
},
"headings": {
"h1": ["Main Heading"],
"h2": ["Subheading 1", "Subheading 2"],
"h3": []
},
"links": {
"internal": ["https://example.com/other"],
"external": ["https://external.com"]
},
"images": [
{ "src": "https://example.com/image.jpg", "alt": "Alt text" }
],
"wordCount": 1500
}

Use Cases

SEO audits
Content analysis
Site structure mapping
Broken link detection
Meta tag analysis

Pricing

This actor uses pay-per-event pricing:

Example costs:

Crawl 25 pages: 25 × $0.01 = $0.25
Crawl 100 pages: 100 × $0.01 = $1.00

Author

Built by John Rippy | johnrippy.link

🏆 2025 Zapier Automation Hero of the Year — Project Phoenix: A 95-step AI sales pipeline cutting development time by 50%. Read more →

---

Keywords

website crawler, web scraper, site crawler, seo crawler, content extraction, website data extraction