SimilarWeb Scraper
Scrape SimilarWeb data for any domain: global/country/category rank, monthly visits, traffic sources, top countries, top keywords, engagement (bounce rate, time on site), competitors, and AI traffic details.
What it does
This scraper automatically pulls SimilarWeb's domain analytics for any website you give it and returns structured, ready-to-use data. No manual browsing through dashboards — just paste your list of domains and hit Start.
You get global, country, and category rankings, three months of monthly visit history, the full traffic source breakdown (direct / search / social / referrals / mail / paid), top countries, top SEO keywords, competitors, and the new AI Traffic split (which chatbots are sending people to the site).
Who uses it
| Use Case | How It Helps |
|---|---|
| SEO Agencies | Audit thousands of client and competitor domains in minutes — rank, keywords, traffic sources |
| Sales & Lead Gen Teams | Score and prioritize prospects by real traffic and rank instead of guessing |
| Market Researchers | Track market share across an industry with bulk traffic and engagement data |
| E-commerce Owners | See exactly how much traffic competitors get and where it comes from |
| Investors & VCs | Validate startup growth claims with independent traffic and engagement metrics |
| Partnership & BD Teams | Find high-traffic partners and qualify opportunities with hard numbers |
Input
6 options. Required ones are marked.
| Option | Type | What it does |
|---|---|---|
| Domains | List | List of domains to look up on SimilarWeb (e.g. apify.com, stackoverflow.com). |
| Start URLs (optional) | List | Alternative to Domains - paste similarweb.com/website// URLs or just the website URLs. |
| Max Concurrency | Number | How many domains to fetch in parallel. |
| Max Retries Per Domain | Number | How many times to retry (with proxy rotation) when SimilarWeb blocks or rate-limits a request. |
| Request Timeout (seconds) | Number | Per-request timeout. |
| Proxy Configuration | Object | Proxy settings. |
Output fields
32 fields, grouped the way the dataset views show them. Export to JSON, CSV, Excel or pull them through the API.
Overview
- Domain
- Category
- Global Rank
- Country Rank
- Country
- Monthly Visits
- Bounce Rate %
- Pages/Visit
- Avg Visit (s)
- Snapshot Date
Traffic Sources
- Domain
- Direct %
- Search %
- Social %
- Mail %
- Referrals %
- Paid Referrals %
- Monthly Visits
Rankings
- Domain
- Global Rank
- Country Rank
- Country
- Category Rank
- Category
Full Details
- Domain
- Site Name
- Title
- Description
- Category
- Snapshot Date
- Global Rank
- Country Rank
- Country
- Category Rank
- Category Path
- Monthly Visits
- monthlyVisitsHistory
- Stats Month
- Bounce Rate %
- Pages/Visit
- Avg Visit (s)
- Direct %
- Search %
- Social %
- Mail %
- Referrals %
- Paid Referrals %
- topCountries
- topKeywords
- competitors
- AI Total Visits
- AI Referral %
- aiTrafficSplit
- Screenshot
- SimilarWeb URL
- Scraped At
Example output
One record, shortened. Real runs return one row like this per result.
{
"domain": "stackoverflow.com",
"siteName": "stackoverflow.com",
"title": "Newest 'postgresql' Questions",
"description": "Stack Overflow | The World's Largest Online Community for Developers",
"category": "computers_electronics_and_technology/programming_and_developer_software",
"largeScreenshot": "https://site-images.similarcdn.com/image?url=stackoverflow.com&t=1&s=1&h=04179ebf",
"snapshotDate": "2026-04-01T00:00:00+00:00",
"globalRank": 2240,
"countryRank": 2170,
"countryRankCountryCode": "US",
"categoryRank": 63,
"categoryRankCategory": "Computers_Electronics_and_Technology/Programming_and_Developer_Software",
"globalCategoryRank": null,
"monthlyVisits": 31837008,
"engagementMonth": "2026-04",
"bounceRate": 70.26,
"pagesPerVisit": 1.69,
"avgVisitDurationSeconds": 141.72,
"monthlyVisitsHistory": [
{
"month": "2026-02",
"visits": 42405407
},
{
"month": "2026-03",
"visits": 38044480
},
{
"month": "2026-04",
"visits": 31837008
}
],
"trafficSourceDirect": 7.01,
"trafficSourceSearch": 85.6,
"trafficSourceSocial": 1.52,
"trafficSourceMail": 0.24,
"trafficSourceReferrals": 5.21,
"trafficSourcePaidReferrals": 0.03,
"topCountries": [
{
"countryCode": "US",
"countryNumeric": 840,
"share": 22.21
},
{
"countryCode": "IN",
"countryNumeric": 356,
"share": 8.07
},
{
"countryCode": "DE",
"countryNumeric": 276,
"share": 7.29
}
],
"topKeywords": [
{
"keyword": "stack overflow",
"estimatedValue": 149400,
"volume": 149150,
"cpc": 1.42
},
{
"keyword": "stackoverflow",
"estimatedValue": 43680,
"volume": 48040,
"cpc": 1.9
}
],
"competitors": [],
"aiTotalVisits": 184250.5,
"aiReferralTraffic": 0.0058,
"aiTrafficSplit": [
{
"chatbot": "chatgpt.com",
"rank": 1
},
{
"chatbot": "perplexity.ai",
"rank": 2
},
{
"chatbot": "gemini.google.com",
"rank": 3
}
],
"isSmall": false,
"isDataFromGa": false,
"url": "https://www.similarweb.com/website/stackoverflow.com/",
"scrapedAt": "2026-05-06T20:35:00.000Z"
} How to run it
- 1. Open the scraper on Apify and sign in (free plan works).
- 2. Fill in the input: search terms, URLs or filters, and a result limit.
- 3. Run it, then download JSON, CSV or Excel, or pull the dataset through the API.
- 4. Schedule it to get fresh data daily or weekly.
FAQ
Do I need a SimilarWeb account? +
No. You don't need any login, paid plan, or API key — just paste your list of domains and run.
What domains work? +
Anything SimilarWeb has data for. Popular sites return rich data; very small or new domains may have partial data or none at all.
How fresh is the data? +
SimilarWeb refreshes its dataset monthly — every record includes a snapshotDate and engagementMonth so you always know exactly which month the numbers represent.
What about subdomains and paths? +
SimilarWeb works at the registered domain level. Subdomains and paths in your input are normalized to the root domain automatically.
Can I scrape thousands of domains? +
Yes. The scraper processes domains in parallel (2 at a time by default, up to 50) with smart retries, so even very large lists complete in minutes — not hours.
Can I export the data? +
Yes — JSON, CSV, Excel, XML, HTML, RSS. You can also push data directly to Google Sheets, Zapier, Make, or any webhook/API endpoint.
How often should I run this? +
For fresh data, run monthly (right after SimilarWeb's data refresh). You can schedule automatic runs on Apify with just a few clicks.
Does it work with proxies? +
Yes — the scraper uses Apify residential proxies out of the box. SimilarWeb rate-limits datacenter IPs aggressively, so residential is required for reliable results.
Need something this scraper does not do?
Extra fields, a different filter, a recurring feed into your database or a one-off bulk export.
Ask for a custom versionRelated scrapers
SpringerLink Scraper
Global
Scrape SpringerLink journals, books and conference papers: abstracts, author keywords, full author lists with affiliations and ORCIDs, open-access status, PDF links and exact citation counts. Includes incremental monitoring that emits only new or updated papers.
Wiley Analytical Science Scraper
Global
Scrape Wiley's Analytical Science portal: 26 journals covering mass spectrometry, separation science, spectroscopy, proteomics and bioanalysis. Get full abstracts, complete author lists with ORCIDs, keywords, DOIs, volume, issue, dates and exact citation counts. Plus incremental monitoring.
Taylor & Francis Online Scraper
Global
Scrape Taylor & Francis Online: search 4,250 T&F and Routledge journals for article titles, DOIs, full author lists, abstracts, keywords, journal and issue details, view and citation counts, access status and PDF links. Abstracts included for paywalled papers. Incremental monitoring built in.
Document to Markdown
Global
Convert PDF, Word, PowerPoint, Excel, CSV, HTML, EPUB and images from URLs to clean Markdown or text: headings, lists and tables kept, OCR for scans, RAG-ready chunks.