Crawlsmith

Waitrose Product Scraper

Extract grocery products from Waitrose.com — names, prices, unit prices, offers, reviews, categories, and images. Search, category browse, and offers modes with optional price-change monitoring.

What it does

This scraper automatically opens any Coles search or category URL and pulls every product it finds into structured, ready-to-use data. No manual browsing, no copy-paste, no login required.

Two modes of operation:

  1. Keyword Mode — Drop in a list of search terms (e.g. milk, tim tams, gluten free bread) and the scraper builds the search URLs for you.
  2. Direct URL Mode — Paste any Coles search, category, or filtered-results URL and the scraper walks every page until it hits your limit.

You can also combine both — provide some URLs and some keywords in one run.


Who uses it

Use Case How It Helps
Price Comparison Platforms Build a live AU grocery price feed alongside Woolworths, IGA, and Aldi — full SKU coverage with promo tracking
FMCG Brands & Suppliers Monitor where your products sit on Coles, what specials they run, and which brands are out-pricing you
Retail Analysts Track category-level pricing, basket-style promotions, and seasonal pricing rhythms across all categories
Private-Label Manufacturers Benchmark Coles "Coles Brand" pricing against name brands in real time
Cost-of-Living Reporters Measure shrinkflation, weekly specials, and trolley-cost shifts with hard data
AI / Data Teams Feed structured grocery data into pricing models, recommendation engines, and demand-forecasting pipelines

Input

14 options. Required ones are marked.

Option What it does
Search keywords Keywords to search Waitrose for.
Start URLs Waitrose search, category, offers, or product URLs to scrape directly.
Scrape current offers Also scrape the Waitrose Offers page (products currently on promotion).
Maximum products Maximum number of products to collect across the whole run (0 = unlimited).
Expand sub-categories When scraping a category, automatically descend into its sub-categories to cover the full catalogue (Waitrose shows only ~48 products per category page).
Max category depth How many levels deep to descend the category tree when expanding sub-categories.
Maximum concurrency Maximum number of pages fetched in parallel.
Incremental monitoring Track changes across runs.
Monitoring state key Optional name for the monitoring baseline.
Emit unchanged products In incremental mode, also output products whose price and details are unchanged.
Emit expired products In incremental mode, also output products that disappeared from results since the last run.
Skip reposts Drop products that are the same as a recently expired one relisted under a new id.
Notifications Optional webhook / Telegram / Discord / Slack notifications for new and changed products (used with incremental monitoring).
Proxy configuration Waitrose is geo-gated to the UK and bot-protected — a GB residential proxy is strongly recommended.

Output fields

14 fields, grouped the way the dataset views show them. Export to JSON, CSV, Excel or pull them through the API.

Overview

  • Product
  • Brand
  • Size
  • Price
  • On Offer
  • Rating
  • URL
  • Image

Pricing & Offers

  • Product
  • Brand
  • Size
  • Price (£)
  • Display Price
  • Unit Price
  • On Offer
  • Promotions

Monitoring

  • Product
  • Price
  • On Offer
  • Change
  • First Seen
  • Last Seen
  • URL

Example output

One record, shortened. Real runs return one row like this per result.

{
  "productId": "5051077",
  "name": "Coles Full Cream Milk 2L",
  "brand": "Coles",
  "size": "2L",
  "description": null,
  "price": {
    "current": 3.1,
    "was": 3.4,
    "savings": 0.3,
    "currency": "AUD"
  },
  "pricePerUnit": "$1.55 per 1L",
  "promotion": {
    "label": "Save $0.30",
    "type": "PRICE_DROP",
    "startDate": "2026-04-30",
    "endDate": "2026-05-06"
  },
  "availability": "In Stock",
  "category": {
    "breadcrumbs": [
      "Dairy, Eggs & Fridge",
      "Milk",
      "Full Cream"
    ],
    "category": "Dairy, Eggs & Fridge",
    "subCategory": "Milk",
    "aisle": "Aisle 12"
  },
  "url": "https://www.coles.com.au/product/coles-full-cream-milk-2l-5051077",
  "images": [
    "https://shop.coles.com.au/wcsstore/Coles-CAS/images/5/0/5/5051077-th.jpg"
  ],
  "featured": false,
  "adId": null,
  "adSource": null,
  "locationRestricted": false,
  "scrapedAt": "2026-05-02T19:48:00.000Z"
}

How to run it

  1. 1. Open the scraper on Apify and sign in (free plan works).
  2. 2. Fill in the input: search terms, URLs or filters, and a result limit.
  3. 3. Run it, then download JSON, CSV or Excel, or pull the dataset through the API.
  4. 4. Schedule it to get fresh data daily or weekly.

FAQ

Why are Australian residential proxies recommended? +

Coles aggressively blocks data-center IPs and non-AU traffic. Residential AU proxies look like real shoppers and dramatically reduce blocks. They're enabled by default.

Does it handle promotions and "1/2 Price" specials? +

Yes — every product carries a promotion object with the label, type, and start/end dates whenever a special is active.

Can I scrape an entire category? +

Yes — paste any Coles category URL (e.g. /browse/dairy-eggs-fridge/milk) and set max_items_per_url to 0 for unlimited. The scraper walks every page automatically.

Can I export the data? +

Yes — JSON, CSV, Excel, XML, HTML, RSS. You can also push data directly to Google Sheets, Zapier, Make, or any webhook/API endpoint.

How often should I run this? +

For fresh price data, run daily. For weekly specials tracking, run every Wednesday morning when Coles refreshes its catalogue. You can schedule automatic runs on Apify with just a few clicks.

Does it work with proxies? +

Yes — residential proxies are recommended for the best results. The scraper works with Apify's built-in proxy service and is preconfigured for Australia.

What about Woolworths, IGA, and Aldi? +

I can build a scraper for any of those — see the next section.


Need something this scraper does not do?

Extra fields, a different filter, a recurring feed into your database or a one-off bulk export.

Ask for a custom version

Related scrapers