Crawlsmith

SpringerLink Scraper

Scrape SpringerLink journals, books and conference papers: abstracts, author keywords, full author lists with affiliations and ORCIDs, open-access status, PDF links and exact citation counts. Includes incremental monitoring that emits only new or updated papers.

What it does

This scraper automatically searches SpringerLink and collects structured, ready-to-use data on every publication it finds. No manual browsing, no copy-pasting from PDFs — set your filters and hit Start.

Three ways to tell it what you want:

  1. Discovery Mode — give it keywords (one or many), optionally narrowed by year, content type or open access, and it collects every matching publication
  2. Journal Mode — point it at a journal and collect everything that journal has published, newest first
  3. Direct URL Mode — hand it specific article, chapter, journal or search URLs (or just plain DOIs) and it fills in the full record for each one

And two levels of depth:

Mode What you get Coverage measured on 200 newly published items Speed & cost
Fast mode (default) Titles, authors, journals, dates, DOIs, keywords, topics, exact citation counts, open-access status — plus abstracts wherever an open index already has them 25% abstracts, 65% keywords, no affiliations 200 items in 13 seconds. No browser, no proxy.
Full mode Everything above, plus the abstract straight from Springer, author affiliations, corresponding authors and view counts 98% abstracts, 99% keywords, 94% affiliations 200 items in 90 seconds, about 8× the cost. Still needs no proxy.

Switch with the Full mode toggle. Fast mode is the default because most jobs do not need affiliations — but if you are collecting recent publications and want their abstracts, turn full mode on: it is the difference between a quarter of them and nearly all of them.

Neither mode needs a proxy. Springer does run a protection check on its own pages, and full mode gets through it with a real browser rather than by paying for residential traffic.

Who uses it

Use Case How It Helps
Researchers & PhD students Build a complete literature review dataset in minutes instead of weeks of manual searching
Bibliometrics & research analytics teams Track publication volumes, citation growth and topic trends across journals and years
Universities & research offices Monitor where your faculty publishes and which affiliations appear alongside yours
Pharma, biotech & R&D teams Watch a research area continuously and get alerted the moment relevant work appears
AI & data teams Feed clean abstracts and keywords into embeddings, RAG pipelines or literature-screening models
Science publishers & consultants Benchmark competing journals on output, open-access share and citation impact

Input

44 options. Required ones are marked.

Option What it does
Search query What to search for.
Journals Restrict the run to specific journals.
ISSN A single journal ISSN to scope the search to.
Content type Limit results to one kind of publication.
Published from (year) Only include publications from this year onwards.
Published to (year) Only include publications up to and including this year.
Published within Keep only recently published work, counted back from today.
Sort by Result order.
Open access only Keep only freely readable publications.
Minimum citations Drop publications cited fewer times than this.
Must contain Keep a publication only if its title, abstract or keywords contain at least one of these words.
Must not contain Drop publications whose title, abstract or keywords contain any of these words.
Direct URLs Scrape specific SpringerLink URLs — article pages, chapter pages, journal pages or search result pages.
DOIs Scrape publications by DOI (e.g. 10.1007/s10994-025-06923-w).

Plus 30 more options. Full input reference on Apify.

Output fields

16 fields, grouped the way the dataset views show them. Export to JSON, CSV, Excel or pull them through the API.

Overview

  • Title
  • Journal / book
  • Published
  • Authors
  • Citations
  • Open access
  • DOI
  • Link

Abstracts & keywords

  • Title
  • Abstract
  • Keywords
  • Subject areas
  • Journal / book
  • Published
  • Link

Authors & affiliations

  • Title
  • Authors (with affiliations & ORCID)
  • Corresponding
  • Count
  • Journal / book
  • DOI

Monitoring changes

  • Change
  • Title
  • Journal / book
  • Published
  • First seen
  • Last seen
  • Link

Example output

One record, shortened. Real runs return one row like this per result.

{
  "doi": "10.1007/s10994-025-06923-w",
  "title": "Cluster-based multidimensional scaling for large datasets",
  "url": "https://link.springer.com/article/10.1007/s10994-025-06923-w",
  "source": "springer",
  "discoverySource": "crossref",
  "contentType": "article",
  "articleType": "OriginalPaper",
  "journalName": "Machine Learning",
  "journalAbbrev": "Mach Learn",
  "issn": [
    "1573-0565",
    "0885-6125"
  ],
  "isbn": [],
  "volume": "114",
  "issue": "12",
  "pages": "289",
  "articleNumber": "289",
  "publishedDate": "2025-11-25",
  "onlineDate": "2025-11-25",
  "issueDate": "2025-12-01",
  "year": 2025,
  "authors": [
    {
      "name": "Joonas Hämäläinen",
      "affiliation": "Faculty of Information Technology, University of Jyvaskyla, Jyvaskyla, Finland",
      "affiliations": [
        "Faculty of Information Technology, University of Jyvaskyla, Jyvaskyla, Finland"
      ],
      "orcid": "0000-0002-8466-9232",
      "email": "joonas.k.hamalainen@jyu.fi",
      "isCorresponding": true
    },
    {
      "name": "Tommi Kärkkäinen",
      "affiliation": "Faculty of Information Technology, University of Jyvaskyla, Jyvaskyla, Finland",
      "affiliations": [
        "Faculty of Information Technology, University of Jyvaskyla, Jyvaskyla, Finland"
      ],
      "orcid": null,
      "email": null,
      "isCorresponding": false
    }
  ],
  "authorCount": 2,
  "correspondingAuthors": [
    "Joonas Hämäläinen"
  ],
  "abstract": "Multidimensional scaling is a popular dimensionality reduction technique that embeds high-dimensional data into a low-dimensional space while preserving pair...",
  "abstractLength": 1114,
  "keywords": [
    "Multidimensional scaling",
    "Dimensionality reduction",
    "Clustering"
  ],
  "subjects": [
    "Machine Learning",
    "Artificial Intelligence",
    "Simulation and Modeling"
  ],
  "language": "en",
  "isOpenAccess": true,
  "license": "http://creativecommons.org/licenses/by/4.0/",
  "copyright": "2025 The Author(s)",
  "pdfUrl": "https://link.springer.com/content/pdf/10.1007/s10994-025-06923-w.pdf",
  "citationCount": 7,
  "citationSource": "crossref",
  "referenceCount": 118,
  "references": [],
  "accessesCount": 2811,
  "altmetricScore": 1,
  "conferenceTitle": null,
  "detailFetched": true,
  "contentHash": "9f2c1ab84e0d5b7c3f6a10e8d4b2c9f7a1e5d3b8",
  "scrapedAt": "2026-07-28T10:48:41.398Z"
}

How to run it

  1. 1. Open the scraper on Apify and sign in (free plan works).
  2. 2. Fill in the input: search terms, URLs or filters, and a result limit.
  3. 3. Run it, then download JSON, CSV or Excel, or pull the dataset through the API.
  4. 4. Schedule it to get fresh data daily or weekly.

FAQ

Do I get abstracts for paywalled publications? +

Yes, in full mode. Springer shows the abstract, keywords and author details openly even when the full text sits behind a paywall, so those fields come through for gated items too. Only the full text and PDF content stay inaccessible — this scraper never touches them.

Which mode do I need? +

Start with the default fast mode: it is quick, cheap and gives you the complete bibliographic record with keywords and citation counts. Switch on full mode when you specifically need abstracts for very recent papers, author affiliations, corresponding authors or view counts — those live only on Springer's own pages. If you want the abstracts but not the affiliations, there is a middle option: paste a free Springer Nature API key and abstracts arrive as data instead of page reads.

Is there a limit on how many results a search can return? +

No. The website's own search stops at 1,000 results per query, but this scraper uses an unlimited index by default, so a search with 50,000 matches will hand you all 50,000.

How is this different from a free DOI lookup? +

Free bibliographic sources give you titles, authors and dates, and they lag badly on anything published in the last few months. This scraper adds the keywords, topics, exact citation counts and open-access status in fast mode, and in full mode the abstract, author affiliations, corresponding author and view counts that no free lookup carries.

Can I watch a research area continuously? +

Yes — turn on incremental monitoring and schedule the run. Every later run returns only publications that are new or have changed, and can push them to Slack, Telegram, Discord or your webhook.

Can I export the data? +

Yes — JSON, CSV, Excel, XML, HTML, RSS. You can also push data directly to Google Sheets, Zapier, Make, or any webhook/API endpoint.

How often should I run this? +

For fresh data, run daily or weekly. You can schedule automatic runs on Apify with just a few clicks.

Does it work with proxies? +

It works without one. Neither mode needs a proxy: the metadata channels are never blocked, and full mode gets through Springer's protection check with a real browser rather than by buying residential traffic — measured at about one second to clear, with no refusals across several hundred publication pages. A proxy can still be configured, and if you do, it is held back and used only when a page is genuinely refused, so you are never billed for residential traffic speculatively. Springer's check is address-scoped, so switching to Residential is the right move if a run ever does report refused pages.

Can I get abstracts without opening Springer's pages? +

Yes, with a free Springer Nature API key (dev.springernature.com — sign up, create an application, copy the key). Paste it into Springer Nature API key and abstracts come back as JSON. The free tier is 500 requests a day and this scraper batches 25 publications per request, so one key covers roughly 12,500 abstracts a day. Affiliations, corresponding authors and view counts are not in that API — they exist only on the publication page, so they still need full mode.

Need something this scraper does not do?

Extra fields, a different filter, a recurring feed into your database or a one-off bulk export.

Ask for a custom version

Related scrapers