Taylor & Francis Online Scraper
Scrape Taylor & Francis Online: search 4,250 T&F and Routledge journals for article titles, DOIs, full author lists, abstracts, keywords, journal and issue details, view and citation counts, access status and PDF links. Abstracts included for paywalled papers. Incremental monitoring built in.
What it does
This scraper automatically works through Taylor & Francis Online search results and collects structured, ready-to-use data for every article it finds. No manual browsing, no copy-pasting from result pages — set your filters and hit Start.
Two ways to point it at what you want:
- Discovery Mode — describe what you are looking for (keyword, author, journal, years, article type) and the scraper walks the whole result set for you, however large it is.
- Direct URL Mode — paste specific addresses: search result pages, journal issue pages, individual article pages, or plain DOIs.
Who uses it
| Use Case | How It Helps |
|---|---|
| Researchers & PhD students | Build a complete literature review in minutes instead of clicking through hundreds of result pages |
| Bibliometrics & research analytics teams | Get citation counts, view counts and Altmetric scores across thousands of papers in one consistent table |
| University libraries | Track exactly what a department publishes, in which journals, and how often it gets cited |
| Pharma & biotech intelligence | Monitor new publications on a compound, method or indication and get alerted the day they appear |
| AI & data teams | Feed clean abstracts and keywords into a model, a search index or a RAG pipeline |
| Science journalists & editors | Spot high-Altmetric, high-attention papers the moment they are published |
Input
46 options. Required ones are marked.
| Option | Type | What it does |
|---|---|---|
| Search query | Text | What to search for across the whole record — title, abstract, keywords and full text (e.g. "mass spectrometry"). |
| Words in the title | Text | Only match articles with these words in the title. |
| Words in the abstract | Text | Only match articles with these words in the abstract. |
| Author | Text | Author name to search for (e.g. "Smith"). |
| Journals | List | Limit the run to specific journals. |
| Article type | Text | Limit results to one kind of article. |
| Published from (year) | Number | Only include articles published in this year or later. |
| Published to (year) | Number | Only include articles published up to and including this year. |
| Sort by | Text | Result order. |
| Open access only | Yes / no | Keep only freely readable articles. |
| Minimum citations | Number | Drop articles cited fewer times than this. |
| Must contain | List | Keep an article only if its title, abstract or keywords contain at least one of these words. |
| Must not contain | List | Drop articles whose title, abstract or keywords contain any of these words. |
| Direct URLs | List | Scrape specific Taylor & Francis addresses — search result pages, journal issue pages, article pages, or plain DOIs (10.1080/…). |
Plus 32 more options. Full input reference on Apify.
Output fields
17 fields, grouped the way the dataset views show them. Export to JSON, CSV, Excel or pull them through the API.
Overview
- Title
- Journal
- Published
- Authors
- Citations
- Views
- Access
- DOI
- Link
Abstracts & keywords
- Title
- Abstract
- Keywords
- Journal
- Published
- DOI
- Link
Authors
- Title
- Authors
- Count
- Journal
- Published
- DOI
Citations & metrics
- Title
- Citations
- Views
- Altmetric
- Journal
- Year
- Access
- DOI
Monitoring changes
- Change
- Title
- Journal
- Published
- First seen
- Last seen
- Link
Example output
One record, shortened. Real runs return one row like this per result.
{
"doi": "10.1080/00032710701484459",
"title": "Electrospray Ionization Mass Spectrometry of Hexanitrohexaazaisowurtzitane (CL-20)",
"url": "https://www.tandfonline.com/doi/full/10.1080/00032710701484459",
"source": "tandf",
"discoverySource": "site-search",
"contentType": "article",
"articleType": "research-article",
"journalName": "Analytical Letters",
"journalCode": "lanl20",
"issn": [
"0003-2719",
"1532-236X"
],
"volume": "40",
"issue": "10",
"pages": "1972-1978",
"publishedDate": "2007-08-18",
"onlineDate": "2007-08-18",
"receivedDate": "2007-04-10",
"acceptedDate": "2007-05-28",
"year": 2007,
"authors": [
{
"name": "J. A. Campbell",
"given": "J. A.",
"family": "Campbell",
"affiliation": null,
"affiliations": [],
"orcid": null,
"email": null,
"isCorresponding": false
},
{
"name": "J. E. Szecsody",
"given": "J. E.",
"family": "Szecsody",
"affiliation": null,
"affiliations": [],
"orcid": null,
"email": null,
"isCorresponding": false
}
],
"authorCount": 4,
"abstract": "Hexanitrohexaazaisowurzitane (CL-20) is a newly developed propellant and has been examined using negative ion electrospray mass spectrometry. The primary ion...",
"abstractLength": 211,
"keywords": [
"Negative ion electrospray mass spectrometry",
"CL-20",
"adduct ions"
],
"subjects": [],
"language": "en",
"isOpenAccess": false,
"accessType": "restricted",
"license": null,
"pdfUrl": null,
"citationCount": 4,
"citationSource": "crossref",
"viewsCount": 251,
"altmetricScore": 0,
"scrapedAt": "2026-07-28T12:00:00.000Z",
"contentHash": "a3f9c1e0b5d47e2a8c6f1b9d0e3a7c5f2b8d4e6a"
} How to run it
- 1. Open the scraper on Apify and sign in (free plan works).
- 2. Fill in the input: search terms, URLs or filters, and a result limit.
- 3. Run it, then download JSON, CSV or Excel, or pull the dataset through the API.
- 4. Schedule it to get fresh data daily or weekly.
FAQ
Do I get abstracts for papers I don't have a subscription to? +
Yes. Taylor & Francis publishes the abstract, keywords and all the dates for every paper, whether the full text is paywalled or not — and that is exactly what this scraper collects. Only the full text itself stays behind the paywall, and it is never touched.
Is there a limit on how many results a search can return? +
No. Unlike most academic sites there is no result ceiling here — a search matching 195,000 papers can be collected right down to the last record. Use "Max Results" to stop wherever you like.
How do I find a journal's code? +
It is the short code in the journal's own address: for tandfonline.com/journals/lanl20 the code is lanl20. If you mistype one, the run tells you immediately instead of quietly returning unrelated papers.
Which mode do I need? +
Start with the default: you get the complete bibliographic record plus full abstracts, keywords and topics, and it stays cheap because abstracts come through a small dedicated request rather than a whole page download. Switch on full mode only if you need the page range, the received/accepted dates or the article's own view and Altmetric counters — those exist nowhere else.
Are the abstracts in the cheap mode shortened? +
No. They are the complete abstracts, identical to what the article page shows, including multi-section abstracts with their Background/Methods/Results headings.
Will keywords be there for a paper published yesterday? +
Usually not. Keywords and topics come from an open scholarly index that covers virtually everything older than about a week, but the newest few days can be missing. If you are monitoring brand-new papers and need their keywords, turn on full mode, which reads them from the article page directly.
Can I get an alert when a new paper on my topic appears? +
Yes — turn on incremental monitoring, schedule the run daily, and add a Slack, Discord, Telegram or webhook address. You will only hear about papers that are genuinely new.
Can I export the data? +
Yes — JSON, CSV, Excel, XML, HTML, RSS. You can also push data directly to Google Sheets, Zapier, Make, or any webhook/API endpoint.
How often should I run this? +
For fresh data, run daily or weekly. You can schedule automatic runs on Apify with just a few clicks.
Does it work with proxies? +
Yes — residential proxies are recommended for the best results, and they are switched on by default. The scraper works with Apify's built-in proxy service.
Need something this scraper does not do?
Extra fields, a different filter, a recurring feed into your database or a one-off bulk export.
Ask for a custom versionRelated scrapers
SimilarWeb Scraper
Global
Scrape SimilarWeb data for any domain: global/country/category rank, monthly visits, traffic sources, top countries, top keywords, engagement (bounce rate, time on site), competitors, and AI traffic details.
SpringerLink Scraper
Global
Scrape SpringerLink journals, books and conference papers: abstracts, author keywords, full author lists with affiliations and ORCIDs, open-access status, PDF links and exact citation counts. Includes incremental monitoring that emits only new or updated papers.
Wiley Analytical Science Scraper
Global
Scrape Wiley's Analytical Science portal: 26 journals covering mass spectrometry, separation science, spectroscopy, proteomics and bioanalysis. Get full abstracts, complete author lists with ORCIDs, keywords, DOIs, volume, issue, dates and exact citation counts. Plus incremental monitoring.
Document to Markdown
Global
Convert PDF, Word, PowerPoint, Excel, CSV, HTML, EPUB and images from URLs to clean Markdown or text: headings, lists and tables kept, OCR for scans, RAG-ready chunks.