Google News Scraper
FreeNot checkedScrape Google News articles by keyword. Real-time brand monitoring, PR tracking, competitive intelligence. RSS + browser fallback. Node.js + Puppeteer.
About
Scrape Google News articles by keyword. Real-time brand monitoring, PR tracking, competitive intelligence. RSS + browser fallback. Node.js + Puppeteer.
README
Scrape Google News articles by keyword in real time. Monitor brand mentions, track PR coverage, and gather competitive intelligence from 50,000+ news sources worldwide. Built with Node.js, Puppeteer, and the Apify platform.
Run on Apify Available on RapidAPI License: ISC
What It Does
This Google News scraper extracts articles from Google News using a triple-fallback architecture:
- RSS Feed (fastest, most reliable) — Parses Google News RSS/XML feeds
- Google News Website (browser-based) — Scrapes the news.google.com interface directly
- Google Search News Tab (deepest) — Falls back to Google's main search news results
If one approach fails (CAPTCHA, rate limit), it automatically tries the next. You get results even when Google blocks standard methods.
What Data You Get
{
"keyword": "Tesla",
"title": "Tesla Announces New Gigafactory in Southeast Asia",
"source": "Reuters",
"publishedAt": "2026-03-10T14:30:00.000Z",
"snippet": "Tesla Inc said on Monday it would build a new manufacturing facility...",
"articleUrl": "https://www.reuters.com/business/autos/tesla-new-gigafactory-2026-03-10/",
"imageUrl": "https://...",
"fullText": "Full article text extracted from source (optional)...",
"scrapedAt": "2026-03-10T15:00:00.000Z"
}
Quick Start
Using the Apify API (Recommended)
The fastest way to get started — no setup required:
cURL
curl "https://api.apify.com/v2/acts/george.the.developer~google-news-monitor/run-sync-get-dataset-items?token=YOUR_API_TOKEN" \
-X POST \
-d '{
"keywords": ["Tesla", "OpenAI"],
"timeRange": "past_day",
"maxArticlesPerKeyword": 20
}' \
-H 'Content-Type: application/json'
Node.js
import { ApifyClient } from 'apify-client';
const client = new ApifyClient({ token: 'YOUR_API_TOKEN' });
const run = await client.actor('george.the.developer/google-news-monitor').call({
keywords: ['Tesla', 'OpenAI', 'artificial intelligence'],
timeRange: 'past_day',
maxArticlesPerKeyword: 20,
extractFullText: false,
language: 'en',
country: 'US',
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(`Found ${items.length} articles`);
items.forEach(article => {
console.log(`[${article.source}] ${article.title}`);
});
Python
from apify_client import ApifyClient
client = ApifyClient("YOUR_API_TOKEN")
run = client.actor("george.the.developer/google-news-monitor").call(run_input={
"keywords": ["Tesla", "OpenAI"],
"timeRange": "past_day",
"maxArticlesPerKeyword": 20,
"extractFullText": False,
"language": "en",
"country": "US",
})
for article in client.dataset(run["defaultDatasetId"]).iterate_items():
print(f"[{article['source']}] {article['title']}")
print(f" URL: {article['articleUrl']}")
Use Cases
- Brand Monitoring — Track mentions of your company, product, or competitors across 50,000+ news sources
- PR & Media Intelligence — Measure press coverage, identify trending stories about your brand
- Competitive Intelligence — Monitor competitor announcements, partnerships, and product launches
- Market Research — Track industry trends, regulatory changes, and market signals
- Content Curation — Aggregate news for newsletters, dashboards, or AI-powered analysis
- AI Training Data — Build datasets of news articles for NLP, sentiment analysis, or LLM fine-tuning
Input Parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
keywords |
string[] | required | Keywords or brand names to search |
timeRange |
string | past_day |
anytime, past_hour, past_day, past_week, past_month |
maxArticlesPerKeyword |
number | 20 | Max articles per keyword (1-100) |
extractFullText |
boolean | false | Visit each article to extract full text |
language |
string | en |
Language code (en, es, fr, de, ja, etc.) |
country |
string | US |
Country code (US, GB, DE, FR, JP, etc.) |
proxyConfiguration |
object | — | Proxy settings (Apify Proxy recommended) |
How It Works (Architecture)
Input Keywords
│
▼
┌─────────────────────┐
│ Approach 1: RSS │──── Success? ──► Parse XML ──► Results
│ (news.google.com) │
└─────────────────────┘
│ Failed/CAPTCHA
▼
┌─────────────────────┐
│ Approach 2: Web │──── Success? ──► Parse DOM ──► Results
│ (news.google.com) │
└─────────────────────┘
│ Failed/CAPTCHA
▼
┌─────────────────────┐
│ Approach 3: Search │──── Success? ──► Parse DOM ──► Results
│ (google.com/news) │
└─────────────────────┘
│
▼
Deduplicate + Optional Full-Text Extraction
│
▼
Output to Dataset
Run on Apify
Run this actor on Apify — no setup, no infrastructure, pay only for what you use.
- Cost: ~$0.003 per article found
- Speed: 20-100 articles per minute depending on settings
- Reliability: Triple-fallback ensures results even when Google blocks standard methods
Also Available on RapidAPI
Prefer a standard REST API? This scraper is also available on RapidAPI with simple API key authentication:
- Free tier: 50 requests/month
- Pro: $19/month (1,000 requests)
- Ultra: $49/month (5,000 requests)
- Mega: $129/month (20,000 requests)
Limitations
- This tool does not bypass paywalled article content. Full-text extraction works on freely accessible articles only.
- Google may serve CAPTCHAs for high-volume requests. Using Apify Proxy significantly reduces this.
- Rate limiting applies — respect Google's terms of service.
Related Tools
- LinkedIn Employee Scraper — Extract employee data from any company
- YouTube Transcript Extractor — Get video transcripts for AI/RAG
- Website Contact Scraper — Find emails & contacts from any website
- US Tariff Lookup — Look up import duty rates & HS codes
License
ISC License. See LICENSE for details.
Built by george.the.developer on Apify.
from github.com/the-ai-entrepreneur-ai-hub/google-news-scraper
Installing Google News Scraper
This server has no published package — it is built from source. Open the repository and follow its README.
▸ github.com/the-ai-entrepreneur-ai-hub/google-news-scraperFAQ
Is Google News Scraper MCP free?
Yes, Google News Scraper MCP is free — one-click install via Unyly at no cost.
Does Google News Scraper need an API key?
No, Google News Scraper runs without API keys or environment variables.
Is Google News Scraper hosted or self-hosted?
Self-hosted: the server runs locally on your machine via the install command above.
How do I install Google News Scraper in Claude Desktop, Claude Code or Cursor?
Open Google News Scraper on unyly.org, pick your client tab (Claude Desktop, Claude Code, Cursor) and press Install — the config is generated automatically, no JSON editing.
Related MCPs
Playwright
Browser automation, scraping, screenshots
by MicrosoftPuppeteer
Browser automation and web scraping.
by modelcontextprotocolGarmin Connect
An MCP server for Garmin Connect that provides access to fitness activities, health statistics, and sleep data by routing requests through a headless browser to
by etweisbergHiggsfield Unlimited
MCP server for Higgsfield AI that enables unlimited-mode image, video, audio generation, uploads, and job management via multiple parallel accounts, using brows
by nukIeeropentabs-dev/opentabs
Plugin-based server and Chrome extension that gives AI agents access to web apps through your authenticated browser session, with a plugin SDK.
by opentabs-devhlydecker/ucsc-genome-mcp
Interact with the UCSC Genome Browser API to find genomes, chromosomes and more.
by hlydecker34892002/bilibili-mcp-js
Search Bilibili content, with LangChain integration examples.
by 34892002achiya-automation/safari-mcp
Native Safari browser automation without a Chrome dependency.
by achiya-automationagent-infra/mcp-server-browser
Browser automation using Puppeteer, supporting local and remote browser connections.
by bytedanceaparajithn/agent-scraper-mcp
Web scraping: clean content extraction, CSS-selector scraping, Playwright screenshots, link and metadata extraction, and Google search.
by aparajithnCompare Google News Scraper with
Not sure what to pick?
Find your stack in 60 seconds
Author?
Embed badge for your README
Browse similar
All browse MCPs
