Command Palette

Search for a command to run...

UnylyUnyly
Browse all

Google News Scraper

FreeNot checked

Scrape Google News articles by keyword. Real-time brand monitoring, PR tracking, competitive intelligence. RSS + browser fallback. Node.js + Puppeteer.

GitHubEmbed

About

Scrape Google News articles by keyword. Real-time brand monitoring, PR tracking, competitive intelligence. RSS + browser fallback. Node.js + Puppeteer.

README

Scrape Google News articles by keyword in real time. Monitor brand mentions, track PR coverage, and gather competitive intelligence from 50,000+ news sources worldwide. Built with Node.js, Puppeteer, and the Apify platform.

Run on Apify Available on RapidAPI License: ISC

What It Does

This Google News scraper extracts articles from Google News using a triple-fallback architecture:

  1. RSS Feed (fastest, most reliable) — Parses Google News RSS/XML feeds
  2. Google News Website (browser-based) — Scrapes the news.google.com interface directly
  3. Google Search News Tab (deepest) — Falls back to Google's main search news results

If one approach fails (CAPTCHA, rate limit), it automatically tries the next. You get results even when Google blocks standard methods.

What Data You Get

{
  "keyword": "Tesla",
  "title": "Tesla Announces New Gigafactory in Southeast Asia",
  "source": "Reuters",
  "publishedAt": "2026-03-10T14:30:00.000Z",
  "snippet": "Tesla Inc said on Monday it would build a new manufacturing facility...",
  "articleUrl": "https://www.reuters.com/business/autos/tesla-new-gigafactory-2026-03-10/",
  "imageUrl": "https://...",
  "fullText": "Full article text extracted from source (optional)...",
  "scrapedAt": "2026-03-10T15:00:00.000Z"
}

Quick Start

Using the Apify API (Recommended)

The fastest way to get started — no setup required:

cURL

curl "https://api.apify.com/v2/acts/george.the.developer~google-news-monitor/run-sync-get-dataset-items?token=YOUR_API_TOKEN" \
  -X POST \
  -d '{
    "keywords": ["Tesla", "OpenAI"],
    "timeRange": "past_day",
    "maxArticlesPerKeyword": 20
  }' \
  -H 'Content-Type: application/json'

Node.js

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_API_TOKEN' });

const run = await client.actor('george.the.developer/google-news-monitor').call({
    keywords: ['Tesla', 'OpenAI', 'artificial intelligence'],
    timeRange: 'past_day',
    maxArticlesPerKeyword: 20,
    extractFullText: false,
    language: 'en',
    country: 'US',
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(`Found ${items.length} articles`);
items.forEach(article => {
    console.log(`[${article.source}] ${article.title}`);
});

Python

from apify_client import ApifyClient

client = ApifyClient("YOUR_API_TOKEN")

run = client.actor("george.the.developer/google-news-monitor").call(run_input={
    "keywords": ["Tesla", "OpenAI"],
    "timeRange": "past_day",
    "maxArticlesPerKeyword": 20,
    "extractFullText": False,
    "language": "en",
    "country": "US",
})

for article in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(f"[{article['source']}] {article['title']}")
    print(f"  URL: {article['articleUrl']}")

Use Cases

  • Brand Monitoring — Track mentions of your company, product, or competitors across 50,000+ news sources
  • PR & Media Intelligence — Measure press coverage, identify trending stories about your brand
  • Competitive Intelligence — Monitor competitor announcements, partnerships, and product launches
  • Market Research — Track industry trends, regulatory changes, and market signals
  • Content Curation — Aggregate news for newsletters, dashboards, or AI-powered analysis
  • AI Training Data — Build datasets of news articles for NLP, sentiment analysis, or LLM fine-tuning

Input Parameters

Parameter Type Default Description
keywords string[] required Keywords or brand names to search
timeRange string past_day anytime, past_hour, past_day, past_week, past_month
maxArticlesPerKeyword number 20 Max articles per keyword (1-100)
extractFullText boolean false Visit each article to extract full text
language string en Language code (en, es, fr, de, ja, etc.)
country string US Country code (US, GB, DE, FR, JP, etc.)
proxyConfiguration object — Proxy settings (Apify Proxy recommended)

How It Works (Architecture)

Input Keywords
     │
     ▼
┌─────────────────────┐
│  Approach 1: RSS    │──── Success? ──► Parse XML ──► Results
│  (news.google.com)  │
└─────────────────────┘
     │ Failed/CAPTCHA
     ▼
┌─────────────────────┐
│  Approach 2: Web    │──── Success? ──► Parse DOM ──► Results
│  (news.google.com)  │
└─────────────────────┘
     │ Failed/CAPTCHA
     ▼
┌─────────────────────┐
│  Approach 3: Search │──── Success? ──► Parse DOM ──► Results
│  (google.com/news)  │
└─────────────────────┘
     │
     ▼
  Deduplicate + Optional Full-Text Extraction
     │
     ▼
  Output to Dataset

Run on Apify

Run this actor on Apify — no setup, no infrastructure, pay only for what you use.

  • Cost: ~$0.003 per article found
  • Speed: 20-100 articles per minute depending on settings
  • Reliability: Triple-fallback ensures results even when Google blocks standard methods

Also Available on RapidAPI

Prefer a standard REST API? This scraper is also available on RapidAPI with simple API key authentication:

  • Free tier: 50 requests/month
  • Pro: $19/month (1,000 requests)
  • Ultra: $49/month (5,000 requests)
  • Mega: $129/month (20,000 requests)

Limitations

  • This tool does not bypass paywalled article content. Full-text extraction works on freely accessible articles only.
  • Google may serve CAPTCHAs for high-volume requests. Using Apify Proxy significantly reduces this.
  • Rate limiting applies — respect Google's terms of service.

Related Tools

License

ISC License. See LICENSE for details.


Built by george.the.developer on Apify.

from github.com/the-ai-entrepreneur-ai-hub/google-news-scraper

Installing Google News Scraper

This server has no published package — it is built from source. Open the repository and follow its README.

▸ github.com/the-ai-entrepreneur-ai-hub/google-news-scraper

FAQ

Is Google News Scraper MCP free?

Yes, Google News Scraper MCP is free — one-click install via Unyly at no cost.

Does Google News Scraper need an API key?

No, Google News Scraper runs without API keys or environment variables.

Is Google News Scraper hosted or self-hosted?

Self-hosted: the server runs locally on your machine via the install command above.

How do I install Google News Scraper in Claude Desktop, Claude Code or Cursor?

Open Google News Scraper on unyly.org, pick your client tab (Claude Desktop, Claude Code, Cursor) and press Install — the config is generated automatically, no JSON editing.

Related MCPs

Compare Google News Scraper with

Not sure what to pick?

Find your stack in 60 seconds

Author?

Embed badge for your README

Browse similar

All browse MCPs