Googlebot-News

Difference from the standard Googlebot

The classic Googlebot crawls every reachable page for the general web index. Googlebot-News is a second user-agent name that Google uses when newsworthy URLs are considered for the news index. Both share renderer, resource budget and IP range — the difference lies in the target index and in the frequency: news URLs are often rechecked every few minutes so that current reports reach "Top Stories" without delay.

Robots.txt and meta robots tags allow separate control. A site can permit Googlebot but block Googlebot-News — or the other way round. Anyone who deliberately wants only long-form content in the classic index and seeks no news exposure blocks Googlebot-News with a targeted Disallow.

How a website is admitted to Google News

Since December 2019 there has been no separate application review. Google replaced the former "News Producer" application procedure with an automatic assessment: in principle every public website can appear in Google News, provided it meets the editorial standards and does not block Googlebot-News.

Anyone wishing to steer visibility deliberately also registers in the Publisher Center (publishercenter.google.com). The Publisher Center supplies publication branding (logo, colour), allows curated sections, activates rich results in the news carousel and makes performance reports per publication accessible. It is not a requirement for inclusion, but it is a must for professional publishers.

The basic requirements remain: transparent authorship with full name and short profile, a clear imprint, editorial standards and correction notices, consistent publication frequency, as well as editorial separation between journalism and advertising.

News sitemap (sitemap-news.xml) — mandatory for current reports

A news sitemap is the fastest way for Googlebot-News to discover new articles within a few minutes. It follows its own XML namespace and is strictly limited: a maximum of 1,000 URLs, exclusively articles younger than 48 hours. Older reports are removed and kept accessible via the regular sitemap.xml.

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
        xmlns:news="http://www.google.com/schemas/sitemap-news/0.9">
  <url>
    <loc>https://beispiel-magazin.de/news/2026-07-01-titel</loc>
    <news:news>
      <news:publication>
        <news:name>Beispiel-Magazin</news:name>
        <news:language>de</news:language>
      </news:publication>
      <news:publication_date>2026-07-01T09:15:00+02:00</news:publication_date>
      <news:title>Titel der Meldung</news:title>
    </news:news>
  </url>
</urlset>

The news sitemap should additionally be submitted in Search Console as a separate sitemap and entered in the robots.txt under Sitemap: — where possible with a dedicated file name such as sitemap-news.xml, to separate it clearly from the main index.

NewsArticle schema as a rich-result requirement

Without NewsArticle schema, reports do appear in the news index, but not in rich results (carousel with image, publisher logo, byline). For the rich-result markup Google requires at least the following fields in JSON-LD:

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "NewsArticle",
  "headline": "Titel der Meldung",
  "image": ["https://beispiel-magazin.de/images/artikel.jpg"],
  "datePublished": "2026-07-01T09:15:00+02:00",
  "dateModified": "2026-07-01T11:30:00+02:00",
  "author": [{
    "@type": "Person",
    "name": "Anna Schmitt",
    "url": "https://beispiel-magazin.de/autoren/anna-schmitt"
  }],
  "publisher": {
    "@type": "Organization",
    "name": "Beispiel-Magazin",
    "logo": {
      "@type": "ImageObject",
      "url": "https://beispiel-magazin.de/logo.png"
    }
  }
}
</script>

The headline must match the visible <h1>, the image must be at least 1,200 pixels wide and the author must have their own verifiable author page. If one of these fields is missing, the report loses its rich-result status — and with it, usually, the carousel placement.

robots.txt: controlling Googlebot-News specifically

Googlebot-News can be controlled independently of the standard Googlebot via robots.txt. Four practical patterns:

# Muster 1: alles erlauben (Standard)
User-agent: *
Allow: /

# Muster 2: klassisches Web crawlen, News-Sektion aus News-Index ausschliessen
User-agent: Googlebot-News
Disallow: /archiv/
Disallow: /kolumne/

# Muster 3: nur News-Sektion an News-Crawler freigeben
User-agent: Googlebot-News
Allow: /nachrichten/
Disallow: /

# Muster 4: komplett aus Google News raus, klassisches Ranking bleibt
User-agent: Googlebot-News
Disallow: /

A frequent mistake: User-agent: Googlebot with Disallow: / also blocks Googlebot-News, because the news crawler inherits the standard directives. Anyone who wants only news traffic without appearing in the classic index has to treat both user agents separately — and as a rule needs professional advice, because this is an atypical special case.

Content signals that Googlebot-News rewards

Google distinguishes news content from evergreen content through several signals at once:

• Timeliness — datePublished in structured data, <time datetime> in the HTML, date information in the visible text.
• Authorship — byline with full name, linked author profile with short biography and further articles.
• Publication context — clear imprint, transparent editorial standards, correction policy.
• Canonical URL structure — every article under a unique URL, no duplicate news, no auto-generation of content.
• Event relevance — the report demonstrably refers to a current event, not to timeless evergreen material.
• Consistent frequency — regular publication over months. Irregular one-report sites rarely stay in the news index for long.

Why sites drop out of Google News again

Appearing in the news index is no guarantee of a permanent place. Common reasons for removal:

• Editorial standards violated — hallucinated AI content without editorial oversight, missing authors, unclear imprint.
• Clickbait headlines — titles that the article does not deliver on.
• Date manipulation — repeatedly redating old articles to pass them off as "new". Google recognises this by comparing older crawl snapshots.
• Duplicates — identical reports under several URLs, without rel="canonical".
• Violation of the content policies — hate speech, false medical claims, terrorist propaganda or fabricated quotations lead to manual exclusion without warning.
• Excessive advertising above the fold — interstitials, auto-play video, more than 30 % ad space above the first visible line.

The news exclusion is visible in the Publisher Center and can be addressed there via a reconsideration request — once the cause has demonstrably been resolved.