←% cd ..

Auto-Push URLs to Bing Search Engines with IndexNow

|indexnow
TechAutomationSearchGitHub ActionsIndexNow
日本語で読む
Auto-Push URLs to Bing Search Engines with IndexNow

IndexNow — Push-Notifying Search Engines Yourself

For starters, I registered my sitemap.xml with two major search platforms (Google Search Console and Bing Webmaster Tools), thinking "Alright, crawling is sorted." But that's the tragedy of running a minor site—it takes forever to actually get crawled.

That's where IndexNow comes in (and hopefully helps).
IndexNow is a protocol jointly announced by Microsoft and Yandex in 2021 that lets you actively notify search engines of new or updated pages.

  • Supported search engines: Bing, Yandex, Naver (South Korea), Seznam (Czech Republic)
  • Google: Not on board (Google prefers to discover things using their own crawlers)
  • How it works: Simply send your URLs via an HTTP POST request

It sounded promising, and since it's already integrated right into the Bing Webmaster Tools menu, I figured it was finally time to set it up. It’s a shift from a passive approach ("Here's my sitemap, please grab it whenever you visit") to a proactive stance ("I just published a new post, come index it right now!").

Going Public Is What Proves Domain Ownership

Usually, Web APIs handle authentication using a secret API key. IndexNow, however, goes for a much simpler approach:

  1. Generate a key however you want (an 8 to 128-character alphanumeric string with hyphens).
  2. Place a text file containing that key at https://yourdomain/<key>.txt.
  3. Include the key and keyLocation in your API requests.
  4. The search engine sends a GET request to the keyLocation and verifies the content matches.

At first glance, putting a key in a public folder might seem insecure, but IndexNow's authentication isn't about keeping the key itself a secret. Sure, it's better if it doesn't get leaked, but both the key and keyLocation are sent in plain text in the API request anyway, and the search engines log them on their end anyway.
What actually keeps it secure is the "proof of control" concept: the fact that you can host content at this URL proves you own the domain. It’s the same logic behind ACME challenges for HTTPS certificates or Google Search Console's HTML file verification.

Honestly, even if a bad actor gets hold of your API key, the only thing they can do is ask Bing to index your site's pages. Since it's locked down to your domain, leaks aren't really an issue. It’s a quietly clever design.

Setting It Up in 5 Minutes

Bing Webmaster Tools IndexNow API Key generation screen

  1. Get an API key from Bing Webmaster Tools
    As I mentioned, you can technically generate your own key as long as it fits the rules. But since Bing has "IndexNow" built right into the left menu of its dashboard, with a [Generate] button that creates the key for you, I went with that.
    It gives you a UUID-like 32-character hexadecimal string.

  2. Place the key file at the root
    Since this blog is built on Next.js and hosted on Vercel, I just needed to drop it in public/<key>.txt. The file content is literally just the key string.
    Once pushed and deployed, as long as it's accessible at https://diary.dazydayz.com/<key>.txt, you're good to go.

  3. Test the API
    There are two ways to send URLs to IndexNow: \

    • GET: Send one URL at a time using ?url=X&key=Y&keyLocation=Z \
    • POST: Send up to 10,000 URLs at once in a urlList array
      Since I publish 2+ URLs per post on this blog (JA and EN, possibly more languages later), batching them into one POST request is more efficient than firing them off one by one — so I went with POST urlList.

Let's test it using curl:

curl -X POST "https://api.indexnow.org/indexnow" \
  -H "Content-Type: application/json" \
  -d '{
    "host": "diary.dazydayz.com",
    "key": "<KEY>",
    "keyLocation": "https://diary.dazydayz.com/<KEY>.txt",
    "urlList": [
      "https://diary.dazydayz.com/ja/posts/2026/05/tokuryu",
      "https://diary.dazydayz.com/en/posts/2026/05/tokuryu"
    ]
  }'

I tried testing it with the URLs from the Tokuryu post and got a 202 back.

202 Accepted: URL received. IndexNow key validation pending.

This status means the URL was received, but the key hasn't been validated yet. Apparently on the first API call, IndexNow hadn't fetched and verified the keyLocation content.
When I resent the same request a short while later, I got a 200 OK.

Automating It with GitHub Actions

Running curl manually every time is a pain, and automation is the whole point, so I set up a workflow to automatically ping IndexNow whenever changes are pushed to posts/.

The Script (scripts/indexnow-ping.ts)

This script filters the modified markdown files passed as arguments, grabs only those with status: published, and POSTs their corresponding public URLs to the IndexNow API:

const HOST = "diary.dazydayz.com";
const KEY = process.env.INDEXNOW_KEY!;

// posts/ja/2026/05/indexnow.md → /ja/posts/2026/05/indexnow
const m = file.match(/^posts\/(ja|en)\/(.+)\.md$/);
urls.push(`https://${HOST}/${m[1]}/posts/${m[2]}`);

await fetch("https://api.indexnow.org/indexnow", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    host: HOST,
    key: KEY,
    keyLocation: `https://${HOST}/${KEY}.txt`,
    urlList: urls,
  }),
});

I also added support for a --dry-run flag so I can double-check which URLs will be notified locally before actually sending them.

$ npx tsx scripts/indexnow-ping.ts posts/ja/2026/05/indexnow.md --dry-run
Notified 1 URL to IndexNow:
   - https://diary.dazydayz.com/ja/posts/2026/05/indexnow

The Workflow (.github/workflows/indexnow.yml)

on:
  push:
    paths:
      - 'posts/ja/**.md'
      - 'posts/en/**.md'
    branches:
      - main

jobs:
  ping:
    steps:
      - uses: actions/checkout@v5
        with:
          fetch-depth: 2

      - name: Detect changed posts
        id: changes
        run: |
          FILES=$(git diff --name-only --diff-filter=AM HEAD~1 HEAD \
            -- 'posts/ja/**.md' 'posts/en/**.md' | tr '\n' ' ')
          echo "files=$FILES" >> $GITHUB_OUTPUT

      - name: IndexNow ping
        if: steps.changes.outputs.files != ''
        env:
          INDEXNOW_KEY: ${{ secrets.INDEXNOW_KEY }}
        run: npx tsx scripts/indexnow-ping.ts ${{ steps.changes.outputs.files }}

Key points:

  • We use --diff-filter=AM to only pick up Added or Modified files (ignoring Deleted ones).
  • The status: published filtering is handled inside the script (so draft posts don't trigger a ping).
  • The API key is read from GitHub secrets (meaning no hardcoded keys in the script or workflow files).

Wrapping Up

Pushing this article itself is the first live test of the workflow.
If everything runs smoothly, I should see a recent crawl record for this URL in Bing Webmaster Tools' URL Inspection tool within a few minutes after pushing.

With this setup, pushes now trigger an automatic ping to multiple search engines within minutes.
Unfortunately, Google isn't part of this alliance, so I'll still have to wait for their regular GSC crawl cycle.