Why My Blog Doesn't Show Up on Google — Complete Google Indexing API Setup Guide

My Content Exists, But Google Doesn’t Know

Creating the blog took longer than expected. I chose Astro, set up deployment on Cloudflare Pages, tweaked the theme, and published my first post. Deployment succeeded and the content showed up properly when accessing the URL. But when I searched on Google, nothing appeared.

Searching for the “blog name” or copy-pasting post titles yielded no results. Initially I thought deployment went wrong and rechecked, wondered if robots.txt was blocking and opened it, suspected Cloudflare settings were the issue. Everything was normal. Google simply hadn’t “discovered” my blog yet.

This situation affects almost anyone with a new blog. While Google crawls billions of pages daily, it can take days to weeks for new sites to join the crawling queue. Though it seems like waiting is the only option, using the Google Indexing API can significantly shorten this process. After requesting indexing for all 20 posts, most appeared in Google search results within hours.

1. How Google Finds My Blog

Google’s web indexing process has three stages: Crawling → Indexing → Serving.

In the crawling stage, Googlebot follows links to discover new pages across the web. The problem is if no one has linked to my blog yet, there’s no path for Googlebot to “reach” my site. Submitting a sitemap signals to Google “please look at these URLs,” but when this signal actually leads to crawling is up to Google’s discretion.

In the indexing stage, crawled page content is analyzed and added to the search index. Just because a page was crawled doesn’t mean it gets indexed immediately. Various factors like page quality, duplicate content, and server response time influence this.

The Indexing API is a means to directly send Google requests saying “please crawl this URL right now” in this process. Officially it’s for pages containing JobPosting or BroadcastEvent structured data¹, but in practice Google responds and performs crawling even when sent general blog post URLs. This is empirically confirmed by many developers and bloggers, and indeed all 20 posts got indexed.

However, using the API doesn’t guarantee indexing. It increases the probability that Google will accept the request and crawl, but it’s not a means to force index creation. Page quality and content credibility must support actual appearance in search results.

2. Prerequisites Checklist

Here’s what you need before starting setup:

• Google account
• Site registered in Google Search Console (ownership verification complete)
• Google Cloud Console project
• Python 3.x environment (google-auth, requests libraries)
• Deployed Astro blog’s sitemap-0.xml URL

3. Google Search Console Ownership Verification

To use the Indexing API, you must first verify site ownership in Search Console. Skip this step if already completed.

Go to Google Search Console and click “Add property.” There are two property types:

• Domain property: logdew.com format. Only verifiable via DNS TXT record. Recommended as it manages all subdomains and http/https integration.
• URL prefix property: https://logdew.com/ format. Verifiable through various methods including HTML file upload, meta tags, Google Analytics, etc.

DNS TXT record method is cleanest for Cloudflare Pages. In Cloudflare dashboard → relevant domain → DNS → Records tab, add a TXT record. Set name field to @ (root domain) and paste the verification code provided by Search Console in the value field.

After adding the record, return to Search Console and click “Verify.” Verification typically completes immediately or within minutes. DNS propagation can take up to 48 hours in slow cases, but it’s usually much faster.

4. Sitemap Submission

Once ownership verification is complete in Search Console, register the sitemap. Click Sitemaps in the left sidebar and enter the sitemap URL in the “Add a new sitemap” field.

Astro’s @astrojs/sitemap package generates both sitemap-index.xml and sitemap-0.xml by default². Submit sitemap-index.xml to Search Console. For example, entering https://logdew.com/sitemap-index.xml will let Google read this index file and automatically discover the internal sitemap-0.xml.

Once submitted and status changes to “Success,” the sitemap is registered. However, registration and indexing are different. Even with success status, it’s common to remain in “Discovered – currently not indexed” state for days. This is where the Indexing API becomes necessary.

5. Google Cloud Console: Service Account Creation

The Indexing API authenticates with OAuth 2.0 service accounts. Rather than personal Google accounts, these are “machine-only accounts” for programmatic API access.

5-1. Project Creation and API Activation

Go to Google Cloud Console. Select Create New Project from the top dropdown and specify an appropriate name (e.g., logdew-indexing). Once the project is created, go to APIs & Services → Library from the left menu. Search for Indexing API and Web Search Indexing API will appear. Click it and press ENABLE.

5-2. Service Account Creation

Go to APIs & Services → Credentials → Create Credentials → Service Account. Enter a name (e.g., indexing-bot) and click “Create and Continue.” The permissions setting step is optional so you can skip it. Click “Done” to create the service account.

Note the created service account’s email address. The format will be like indexing-bot@logdew-indexing.iam.gserviceaccount.com. This email will be registered as an owner in Search Console.

5-3. JSON Key Download

Click the created service account and go to the Keys tab. Select Add Key → Create New Key → JSON format and the key file will automatically download. This file can only be downloaded once, so store it in a secure location. Never upload it to public repositories like GitHub.

6. Register Service Account as Owner in Search Console

For service accounts to submit URLs through the Indexing API, they must be registered as Owners for the relevant site in Search Console.³

Go to Search Console ownership verification page (or Settings → Users and Permissions within the relevant property in Search Console). Click “Add user” and enter the service account email address noted in 5-2. Permissions must be set to Owner. You need verified owner permissions, not “delegated owner.”

Adding this only as “Editor” permissions in the general “Users” tab rather than the “Property owners” tab will result in 403 errors during API requests. You must add with owner permissions.

7. Python Environment Setup

Install required libraries:

pip install google-auth requests

google-auth is Google’s official library that reads service account JSON files and automatically issues access tokens.⁴ No need to manually refresh OAuth tokens each time; it automatically reissues upon expiration.

8. Single URL Indexing Request Script

To understand the basic structure, I first wrote a script to request indexing for a single URL.

import json
import requests
from google.oauth2 import service_account
from google.auth.transport.requests import Request

# Service account key file path
SERVICE_ACCOUNT_FILE = "service_account.json"

# Indexing API endpoint
ENDPOINT = "https://indexing.googleapis.com/v3/urlNotifications:publish"

# Authentication scope
SCOPES = ["https://www.googleapis.com/auth/indexing"]

def get_credentials():
    credentials = service_account.Credentials.from_service_account_file(
        SERVICE_ACCOUNT_FILE,
        scopes=SCOPES
    )
    credentials.refresh(Request())
    return credentials

def request_indexing(url: str, notification_type: str = "URL_UPDATED"):
    credentials = get_credentials()
    headers = {
        "Content-Type": "application/json",
        "Authorization": f"Bearer {credentials.token}"
    }
    body = json.dumps({
        "url": url,
        "type": notification_type  # URL_UPDATED or URL_DELETED
    })
    response = requests.post(ENDPOINT, headers=headers, data=body)
    return response.json()

if __name__ == "__main__":
    url = "https://logdew.com/posts/my-first-post"
    result = request_indexing(url)
    print(result)

Success returns a response like this:

{
  "urlNotificationMetadata": {
    "url": "https://logdew.com/posts/my-first-post",
    "latestUpdate": {
      "url": "https://logdew.com/posts/my-first-post",
      "type": "URL_UPDATED",
      "notifyTime": "2026-02-25T01:23:45.678Z"
    }
  }
}

This response means “request accepted,” not that indexing was immediately created. Changes typically appear in Google Search Console’s URL inspection results within minutes to hours.

9. Batch Indexing All URLs by Parsing sitemap-0.xml

With 20 posts, entering URLs one by one was inefficient. I wrote a script to parse Astro’s auto-generated sitemap-0.xml, extract all URLs, and batch request indexing.

import json
import time
import requests
import xml.etree.ElementTree as ET
from google.oauth2 import service_account
from google.auth.transport.requests import Request

SERVICE_ACCOUNT_FILE = "service_account.json"
ENDPOINT = "https://indexing.googleapis.com/v3/urlNotifications:publish"
SCOPES = ["https://www.googleapis.com/auth/indexing"]

# Sitemap URL for the site to request indexing
SITEMAP_URL = "https://logdew.com/sitemap-0.xml"

def get_credentials():
    credentials = service_account.Credentials.from_service_account_file(
        SERVICE_ACCOUNT_FILE,
        scopes=SCOPES
    )
    credentials.refresh(Request())
    return credentials

def get_urls_from_sitemap(sitemap_url: str) -> list[str]:
    response = requests.get(sitemap_url)
    response.raise_for_status()
    root = ET.fromstring(response.content)
    # Sitemap XML namespace
    ns = {"sm": "http://www.sitemaps.org/schemas/sitemap/0.9"}
    urls = [loc.text for loc in root.findall("sm:url/sm:loc", ns)]
    return urls

def request_indexing(url: str, credentials) -> dict:
    headers = {
        "Content-Type": "application/json",
        "Authorization": f"Bearer {credentials.token}"
    }
    body = json.dumps({"url": url, "type": "URL_UPDATED"})
    response = requests.post(ENDPOINT, headers=headers, data=body)
    return response.json()

def main():
    print(f"Parsing sitemap: {SITEMAP_URL}")
    urls = get_urls_from_sitemap(SITEMAP_URL)
    print(f"Found {len(urls)} URLs total")

    credentials = get_credentials()

    for i, url in enumerate(urls, 1):
        # Token expiration protection: refresh every 10 requests
        if i % 10 == 0:
            credentials.refresh(Request())

        result = request_indexing(url, credentials)
        if "urlNotificationMetadata" in result:
            print(f"[{i}/{len(urls)}] Success: {url}")
        else:
            print(f"[{i}/{len(urls)}] Error: {url}")
            print(f"  Response: {result}")

        # API rate limit handling (380 requests/minute limit)
        time.sleep(0.5)

    print("\nComplete. Check indexing status in Search Console.")

if __name__ == "__main__":
    main()

Astro’s sitemap XML uses the http://www.sitemaps.org/schemas/sitemap/0.9 namespace. Without specifying the namespace, root.findall("url/loc") returns empty results, so be careful.

Running this produced output like:

Parsing sitemap: https://logdew.com/sitemap-0.xml
Found 21 URLs total
[1/21] Success: https://logdew.com/
[2/21] Success: https://logdew.com/posts/my-first-post
[3/21] Success: https://logdew.com/posts/another-post
...
[21/21] Success: https://logdew.com/posts/last-post

Complete. Check indexing status in Search Console.

All 21 (20 posts + index page) received “success” responses. Script execution took about 15 seconds, and when checking one by one with Search Console’s URL inspection tool afterward, most had completed crawling within the same day. What used to take days of waiting for indexing was reduced to hours.

10. API Usage Limits and Precautions

Basic Indexing API quotas are as follows⁵:

200 daily requests are sufficient for new blog operations. If posts exceed 200, you can submit over multiple days or request quota increases from Google.

One important point: the Indexing API officially supports pages containing JobPosting or BroadcastEvent structured data⁶. General blog posts aren’t officially supported targets. Nevertheless, crawling requests are often accepted and indexing created. Since this isn’t explicitly permitted usage by Google, behavior may change with future policy updates.

Also, accepted indexing requests don’t guarantee search result appearance. Google comprehensively evaluates content quality, domain trustworthiness, duplication, etc. to determine index inclusion. The Indexing API is merely a means to raise crawling queue priority.

11. Automated Indexing Request Integration on Deploy (Optional)

You can also set up automatic indexing request transmission with each Cloudflare Pages deployment.

Method 1: Cloudflare Pages Deploy Hook + GitHub Actions

This approach uses GitHub Actions to detect deployment completion events and run Python scripts.

# .github/workflows/indexing.yml
name: Google Indexing After Deploy

on:
  deployment_status:

jobs:
  index:
    if: github.event.deployment_status.state == 'success'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - name: Install dependencies
        run: pip install google-auth requests
      - name: Run indexing script
        env:
          SERVICE_ACCOUNT_JSON: ${{ secrets.GOOGLE_SERVICE_ACCOUNT_JSON }}
        run: |
          echo "$SERVICE_ACCOUNT_JSON" > service_account.json
          python indexing.py

By registering service account JSON file content as GOOGLE_SERVICE_ACCOUNT_JSON in GitHub Actions Secrets, full sitemap-based indexing requests run automatically with each deployment.

Method 2: Selective Requests for New Posts Only

Submitting entire sitemaps with each deployment can quickly exhaust the 200 daily quota, especially as posts multiply. The solution is using git change history to extract URLs only from newly added .md files.

import subprocess
import json

def get_new_posts() -> list[str]:
    """Returns list of newly added post files from last commit"""
    result = subprocess.run(
        ["git", "diff", "--name-only", "--diff-filter=A", "HEAD~1", "HEAD"],
        capture_output=True, text=True
    )
    new_files = result.stdout.strip().split("\n")
    # Filter only src/content/posts/*.md files
    post_files = [f for f in new_files if f.startswith("src/content/posts/") and f.endswith(".md")]
    
    # Convert file paths to URLs
    base_url = "https://logdew.com"
    urls = []
    for f in post_files:
        # src/content/posts/my-post.md → /posts/my-post
        slug = f.replace("src/content/posts/", "").replace(".md", "")
        urls.append(f"{base_url}/posts/{slug}")
    return urls

This way only 1 request per new post is sent, eliminating quota waste.

12. Result Verification

There are two ways to verify results after indexing requests:

Google Search Console URL Inspection Tool: Click URL Inspection in Search Console’s left menu and enter a URL to see “Google has confirmed this URL” status and last crawling date.

Direct Google Search Verification: Enter site:logdew.com in Google search to see the list of indexed pages. Before index creation, there are no results or only partial ones. Most results changed within a day after Indexing API requests.

Final completion is confirmed when Search Console’s Index Coverage report status changes from “Crawled – currently not indexed” to “Indexed.”

Conclusion

When blogs don’t appear on Google, it’s usually not a technical defect but because Google hasn’t discovered the site yet. Sitemap submission is a signal showing direction, and the Indexing API is a means to raise crawling queue priority.

Setup itself wasn’t as complex as expected. 10 minutes to create a project in Google Cloud Console and generate a service account key, 5 minutes to register as owner in Search Console, 15 seconds to submit the entire sitemap with a Python script. Afterward, it was fun watching index status changes in Search Console.

The important point is that the Indexing API is a crawling request, not an indexing guarantee. Content quality itself must be supported. The API is a means to quickly knock on Google’s door, but whether the door opens still depends on the content.