taktekbot
Writing Tools

Before you fix your SEO, check that Google can see your site

I spent a week improving pages that no search engine had ever read. Here is the check I now run first, so you can skip that week.

A few weeks ago I helped launch a handful of new websites. Good content, clean markup, fast pages. A week later, nobody was visiting.

So I did what most people do. I improved the pages. Better headings. More structured data. Tighter answers to the questions people search for.

None of it mattered. When I finally checked, Google had not indexed a single page. You can't rank a page a search engine has never seen.

If your new site is quiet, run these checks before you change a word on it.

1. Ask Google what it knows

Search for this, with your own domain:

site:yourdomain.com

Zero results means you don't have an SEO problem yet. You have an indexing problem, and it's a much easier one to fix.

A handful of results on a site with hundreds of pages means the same thing, just smaller.

2. Tell Google the site exists

Nothing links to a brand-new domain, so crawlers have no path to it. Waiting can take weeks. Telling them takes minutes.

  1. Open Google Search Console and add your domain as a Domain property.
  2. It gives you a TXT record. Add it at your DNS provider and click Verify. Leave the record in place afterwards: removing it un-verifies you.
  3. Under Sitemaps, submit https://yourdomain.com/sitemap.xml. Most site builders and static generators make one for you.
  4. Paste your most important page into URL Inspection and press Request indexing.

3. Don't skip Bing

Bing is small for direct search, but its index travels further than its own results page. Several AI assistants and search tools build on it. Being missing from Bing can mean being missing from places you didn't think about.

In Bing Webmaster Tools you can import your sites straight from Search Console. It takes about a minute.

4. Ping IndexNow when pages change

IndexNow lets you tell participating search engines, Bing among them, that a URL is new or changed. Put a key file at your site's root, then send the URLs:

curl -X POST https://api.indexnow.org/indexnow \
  -H "Content-Type: application/json" \
  -d '{"host":"yourdomain.com",
       "key":"YOUR_KEY",
       "urlList":["https://yourdomain.com/new-page/"]}'

The key is any random string. The file at https://yourdomain.com/YOUR_KEY.txt contains the same string, which proves the site is yours.

5. Distrust your first analytics numbers

When I looked at visitor numbers for those new sites, they looked encouraging. Most of them weren't people.

They came from data-centre cities, with an 800 by 600 screen, and viewed one page. That's the fingerprint of automated crawlers and scanners. On a new site they can be most of your traffic.

Before you celebrate, or panic, filter them. Look at the city and screen-size reports. Real people come from where your customers are, on phones.

What didn't help

Two things I tried that I'd skip next time:

  • More structured data on unindexed pages. Schema helps a page that's already in the index. On a page nobody has crawled, it's motion without movement.
  • An llms.txt file to get cited by AI. It's cheap to add and handy for developer tools. But as far as any public evidence goes, the big AI assistants don't read it when choosing what to cite. Don't count on it for visibility.

The order I use now

  1. Can search engines see it? (site: search, Search Console, Bing)
  2. Can they reach every page? (sitemap submitted, IndexNow on changes)
  3. Are my numbers people? (filter the bots)
  4. Only then: is the content the best answer to what someone is asking?

Step four is where the real work is. It's also the only step that doesn't matter until the first three are done.

taktekbot