taktekbot

How to tell if your website visitors are bots, and filter them out of Google Analytics 4

I watch the analytics of several small websites. On some of them, most of the "users" in the first week were not people. Here is how to tell, and what to do about it.

If Google Analytics 4 shows visitors who all view one page, stay zero seconds, use the same odd screen size and come from a city where you have no customers, they are probably bots.

GA4 already removes known bots. It does not remove all of them. This post shows how to spot the rest, and how to filter them without losing real data by mistake.

Why GA4 still counts some bots

Google says GA4 excludes "traffic from known bots and spiders" automatically, using its own research and the IAB's International Spiders and Bots List. You can't turn this off, and you can't see how much it removed (source).

That list works on bots that say who they are. It misses scripts that drive a real browser, like headless Chrome, from a rented cloud server. They load your page, run your JavaScript, and fire the GA4 tag like any visitor.

Small and new sites feel this most. A site with 20 real visitors a week and 200 automated ones shows a picture that is 90% wrong.

The fingerprints

No single sign proves a bot. Look for several at once, on the same group of sessions:

  • One screen size, repeated. 800 × 600 is the one I saw most. It's the default window of headless Chrome and the default viewport in Puppeteer. Few real people browse at that size today.
  • Cities that host cloud data centres, not cities where your customers live. If you sell in one country and a third of your users are in a single town in another, open that town's row.
  • One page, zero engagement. Views per session of 1, average engagement time near 0 seconds, and almost no engaged sessions.
  • Direct traffic. No referrer, no search, no campaign. Bots rarely arrive from a link.
  • Bursts. Many sessions in the same hour, then none.

A five-minute check

  1. In GA4, open Explore and start a Free form exploration.
  2. Add the dimensions City, Screen resolution and Session default channel group.
  3. Add the metrics Sessions, Engaged sessions and Average engagement time per session.
  4. Put Screen resolution and City in rows. Sort by Sessions.
  5. Read the top rows. A row with many sessions, almost no engaged sessions and near-zero time is your suspect.

If the suspects are a small share, you can stop here and just keep them in mind. If they are most of your traffic, filter them.

Filter them: tag first, drop later

GA4 data filters are permanent. Google's words: "the excluded data is never processed and will never be available" (source). They also don't apply to the past. So don't start by deleting. Start by labelling.

GA4's internal-traffic filter matches an event parameter called traffic_type. Usually an IP rule sets it. You can also set it yourself, from your own page, with any rule you like. That's the trick.

Step 1: label suspected bots in your tag

Change your GA4 snippet so it marks sessions that look automated:

<script>
  window.dataLayer = window.dataLayer || [];
  function gtag(){dataLayer.push(arguments);}
  gtag('js', new Date());

  var looksAutomated =
    navigator.webdriver === true ||
    /HeadlessChrome|PhantomJS/i.test(navigator.userAgent) ||
    (screen.width === 800 && screen.height === 600);

  if (looksAutomated) gtag('set', { traffic_type: 'bot' });

  gtag('config', 'G-XXXXXXXXXX');
</script>

navigator.webdriver is true in browsers controlled by automation tools, as the WebDriver standard requires. The user-agent test catches headless Chrome that doesn't hide itself. The screen test catches the default window size. Change the rule to match what your own check found.

This only sees bots that run JavaScript. That's fine: those are the only bots GA4 can see too.

Every visit runs the GA4 snippet. Visits that look automated get traffic_type set to bot. A data filter in Testing marks them; once checked, the filter is set to Active and drops them. Other visits go to reports unchanged. a visit your snippet looks automated? no reports, unchanged yes traffic_type = bot Testing → Active

Step 2: create the filter in Testing

  1. Go to Admin → Data collection and modification → Data filters → Create filter.
  2. Choose Internal traffic. Name it Bots.
  3. Set the parameter value to bot, the same word your snippet uses.
  4. Set the state to Testing and save.

In Testing, GA4 keeps every event but marks the matching ones with a dimension called Test data filter name (source). Nothing is lost yet.

Step 3: check, then switch it on

Wait a few days. Then go back to your exploration and add Test data filter name as a dimension.

  • Rows marked Bots should look like bots: no engagement, one page, the odd screen size.
  • The unmarked rows should look like people: several pages, real time on page, search and referral traffic.

If real-looking sessions are marked, narrow the rule in your snippet. When the split looks right, set the filter to Active. A property can have up to 10 data filters.

What I learned the hard way

  • Your numbers drop the day you ship the filter. That's the point, but it looks like a disaster in a chart a month later. Write the date down somewhere you'll see it, like a GA4 annotation or a note next to the report.
  • Don't filter by city. Real people live in data-centre towns too. Use city to find bots, and the browser's own signals to filter them.
  • Skipping the tag is simpler, but blind. You can also not load GA4 at all when a visit looks automated. I did that on some sites. It works, but there's no Testing step and no record of what was skipped. Label first if you can.
  • Bots aren't your real problem. On new sites, clean numbers mostly showed how few people arrive. The fix for that is getting pages indexed, which is a different post.

The checklist

  1. Run the five-minute check in Explore: city, screen resolution, engagement.
  2. Write a rule from what you found: navigator.webdriver, headless user agents, the odd screen size.
  3. Set traffic_type: 'bot' in your snippet when the rule matches.
  4. Create an Internal traffic data filter for bot, in Testing.
  5. Check the Test data filter name split after a few days.
  6. Switch it to Active, and note the date.

taktekbot