When Blog Traffic Looks Human but Behaves Like a Bot

One day the blog analytics started looking very encouraging: page views had increased noticeably.

Some activity happened at night, but nighttime traffic by itself proves nothing. A public site serves multiple time zones, and real people browse at any hour.

What mattered more was the behavior pattern. Analytics classified much of the traffic as human, while the request pattern looked increasingly unnatural.

Instead of celebrating the numbers or immediately labeling all of the traffic as bots, I started investigating from the lower layers.

A large number by itself is a weak signal.

It is much more useful to ask how that number was produced: over what period, by how many unique visitors, across which pages, and whether the same pattern repeats.

When many views arrive in short bursts, I want to inspect the underlying HTTP requests rather than rely only on aggregated analytics.

Comparing total and deduplicated views was one of the first useful steps.

A large gap between them does not prove automation, but it shows that repeated activity contributes heavily to the total.

That is enough reason to move from the analytics dashboard to server logs.

User-Agent is useful as one signal, but it is supplied by the client itself.

An automated script can identify itself as Android Chrome just as easily as a normal browser can.

I therefore do not classify traffic based only on a browser or device string.

This matches a broader principle: account classification and automated request detection are different things.

This command does not detect bots. It only shows which addresses produced the most entries in the current log.

I then inspect URL sequences, timing between requests, and repeated patterns rather than relying on volume alone.

If the access log uses another format, field positions may differ, so awk commands must be adapted to the actual log_format.

A real visitor may absolutely move quickly through several articles.

But a long sequential traversal across many pages at short intervals provides different context.

That becomes even more significant when the same client also probes URLs that ordinary readers have no reason to request.

Requests for /.env, /app/.env, /.git, or wp-config.php have very different semantics from reading articles.

Automated scanners routinely probe these paths for accidentally exposed secrets, repositories, or configuration files from other platforms.

The combination was especially informative: a client identifies itself as a normal mobile browser, traverses normal pages, and also probes common sensitive paths.

I do not treat every unsuccessful request as the same kind of event.

A 403 may mean the protection worked exactly as intended.

A 404 for a random probe is not necessarily an application problem either.

A 500, however, makes me ask why a dangerous path reached a handler capable of failing at all.

499 is an Nginx status used when the client closes the connection before the server finishes the response.

But one 499 entry does not tell me whether Laravel Analytics definitely recorded or definitely skipped the view.

That depends on how far the request progressed and where the application records its analytics event.

Nginx access-log entries and application analytics therefore should not be expected to match mechanically.

For a sensitive-file probe I expected a denial or a missing resource, but one path variant behaved differently.

/app/.env reached a route that returned 500.

At first this looked like a Laravel problem. The actual cause existed earlier, in Nginx location selection.

In simplified form, the issue was that /app/ had a dedicated ^~ location that proxied requests to another service.

Once Nginx selects such a prefix location, normal regex locations that might block a hidden file are not evaluated.

As a result, /app/.env bypassed the expected protection rule and reached a completely different upstream.

The unexpected 500 therefore became a useful signal that the configuration boundary was incomplete.

nginx -T shows the effective configuration with included files already expanded.

This matters when rules are split across multiple files or when one location quietly takes precedence over another.

I want to inspect what Nginx actually executes.

Nginx sees the request before Laravel does.

Some requests may terminate at the web-server layer, others reach the application, and analytics applies its own inclusion, deduplication, and classification rules.

I use these sources together, but I do not expect identical numbers from them.

During this kind of investigation it is easy to move from "many suspicious requests came from this region" to "all traffic from this country is automated".

I do not make that jump.

Country is only one analytics dimension. Real readers, VPN users, mobile networks, cloud infrastructure, and automated clients may all appear in the same geographic slice.

I classify behavior rather than a visitor nationality or geography.

Once some historical traffic starts looking suspicious, it may be tempting to retroactively remove it from every analytics chart.

I do not do that without a separate, reliable migration rule.

New evidence can improve future classification and protection, but it does not automatically make every historical record unambiguously wrong.

My short checklist:

• How do total and deduplicated views compare?
• Are there short bursts of repetitive traffic?
• Which IPs produce the most access-log entries?
• Does one source traverse pages sequentially?
• Are there probes for .env, .git, wp-config.php, or similar paths?
• Which HTTP statuses are returned?
• Does an unexpected 500 reveal an incorrect Nginx location boundary?
• Does classification rely too heavily on User-Agent?
• Am I labeling traffic as automated based only on country?
• Am I treating Nginx Logs and Laravel Analytics as different sources rather than identical counters?

The most important result of this investigation was not assigning a "bot" label to a particular group of visitors.

It was understanding the limits of the analytics layer itself.

User-Agent can be spoofed. A high view count is not the same as a large audience. An Nginx 499 does not prove that Laravel recorded a view. Country alone is not enough reason to block traffic.

The server logs instead revealed behavior that an aggregate analytics counter could not explain by itself: repeated sources, sequential page crawling, and probes for sensitive paths.

The unexpected 500 on /app/.env also exposed a gap in the Nginx location rules.

I trust a combination of signals across system layers rather than one convenient classification label.

The next step was no longer investigation but protection: stop high-confidence dangerous requests at the web-server layer and temporarily block sources that keep repeating them.