Repository navigation
Show bots and AI crawlers seen on the site before any account; lead the listing with card testing - #120
Merged
Conversation
A fresh install had nothing to show until something tripped a trap: the crawlers a site allows (search engines, AI crawlers) were identified and then dropped without a trace. Every request that names a known agent from the shared registry now adds one to a counter keyed by hour, agent id and verification state (verified by DNS, claimed by User-Agent only, or failed verification). Verification is read from the bot detector's own flags. Behind an unconfigured proxy a failed check is counted as a claim. Nothing else about the request is stored, and requests that name no known agent are not counted. - Dashboard widget: known bot visits in the last 24 hours, AI crawlers and agents, checkout bot signals when WooCommerce is active, the busiest bots, and a connect prompt that states what connecting adds - Statistics page: per-bot table for the last 7 days - New table webdecoy_crawler_visits (DB_VERSION 2.11.0), pruned after 30 days, dropped on uninstall; the version bump also creates the AI referrals table on sites that upgraded past 2.10.0 - AgentRegistry::find() to look up a stored agent id - Cloud upsell copy no longer promises indefinite retention
…ility - Short description, title and opening sections lead with card testing, fake orders and checkout bots, then the new bot visibility - Tags: card testing, fake orders, woocommerce, ai crawlers, bot detection - Good-bot count matches the registry (more than 130, not 60+) - Block AI crawlers: PerplexityBot and other AI search crawlers are treated as search engines and are not blocked; the FAQ and the settings label no longer say otherwise - Remove the "indefinite history" claim - Privacy section describes the bot visit counters
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Plugins that grow on WordPress.org show value before anyone signs up. Ours already protects a site with no account, but a fresh install had little to show: the dashboard widget stayed at zero until something tripped a trap, and the crawlers a site allows (search engines, AI crawlers) were identified and then dropped without a trace.
What changes
Bot visit counters (no account needed). In
early_check(), before the good-bot exit, every request that names a known agent from the shared registry adds one to a counter keyed by(hour, agent_id, verification):verified: the detector's reverse and forward DNS check passed (verified_good_bot)failed: the address did not belong to the bot it claimed (bot_verification_failed), unless the site is behind a proxy the plugin has not been told about, where it counts asclaimedclaimed: identified by User-Agent onlyVerification is read from the detector's flags, not worked out a second time. No IP, User-Agent, path or anything else about a request is stored, and requests naming no known agent (every human visitor) are not counted. Nothing is sent anywhere.
Surfaces
Storage. New table
webdecoy_crawler_visits,DB_VERSION2.3.0 to 2.11.0 (created on activation or the next admin load; recording waits for it), pruned after 30 days by the existing hourly cleanup, dropped on uninstall. The version bump also createswebdecoy_ai_referralson sites that upgraded past 2.10.0 without reactivating, since that table was only ever created on activation.Copy fixes found along the way
readme.txt
WebDecoy Bot Detection - Stop Card Testing, Fake Orders & AI Crawlerscard testing, fake orders, woocommerce, ai crawlers, bot detection. Two narrow symptom tags where a small plugin can actually appear on the tag page,woocommercefor association,ai crawlersfor the visibility angle, andbot detectionto keep the name match. Droppedsecurityandspam protection: both are crowded and the readme body still covers spam.* New: the dashboard widget and Statistics page show bots and AI crawlers seen on your site, verified, claimed or failed, with no account. Only counts are kept.Tests
php tests/run.php: 224 passed, 0 failed. Newtests/CrawlerVisitsTest.phpcovers classification (browsers not counted, DNS verdict, proxy downgrade, realBotDetectorflags), summary folding, labels for every registry category, UTC hour buckets, and source-reading guards: the table has no address/UA/path column, the only write is the four-column upsert, andobserve()runs before the good-bot exit. That last guard was checked to fail when the call is moved after the exit.tests/wp-compat:smoke.shSMOKE OK andwoo.shWOO OK (WP 7.1, PHP 8.3, MariaDB, latest WooCommerce), debug.log empty. Requests sent as GPTBot, ClaudeBot and Googlebot (from a non-Google address) producedclaimed,claimedandfailedrows, and the widget and Statistics page rendered them. A browser request produced no row.Follow-ups (not in this PR)
languages/webdecoy.potat release.