Filtration by User-Agents in the Anti-Crawler Option
We've extended the feature for working with user-agents for the Anti-Spam plugin. This feature is available for the WordPress plugin only. To enable user-agent filtration please open your Anti-Spam plugin settings:
WordPress Dashboard -> Settings -> Anti-Spam by CleanTalk
Enable SpamFireWall option:
Then open advanced settings (WordPress Dashboard -> Settings -> Anti-Spam by CleanTalk -> Advanced settings) and enable "Anti-Crawler" option:
You can see User-Agent in your SpamFireWall logs when Anti-Crawler blocks the request. With "To Personal Black & White Lists" button you can add a User-Agent to your Personal Lists.
Please note that you can add user-agents existing in our list only. Google bot and Bing bot whitelisted by default, you can't add them to your personal lists.
You can manage your User-Agent Personal Lists here. Press "Add new filter" and set filter type as User-Agent:
Then select agents from the drop-down list and Blacklist them:
Here is the list of all available User-Agents:
- admantx.com
- AhrefsBot
- Akamai crawler
- Alexabot
- Applebot
- archive.org_bot
- AspiegelBot - HuaweiWebCatBot
- Babbar.tech (Barkrowler)
- Baidu
- Better Uptime Bot
- Bing
- BrightLocalBot
- CleanTalk Uptime bot
- CloudFlare crawler
- DuckDuckGo
- FacebookBot
- FeedBurner
- Feedly Fetcher
- FreshpingBot
- Grapeshot
- GTmetrix
- GumGum-Bot
- HuaweiWebCatBot
- Hypefactors
- Lighthouse
- Mail.ru
- MailChimp
- Majestic MJ12bot
- Meetedgar.com
- MirrorWeb
- Petalbot
- Pingdom.com bot
- Pinterest bot
- Printful WooCommerce Integration
- proximic
- Qwant Web crawler
- Rambler bot
- RankMathApi
- RapidLoad (rapidload.io)
- Rogerbot - MOZ.com
- Screaming Frog SEO Spider
- Semrush
- SEOkicks-Robot
- serpstatbot
- Seznam
- ShipStation
- ShortPixel Image Optimizer
- Site24x7
- Sitechecker
- sogou spider
- StatusCake
- Sucuri Uptime Monitor
- UptimeRobot.com
- Wikipedia crawler
- WooCommerce API
- wordoftravel: Find Travel Blogs and Explore Destinations
- WP Engine Smart Plugin Manager
- WP Rocket
- WPCompress
- Yandex
These User-Agents are system defined and can't be added to your Personal Lists:
- Baidu
- Bing
- CleanTalk
- Yandex
If you haven't found the answer to your question, please, contact our support team:
https://cleantalk.org/my/support/open