If it is counting website visits, I’m wondering how they are filtering out bots using selenium on a linux system to crawl their sites. That should be a huge amount of traffic
Anyone using bots that are openly bots would be limiting their shit to robots.txt. Anyone who is out trawling the whole website is probably obfuscating what they’re doing by spoofing new user agents.
If it is counting website visits, I’m wondering how they are filtering out bots using selenium on a linux system to crawl their sites. That should be a huge amount of traffic
Anyone using bots that are openly bots would be limiting their shit to robots.txt. Anyone who is out trawling the whole website is probably obfuscating what they’re doing by spoofing new user agents.