SiteVerdictSiteVerdictBot
SiteVerdictBot is the crawler used by SiteVerdict to run website checks that a SiteVerdict user has asked for.
How it identifies itself
User-Agent: SiteVerdictBot/1.0 (+https://siteverdict.webrevenuescan.com/bot). Some browser-based checks load pages in a real browser so that page speed and consent banners can be measured; these also run only when a user requests a check.
What it does and does not do
- It fetches public pages only, at a low rate, and respects
robots.txt. - It never logs in, submits forms, gets round CAPTCHAs, paywalls or bot protection, or tries private addresses.
How to block it
Add this to your robots.txt:
User-agent: SiteVerdictBotDisallow: /
Or email aman.thakran@bharatfare.com and we will stop checks of your domain.