Crawler
HowItsBuiltBot
HowItsBuiltBot fetches a website's homepage when someone looks it up on howitsbuilt.fyi, to detect which technologies it uses.
How it behaves
- User agent:
Mozilla/5.0 (compatible; HowItsBuiltBot/1.0; +https://howitsbuilt.fyi/bot) - One request for the homepage per lookup, following at most 3 redirects. No crawling of other pages.
- Results are cached for 24 hours, so a site is fetched at most about once a day per lookup.
- It only runs on user request; it does not crawl the web on its own.
Opting out
If you would rather not be looked up, e-mail lucas@lkmedia.net with your domain and we will exclude it.