AfitoBot

AfitoBot is the web crawler run by Afito. Afito records how apartment buildings in the San Francisco Bay Area lease over time: which units are available, their asking rents and the concessions offered. If you saw this user agent in your logs, this page explains what it did and how to stop it.

AfitoBot/0.1 (+https://afito.app/bot)

What it fetches

Verifying AfitoBot

Requests from AfitoBot are signed with HTTP Message Signatures (RFC 9421) using an Ed25519 key, following Cloudflare's Web Bot Auth. Each request carries Signature-Agent: "https://afito.app", Signature-Input (tag="web-bot-auth") and Signature headers. The public key is published, itself signed, at afito.app/.well-known/http-message-signatures-directory. A request that claims to be AfitoBot but doesn't verify against that key isn't ours.

Rate and operator

What it doesn't do

robots.txt

AfitoBot honors robots.txt (RFC 9309). Before each crawl it reads the site's robots.txt, follows the rules in the User-agent: AfitoBot group or, if there is none, the User-agent: * group, and skips any page they disallow. It keeps a copy for up to 24 hours, so changes take effect within a day. If robots.txt answers with a server error, it fetches nothing from the site until robots.txt can be read. To stop it reading your site, add this to your robots.txt:

User-agent: AfitoBot
Disallow: /

Contact and opt-out

To stop AfitoBot, use the robots.txt rule above (User-agent: AfitoBot). To have images of your building removed, or to opt out of image collection and display, or to send a copyright notice, see afito.app/copyright or email copyright@afito.app; we aim to remove images within one business day. For anything else, such as stopping the crawl of a property, reporting a problem or asking a question, email hello@afito.app with the domain or building.