AfitoBot
AfitoBot is the web crawler run by Afito. Afito records how apartment buildings in the San Francisco Bay Area lease over time: which units are available, their asking rents and the concessions offered. If you saw this user agent in your logs, this page explains what it did and how to stop it.
AfitoBot/0.1 (+https://afito.app/bot)
What it fetches
- Public, logged-out leasing pages and availability feeds of apartment buildings.
- About once a day per building, a few requests at a time. It does not crawl the rest of a site.
- Public listing data: unit numbers, floor plans, asking rents, availability dates and advertised concessions, plus building photos and floor-plan images.
- Afito archives those images and shows them with attribution and a link to the page they came from. They remain the property of their owners.
Verifying AfitoBot
Requests from AfitoBot are signed with HTTP Message Signatures (RFC 9421) using an Ed25519 key, following
Cloudflare's Web Bot Auth.
Each request carries Signature-Agent: "https://afito.app", Signature-Input
(tag="web-bot-auth") and Signature headers. The public key is published, itself
signed, at
afito.app/.well-known/http-message-signatures-directory.
A request that claims to be AfitoBot but doesn't verify against that key isn't ours.
Rate and operator
- Operator: Afito (afito.app), a Bay Area apartment market tracker.
- Rate: about one crawl per building per day, serially (no parallel requests to a host), roughly 2 to 15 requests per building, plus one weekly look at a building's home page and sitemap, and image downloads (at most 8 per building per day). Source of requests: Cloudflare's network (Workers), no fixed IP range.
- Contact: hello@afito.app.
What it doesn't do
- It never signs in, gets around a login or CAPTCHA, or submits forms.
- It collects no personal data about tenants or applicants.
- It doesn't hide its identity: every request carries the user agent above.
robots.txt
AfitoBot honors robots.txt (RFC 9309). Before each crawl it reads the site's robots.txt, follows the rules in
the User-agent: AfitoBot group or, if there is none, the User-agent: * group, and
skips any page they disallow. It keeps a copy for up to 24 hours, so changes take effect within a day. If
robots.txt answers with a server error, it fetches nothing from the site until robots.txt can be read. To stop
it reading your site, add this to your robots.txt:
User-agent: AfitoBot Disallow: /
Contact and opt-out
To stop AfitoBot, use the robots.txt rule above (User-agent: AfitoBot). To have images of your
building removed, or to opt out of image collection and display, or to send a copyright notice, see
afito.app/copyright or email
copyright@afito.app; we aim to remove images within one business
day. For anything else, such as stopping the crawl of a property, reporting a problem or asking a question,
email hello@afito.app with the domain or building.