BenchologyCompetitorBot.

BenchologyCompetitorBot is operated by Benchology to keep an internal record of public changes on the websites of companies in our market: new pages, posts, product and pricing pages, job listings. It is not a search engine and does not republish content.

What it fetches

• About seven public pages per site (home, pricing, blog index, changelog, careers, product), plus robots.txt and the sitemap.
• Once a day, one page at a time, with no parallel requests to the same site.
• Pages are rendered in a headless Chromium. It never logs in, submits forms or solves challenges.

robots.txt

The crawler obeys robots.txt. To exclude it, add:

User-agent: BenchologyCompetitorBot
Disallow: /

A site that refuses it repeatedly (403, 429 or a bot challenge) is retried at most once a week.

How to verify it

Requests carry an HTTP message signature (Web Bot Auth, RFC 9421) with Signature-Agent: "https://competition.seedcast.co". The public key is published at https://competition.seedcast.co/.well-known/http-message-signatures-directory. The user agent ends with BenchologyCompetitorBot/1.0 (+https://competition.seedcast.co/bot) where enabled.

Contact

Operator: Benchology, 251 9th St, San Francisco, CA 94103, United States.

Questions or removal requests: via benchology.co, or block the crawler in robots.txt as above.

Benchology · last updated 2026-10-10