Anubis is an open source software program that adds a proof of work challenge to websites before users can access them in order to deter web scraping. It has been adopted mainly by Git forges and free and open-source software projects.[4][5]
| Anubis | |
|---|---|
.png) The page shown by Anubis during its challenge. | |
| Original author | Xe Iaso |
| Developer | Techaro[1] |
| Initial release | January 19, 2025[1][2] |
| Stable release | 1.23.1[3] 
/ 8 November 2025 |
| Repository | github |
| Written in | Go, JavaScript[1] |
| License | MIT License |
| Website | anubis |
Anubis was created by Xe Iaso in response to Amazon's web crawler overloading their Git server, as the crawler did not respect the robots.txt exclusion protocol and would work around restrictions.[4][6] Iaso lists Hashcash as having inspired the project.[7] The application supports inspecting request elements such as headers like the User-Agent header to determine if the request should require proof of work.
The name Anubis is taken from the Ancient Egyptian god of funerals and judgement, who weighs the hearts of the dead to determine if they are allowed passage into the afterlife, whereas the Anubis software "weighs the soul of incoming HTTP requests".[8]
Design
editAnubis temporarily blocks access to websites until the client completes a cryptographic challenge.[9] This challenge is intended to be a design hurdle and incur a compute cost for web crawlers while minimizing impact on typical visitors (see § Criticism). Challenge solutions remain valid for a tunable period of time using EdDSA-signed tokens, which may be stored as HTTP cookies.[1][10]
The challenge itself is similar to the proof of work algorithms implemented by Hashcash and Bitcoin. Anubis presents a challenge to the client in the form of a random number, to which the client must respond with another number (the nonce) such that when the two numbers are concatenated, the SHA-256 hash of both numbers contains a pre-specified number of leading zeros.[9] SHA-256 is chosen specifically because of its (assumed) pre-image resistance (ie. finding such a nonce is computationally expensive) while simultaneously being cheap to verify. Moreover, tuning the number of required leading zeros controls the difficulty of the challenge: a typical browser can find a solution with four leading zeros (the default) in seconds, while one with six can take several minutes.[1]
By default, Anubis challenges any client claiming to be a browser, by checking if the User-Agent header contains "Mozilla", unless the client is requesting "low-harm" content (eg. robots.txt, the well-known URI).[1][10] Addtionally, Anubis exempts Common Crawl "so [other] scrapers have less incentive to scrape".[1]
Although Anubis could be altered to mine cryptocurrency to serve as proof of work, Iaso has rejected this idea: "I don't want to touch cryptocurrency with a 20 foot pole. I realize I'm leaving money on the table by doing this, but I don't want to alienate the kinds of communities I want to protect."[7]
Mascot
edit
The software's loading screen is branded with a commissioned artwork of Anubis as a jackal-eared anime girl by the European artist CELPHASE.[1][8] The mascot is depicted with a hoodie, skirt and magnifying glass. Before the artwork was ordered, Anubis used an AI-generated placeholder image.[1]
The Anubis mascot is shown to all end users and cannot be altered in the software configuration.[1] The image's feel may clash with websites that have more formal atmospheres, surprising or confusing users of those sites.[8] Altering the branding is an enterprise feature and Iaso has requested that operators not attempt to change it themselves unless they have made financial contributions to the project.[1]
Duke University, which has deployed Anubis for its digital archives, was "hesitant" to use it due to the mascot but has reached an agreement to use the software with custom branding.[1] Jamie Zawinski describes the mascot as "cutesey kawaii bullshit".[7]
Adoption
editIt has been used by a number of projects, including:[11]
Criticism
editTavis Ormandy noted that the cost of scraping websites protected by Anubis is likely negligible: "I don’t think we reach a single cent per month in compute costs until several million sites have deployed Anubis."[16]
References
edit- ^ a b c d e f g h i j k l m n o Brockmeier, Joe (July 10, 2025). "Anubis sends AI scraperbots to a well-deserved fate". LWN.net.
- ^ Iaso, Xe (January 19, 2025). "Block AI scrapers with Anubis". xeiaso.net. Retrieved May 2, 2025.
- ^ "Release 1.23.1". November 8, 2025. Retrieved November 11, 2025.
- ^ a b c Edwards, Benj (March 25, 2025). "Open source devs say AI crawlers dominate traffic, forcing blocks on entire countries". Ars Technica. Retrieved May 2, 2025.
- ^ Bort, Julie (March 27, 2025). "Open source devs are fighting AI crawlers with cleverness and vengeance". TechCrunch. Retrieved May 2, 2025.
- ^ a b c Maiberg, Emanuel (July 7, 2025). "The Open-Source Software Saving the Internet From AI Bot Scrapers". 404 Media. Retrieved July 14, 2025.
- ^ a b c Proven, Liam (July 10, 2025). "Anubis guards gates against hordes of LLM bot crawlers". The Register.
- ^ a b c d "一部のウェブサイトで一瞬だけ表示される「ケモ耳少女のイラスト」は一体何者なのか？" [What in the world is that 'kemonomimi girl artwork' that appears for a moment on some websites?]. Gigazine (in Japanese). August 23, 2025. Retrieved January 9, 2026.
- ^ a b Iaso, Xe (March 20, 2025). "Why does Anubis use Proof-of-Work?". Retrieved January 9, 2026.
- ^ a b Iaso, Xe (March 20, 2025). "How Anubis works". Retrieved January 9, 2026.
- ^ a b "List of known websites using Anubis". anubis.techaro.lol. Retrieved May 10, 2025.
- ^ "Anubis". Techaro. Retrieved July 28, 2025.
- ^ a b c Iaso, Xe (April 12, 2025). "Anubis works". xeiaso.net. Retrieved May 9, 2025.
- ^ "Forum and Wiki Performance; Anubis Deployed". FreeCAD News. April 30, 2025. Retrieved May 11, 2025.
- ^ "The Day Anubis Saved Our Websites From a DDoS Attack". fabulous.systems. May 1, 2025. Retrieved June 19, 2025.
- ^ Ormandy, Tavis. "Anubis". lock.cmpxchg8b.com. Retrieved October 2, 2025.
