Ryabitsev: Creepy crawlies

Konstantin Ryabitsev has written a blog post withhard numbers about the impact of AI crawlers on the Linux kernelrepositories at git.kernel.org:

Today, git.kernel.org receives about 6M daily requests demanding to seerandom commits. Of these, 66% are still immediately batted away with the Anubischallenge, but 33% are now solving the math and getting through to the main site- because apparently what we have to offer is worth spending a ton of cycles tocalculate the Anubis challenge.

It's impossible to tell with certainty which of these are bots and which arereal humans - but chances are, if it's asking for an old commit in a random oldfork, it's probably not a real developer trying to do their work.

With a bunch of generous assumptions, legitimate requests are only about 2%of git.kernel.org traffic - everything else are scrapers.