Blog :: cgit as an Unintentional Honeypot

On a whim, I recently migrated all of my Git repositories to a cgit instance on WIREDNET. It's caused some headaches because the server-side rendering can be quite resource intensive. Mere minutes after setting it up, the fans on my server spun up because the repos were being scraped by ClaudeBot and friends. I hadn't linked to it from anywhere, so it's likely they found it from LetsEncrypt's issuance logs.¹ It was easy enough to mitigate the scraping by putting a robots.txt up, and I was able to forget about it for a month or so, until I was sitting at work and got an alert from Gatus that several of my services were down. I checked the logs and saw that Apache was being hammered by 47.76.0.0/14 (no name provided) and 47.80.0.0/13 (Alibaba Cloud LLC), which I blocked at the firewall level. This was enough to mitigate the attack, but I was still seeing an unusual amount of traffic in my Apache logs. cgit was, once again, being aggressively scraped. Most of my repositories are small, but I had a mirror of Guix from when I was doing GSOC, and forks of a few other popular repositories, and the crawlers were just going through every single link generated by cgit (basically encompassing the entire source tree for every single commit in the repository). I was reminded of Konstantin Ryabitsev's blog post "Creepy Crawlies", which contains this helpful piece of intuition: "if [an IP is] asking for an old commit in a random old fork, it's probably not a real developer trying to do their work." I don't see any reason that someone would be looking that far into a Guix mirror that's several years out of date, so I wrote a fail2ban rule.

[Definition]

failregex = ^git.jakob.space <HOST> .* "(GET|POST|HEAD|OPTIONS) .*guix\.git.*&id=.*"
ignoreregex =

Results

This was so effective that I hit a rate limit on my AbuseIPDB API key. I identified (and blocked) thousands of scanners in a matter of hours. The IPs were mostly diverse, but there were a few especially problematic providers.

Most registered ASNs will list an email address for abuse complaints in their WHOIS entry. Datacamp Limited, M247 Europe, and Leaseweb USA, Inc. are used by several commercial VPN providers, so contacting them probably isn't worth my time. I tried to let Facebook know that their crawler is violating my robots.txt policy, but their address bounces, so they can go fuck themselves. For all the others that list U.S. addresses in their WHOIS records, I explained that there is behavior consistent across thousands of IP addresses² that is degrading my services and (without my mitigations) causing a denial of service. I've been ignored by all but EGIHosting, who opened a ticket and immediately contacted the downstream (who is also taking my complaint seriously), so they immediately gained my respect.

Alibaba Cloud LLC

My investigation revealed that there were actually two things happening at the time my servers went down, which I'm not sure are necessarily related. The 47.76.0.0/14 and 47.80.0.0/13 ranges didn't appear to be scraping. In fact, they weren't sending valid requests to my web server at all. They show up in my ssl_access_log file triggering request timeouts. A search for those IP ranges in my Zeek logs came up dry, so I'm wondering if it was some sort of SYN flooding attempt? Also, HE says that the ranges aren't visible in the gloabl routing table, so they IPs are probably spoofed. I'll keep the ranges blocked until I figure out a better mitigation.

Why Not Anubis?

Anubis inconveniences legitimate users more than scrapers and its challenges are trivial to solve by dedicated scrapers. I recommend reading this blog post for an investigation on Anubis: "Everything Wrong With Anubis('s default configuration)". And in my case, I have a nearly perfect honeypot that just constantly feeds me bad IPs that I can block and report, so I don't see the need.

Anyway, I think more webmasters hosting cgit should do this: host a repository that no one cares about as a trap, and then you know that any IP trying to access it is a crawler. Then you can block on all your other services and report the IPs to the ISP as abusive. They might not listen. But maybe they will. And if enough people complain, maybe someone, somewhere, will do something about it.

—