Can you explain how this fix works?

As you can see, im not an active user of the 4chan crawler, but appriciate work on any module.

Basically when you add a referrer to the browser's request URL to 4chan, it sends your host to 4chan and it can detect you crawling and sends a captcha. So the code removes the referrer of your host being sent to 4chan and then you don't get a captcha.

However there is a host header still sent to cloudflare which is necessary. But only sends i.4cdn.org

https://developer.mozilla.org/en-US/docs/Web/HTML/Element/img#attr-referrerpolicy

As an example: If I ran this code at crawler.net
Crawler.net would be sent in the GET header a bunch throwing the captcha.
Same for 192.168.0.100:5000

4chan doesnt like that many requests from the same referrer (and sending a referrer implies crawling as well) However, I'm not sure how their page loading works on their end and probably turn off referrers in their own code or have some sort of workaround. (Or from 4chan directly their own referrer would be 4chan.org)