Hi, if you noticed this cute illustration when trying to watch a video:
It’s because I had to add CAPTCHA to Invidious. Oh noooo! A CAPTCHA? Yes, a CAPTCHA.
Why you may ask. The reason is simple: BOTS
Since my Invidious instance is the most used one, it has been abused by all type of bots a lot since it’s really easy for bots to scrape it. Even with the API disabled, anyone can access to the a video without any sort of restrictions and extract information from there.
So, why are bots on Invidious a big deal? Because of Youtube, Youtube has been making third party clients harder and harder to use due to the restrictions they make to prevent bots too!. Youtube blocks anyone that makes a lot of requests to their servers with errors like “Sign in to confirm you are not a bot”, which bots really don’t like to respect, bots hate rate limits and they will bomb your website until they squeeze every bit of your functionality from your website or service until it doesn’t work anymore (Like it happened with my Redlib instance some hours ago before doing this post).
In the last month, I had to block the whole Microsoft ASN from my servers, why? Because there was a lot of traffic coming from their IP addresses, and with a lot, I mean a lot, like ~400Mbps, only used across Microsoft IPs. That is not normal at all. When I blocked it, the traffic went back to normal.
I also blocked a lot of VPS IPs from popular providers like Hetzner, Vultr, Digital Ocean, etc, because, in the first place, most of the traffic that comes from those VPS providers is not going to be legitimate. Only a really small percent of people use them to host their own VPN, and those people contacted me to add their VPS IP address to the whitelist, which I did.
And yesterday my Invidious instance was hit by a massive set of IPs addresses coming from Asia, making a lot of requests to my Instance which made me really mad to be honest, because there is literally no way to block those bots. I could have blocked the whole country? Yes, I could have blocked the whole continent? Yes. But that is not going to solve anything, is just a temporary fix.
I always had the thought of blocking absolutely no one on any of my services, so I always refused myself to add any sort of captcha or aggressive blocking to my instances. But in the end, bad people are going to use public services for their own purposes, they don’t care about us, the people that want to make the internet a better place by providing privacy focused services.
I’m not going to waste my time anymore playing the never ending game of cat and mouse. Having to block IPs manually, having to take a look at the logs to identify suspicious requests when something was not working, adding more aggressive rate limits that could affect VPN users, etc.
I want to offer a stable service to everyone, so I had to choose the only decent solution we have to the bot problem, a JavaScript Proof of work CAPTCHA.
The CAPTCHA that I’m using right now is called Anubis, which has been really popular recently on the open source side of things, since it’s easy to setup, works fine, written on a good language that everyone can understand without too much trouble (Golang), and it’s completely open source, anyone can read the code and audit it.
At least is not reCaptcha, hCaptcha, Cloudflare or mCaptcha (which is unmaintained and it doesn’t work really well)
That’s it. I can finally take a deep breath and focus on something other than having to block bots for 2 hours a day.
(Post written with my bare english skills, no translator used, so if I have grammar errors please tell me if you want to tease me lol)
Q&A
I get the CAPTCHA every time I click on a video!
Enable your cookies…
What about us? People that don’t use Javascript at all?
Sorry, but you will have to enable Javascript for now in order to use Invidious and Redlib now. In my opinion, you are better off using an extension that let’s you enable javascript on specific websites, instead of making the modern internet unusable for you.
Are you going to replace the captcha to a non Javascript one?
Maybe, that is being discussed here: https://github.com/TecharoHQ/anubis/issues/95
What about third party clients?
API is disabled, so third party clients have been broken for a long time already. Is this going to be restored anytime soon? I don’t think so. Enabling the API enables everyone to access video information with no limits which defeats the purpose of why I added the CAPTCHA on the first place.
Do you ban any IPs now?
No, all IPs are unbanned, everyone is now able to access it, even from a VPS IP address.
I have an issue with the captcha, where do I report it?
Reporting it with a message like “The captcha doesn’t work, please fix it!” is not enough information to be able to tell what is the problem exactly, so you are better off talking to me via IRC or Matrix so you can give me more information about it.
If you have any additional questions, feel free to ask on the guestbook!