CrowdSec – not your typical Fail2Ban clone

Fail2Ban is great!

While it’s not wrong that CrowdSec can be used as an alternative to Fail2Ban for ssh brute force protection, it’s not the entire truth either. Don’t get me wrong; Fail2Ban is a fantastic tool to enable. Without Fail2Ban there wouldn’t have been any CrowdSec either. The majority of CrowdSec’s founders used to have a successful business in secure hosting where Fail2Ban played a paramount role in keeping infrastructure secure. In other words: We have a profound respect for Fail2Ban and we recognize that it has been a great help to millions of users all over the world for the last 18 years.

That being said, that is exactly the issue with Fail2Ban. It was created back in 2004 when the world looked very different. IT infrastructure was simpler back then and we weren’t amidst a never-ending ransomware pandemic. In other words: Threats were a lot different; attacks were way less sophisticated and cybercrime wasn’t as professionalized as it is today.

Things have changed and so has the need for protecting against cyber attacks. To put it very simply, Fail2Ban was meant to protect against brute force attacks. That’s it. Brute force attacks are relevant then as they are now, so nothing’s wrong with that. CrowdSec can protect against brute force attacks too so no wonder if you think that’s all. Especially since a large part of our marketing strategy in the beginning revolved around comparing CrowdSec with – you guessed it – Fail2Ban. As it turns out, that strategy has a downside in that there is a risk that users might end up thinking that’s it. That there’s nothing else to come for with CrowdSec.

Nothing could be further from the truth.

Distributed architecture

CrowdSec was designed for a distributed reality where the perimeter is everywhere, spread across multiple on-prem and cloud environments. A reality where one has to act fast to block attacks and where automation is key. CrowdSec’s architecture is API-driven, meaning that all elements of the CrowdSec stack; agent, bouncer, cscli, lapi (local api), alerting, capi (central api) can run distributed.

You can have more bouncers; one at every perimeter you may have in your infrastructure or you can even have more agents which collect all your logs, parse it, detect attacks and send them to one master agent which sends out alerts, reports, and communicates with the CrowdSec console. In other words: Unlike Fail2Ban CrowdSec gives you the possibility to split up detection and remediation; detect on one server – remediate on one or more others.

Obviously, this is most useful for a company that uses CrowdSec to protect its large, global infrastructure. So if you’re a self-hoster who just likes to have a secure home network and uses CrowdSec for that, no problem – you just won’t discover the ability to have a distributed architecture.

Collaboration is key

CrowdSec is also meant for a reality where collaboration is key; the number of cyber attacks has skyrocketed these past few years and nothing suggests that will change in the near future. There’s simply too much easy money to earn. So cybercriminals have professionalized digital extortion – what is now known as ransomware. In the beginning, one needed a number of skills to start in this business but as time has passed, the mindset of cloud computing has shown us that also crime can be sold as a service. Basically, all the bits and pieces needed to be a cybercriminal can be bought from professional enterprises with 24/7/365 support, SLAs, and everything else you know for legit businesses. If you throw huge profit earnings in the mix for everybody (but the victim, of course) you have a business that thrives.

The point is that we need other measures to protect against cybercriminals than what we’ve seen so far. We need to look differently at the challenge at hand and realize that this is not one of those challenges that can be solved just by throwing money at it. If that was the case, an incident like J.P. Morgan’s data breach in 2014 would never have happened since at that time J.P. Morgan had a yearly cybersecurity budget of 800M$. Almost 1B$! Yet it obviously wasn’t enough. Could it be that the approach is wrong? That we simply need to work together in a more effective manner to prevent attacks?

At CrowdSec we firmly believe that this is the case; who in their right mind fights a beehive? Here the beehive is all CrowdSec users who share information about the attacks they see, send it to our consensus engine which filters out false positives and distributes the list of verified attacks to all users who then has the possibility to protect themselves from attacks that haven’t even occurred yet. In a recent blog post we concluded that 92% of attacks on two identical servers (one protected by the CrowdSec firewall bouncer, one not; both had the CrowdSec agent installed) were from actors already known by the community; hence already blocked on the server that enjoyed CrowdSec protection.

Clearly, this is just one example and a very narrow use case. Also, have in mind that CrowdSec is still a young product. At the time of writing, we have around 40k+ agents in our network. We collect close to 1M signals on a daily basis and we have around 2.3M IPs in our un-curated, ’smoke’ database and around 21-23k IPs in the verified ‘fire’ database. Imagine the numbers when we have reached our goal of 1M agents in 2024. Wow.

We are constantly looking for new ways to shut down the cybercriminals; one path we’re also working on is to distribute our blocklist to BGP routers that are usually used to route traffic at the internet core level meaning that it would be possible to prevent the IPs that are deemed malevolent by CrowdSec from being reachable by anyone.

And this is exactly what we want; the ultimate goal of CrowdSec is to put cybercriminals out of business. So by being a part of CrowdSec you’re helping out in a good cause; basically, you’re part of a modern cyber- neighborhood watch for the collective good. That’s something to be proud of – especially when the attacks that your CrowdSec agent has collected prevent real attacks.

CrowdSec detects deviations of any kind

The real beauty of CrowdSec is what it really is; an advanced framework for detecting deviations of any kind. Literally any kind. Due to the flexible nature of CrowdSec and the plugin-like architecture where any data sources can be connected, CrowdSec can literally assess any kind of data; once input has been digested by CrowdSec it’s ‘just’ an object that is assessed depending on which scenarios are installed.

In short, if you can describe a situation you want to investigate in YAML, CrowdSec can detect it in the data. The right data source, parser, or scenario might not be available yet but we are constantly updating the list of available integrations available at https://hub.crowdsec.net/. And given that CrowdSec is FOSS, feel free to contribute your own parser, scenario, or even a bouncer like our community member Fabien did.

Functional flexibility means that CrowdSec can detect a number of different incidents both on L3 and L7 of the OSI model: the network and the application layers. It does so by looking at the log files that are being generated, parsing them, and based on the currently installed scenarios, it detects those incidents.

Examples of attacks CrowdSec is protecting from:

• Bot scraping: Bots scraping your website for data. This consumes resources and might be a business problem if you live off those data

• Bot scalping: Scalping is when you buy goods only for the purpose of reselling them at a higher price. If you’re an event organizer you don’t want popular tickets to concerts to be sold in no time to bots, only to be resold for 10x the price.

• Data exfiltration (PII, credit card information): Data theft is a massive problem, not only because of GDPR but because ordinary people and banks lose money when credit card information is stolen and abused.

• L7 DDoS attacks: There are many kinds of DDoS attacks that exist on either the network- or application layer. CrowdSec can help mitigate those on L7 where bots usually enter a website and exhaust resources of various kinds. A DDoS attack is defined by the scale; thousands or even hundreds of thousands of bots (or users) enter the website simultaneously effectively forcing it to its knees.

• Credit card/credential stuffing: When credit card information/credentials are sold in bulk by hackers, the buyer wants to know if they work. So they test credit cards by purchasing at scale in an automated manner. The same thing goes for credentials; the same IP (usually) cycles through them at scale. This is quite easy for CrowdSec to detect because it’s so far from ordinary user behavior (one IP creates numerous HTTP connections vs just one or two when it’s a normal user interacting on a website).

In all these cases CrowdSec can make a real difference: detect attacks and mitigate them either by blocking the perpetrator using a firewall to filter the bad guys out or forcing them through a CAPTCHA challenge to prove they’re not a bot (which they most likely are since they were blocked in the first place).

However, it does make sense to not just block connections when they’re bad for a couple of reasons. First of all, for an e-commerce website blocking users can cost money which is always bad. Another reason is that a block can cause collateral damage if NAT is involved somehow and you happen to end up sharing the WAN IP of an attacker. In both cases, the non-bot user has the possibility to access the protected service regardless of using a malevolent IP or not – and should CrowdSec end up with a false positive it’s a matter of issuing a simple command to unban the IP and allow access again if you don’t want to wait 4 hours, which is the default ban time of CrowdSec. To compare, the default ban time in Fail2Ban is only 10 minutes.

Getting started

How to get started with CrowdSec is fairly easy depending on your platform. Instructions can be found on our documentation website. CrowdSec supports a large range of operating systems and platforms:

• Linux: Most major distros such as Debian, Ubuntu, CentOS and others, including Armbian and Raspberry Pi OS for ARM processors‍
• FreeBSD‍
• Windows‍
• OpenWRT‍
• Synology‍
• OPNsense‍
• Docker‍
• K8s (including k3s)
• ‍Helm‍
• Cloud computing (For the time being just AWS)

If you’re into automation, CrowdSec also supports Ansible, Puppet, Chef and others.

To get a better understanding of what CrowdSec can do there is a number of articles available, both by us and community members here, here, and here just to name a few. Also, there is a number of videos on YouTube available made both by us and community members.

Join our community

Follow this conversation on the value of investing in the security knowledge sharing economy. This profound yet fun discussion explores the power of a cybersecurity community information sharing platform as a means to not only secure your business but the neighbors around you. By collaborating and sharing information about cyber threats, we can help secure whole communities, society, and, therefore, humanity.

‍