Aug 27, 2025

Strengthen Security and Protection with CrowdSec’s Open Source Web Application Firewall

Web applications are a prime target for attackers, and the threat is only growing, as the Verizon DataBreach Incident Report highlighted. But what if you could block over 75% of malicious traffic before it even reaches your server, with just a few commands?

In this article, we’ll show you how to use the CrowdSec WAF to protect your web applications from malicious IPs scanning the internet to exploit vulnerabilities.

Why choose CrowdSec WAF over other WAFs?

Now you may be wondering why I should use the CrowdSec WAF versus, let’s say, ModSecurity? Both are open source web application firewalls that allow you to gain visibility into HTTP(S) traffic.

For the tutorial, we will set up a reverse proxy (Nginx) boosted with CrowdSec in front of our web server (Apache) to block malicious traffic before it reaches our application.

This article dives into the technical details of configuring CrowdSec WAF, showcasing performance metrics and practical examples to demonstrate its effectiveness.

To achieve robust protection, we’ll use two key components that work in tandem: the Security Engine and the Web Application Firewall (WAF) – enabled by an AppSec-capable Remediation Component, also known as a Bouncer, in our case, CrowdSec’s NGINX Bouncer.

The Security Engine: excels at identifying persistent or recurring behaviors. It analyzes your web server/reverse proxy logs to identify suspicious patterns of behavior. For example, the http-probing scenario detects IPs rapidly requesting a large number of non-existent files – a common tactic used by vulnerability scanners searching known vulnerabilities, backdoors, or publicly exposed admin interfaces. While powerful and able to protect a large number of services from various log sources, the Security Engine reacts after the suspicious event, once it’s logged by your web server.

The Web Application Firewall (WAF): The WAF acts as your immediate gatekeeper, blocking malicious requests before they even reach your application or backend. With the help of the bouncer relaying the requests to the AppSec engine, it will apply virtual patching rules to block requests that are, without a doubt, malevolent. A great example is the vpatch-env-access rule, which blocks requests attempting to access .env files (which should never be publicly accessible!). Our vpatching collection has hundreds of rules tailored to block vulnerability attempts precisely, limiting the risk of false positives.

Together, these components provide layered protection, making it significantly harder for attackers to succeed.

WAFs are powerful, but no matter what WAF vendors make you believe, determined attackers can often find ways to bypass your WAF configuration. Here, the Security Engine will rely on the WAF detection to make longer-term decisions against repeating malicious IPs. This is what the appsec-vpatch scenario does: it bans IPs that trigger at least two distinct WAF rules for several hours.

Initial Setup

For our experiment, we’ll set up two Ubuntu 24.04 servers. One will be creatively called webserver, and the other will be called nginx-RP. We’ll install apache on webserver, configured to listen on port 3000, and nginx on nginx-RP.

We’ll deploy the following configuration for nginx as a reverse proxy.

/etc/nginx/sites-enabled/default

 server {
    listen 80;

    server_name _;

    location / {
        proxy_pass http://X.X.X.X:3000;  # Your backend app
    	 proxy_set_header X-Real-IP $remote_addr;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }
}

And the following Apache config:

/etc/apache2/sites-enabled/000-default.conf

 
	# The ServerName directive sets the request scheme, hostname and port that
	# the server uses to identify itself. This is used when creating
	# redirection URLs. In the context of virtual hosts, the ServerName
	# specifies what hostname must appear in the request's Host: header to
	# match this virtual host. For the default virtual host (this file) this
	# value is not decisive as it is used as a last resort host regardless.
	# However, you must set it for any further virtual host explicitly.
	#ServerName www.example.com

	ServerAdmin webmaster@localhost
	DocumentRoot /var/www/html

	RemoteIPHeader X-Real-IP
	RemoteIPTrustedProxy X.X.X.X

	# Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
	# error, crit, alert, emerg.
	# It is also possible to configure the loglevel for particular
	# modules, e.g.
	#LogLevel info ssl:warn

	ErrorLog ${APACHE_LOG_DIR}/error.log
	CustomLog ${APACHE_LOG_DIR}/access.log combined

	# For most configuration files from conf-available/, which are
	# enabled or disabled at a global level, it is possible to
	# include a line for only one particular virtual host. For example the
	# following line enables the CGI configuration for this host only
	# after it has been globally disabled with "a2disconf".
	#Include conf-available/serve-cgi-bin.conf

💡The two relevant parts here are VirtualHost *:3000 to make Apache listen on port 3000, and RemoteIPHeader + RemoteIPTrustedProxy to ensure our logs contain the real IPs and not only the IP of the reverse proxy server.

So, if we now access the public IP of our reverse proxy, we’ll see Apache’s default page.

We do have our Nginx logs:

 ==> /var/log/nginx/access.log <==
X.X.X.X - - [22/May/2025:08:32:49 +0000] "GET / HTTP/1.1" 200 3121 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/136.0.0.0 Safari/537.36"
X.X.X.X - - [22/May/2025:08:32:49 +0000] "GET /icons/ubuntu-logo.png HTTP/1.1" 200 3322 "http://X.X.X.X/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/136.0.0.0 Safari/537.36"
X.X.X.X - - [22/May/2025:08:32:50 +0000] "GET /favicon.ico HTTP/1.1" 404 245 "http://X.X.X.X/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/136.0.0.0 Safari/537.36"

And the Apache logs:

 ==> /var/log/apache2/access.log <==
X.X.X.X - - [22/May/2025:08:32:49 +0000] "GET / HTTP/1.1" 200 3404 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/136.0.0.0 Safari/537.36"
X.X.X.X - - [22/May/2025:08:32:49 +0000] "GET /icons/ubuntu-logo.png HTTP/1.1" 200 3552 "http://X.X.X.X/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/136.0.0.0 Safari/537.36"
X.X.X.X - - [22/May/2025:08:32:50 +0000] "GET /favicon.ico HTTP/1.1" 404 438 "http://X.X.X.X/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/136.0.0.0 Safari/537.36"

Welcome to the internet

Definitely not because I forgot about my ongoing experiment, but here we are a day later, checking back on our setup! Within 24 hours, our Web Server saw 630 HTTP Requests from 112 distinct IPs. But, this must be good, right?

We can use CrowdSec IPDex (shameless plug) to check if those IPs are known favorably or unfavorably:

Shockingly, out of the 112 IPs, 92% of them are already known by CrowdSec, 45% of them are already part of CrowdSec’s Community Blocklist, and their intent is clear: 90% are coming after your web server, and it’s not to help you rank on Google. 🙂

Time to beef up our security with the Security Engine

As we’ve just seen, as soon as our server is online, hordes of malicious IPs will jump on it with clearly bad intentions. What is currently happening is this:

Thus, it is time to step up our security with CrowdSec. We will deploy the Security Engine, the WAF, and an Nginx bouncer on our Reverse Proxy server so that we can achieve this:

To install CrowdSec on our reverse proxy, let’s grab the crowdsec repository:

 $ curl -s https://install.crowdsec.net | sudo sh

And let’s install CrowdSec:

 # apt install crowdsec

The relevant part of the install log is the following:

 Creating /etc/crowdsec/acquis.yaml
INFO[2025-05-22 09:40:27] crowdsec_wizard: service 'nginx': /var/log/nginx/error.log /var/log/nginx/access.log
INFO[2025-05-22 09:40:27] crowdsec_wizard: service 'ssh': /var/log/auth.log
INFO[2025-05-22 09:40:27] crowdsec_wizard: service 'linux': /var/log/syslog /var/log/kern.log

It detected we’re running Nginx, and will automatically install the relevant scenarios and start monitoring the associated logfiles!

Then, we will enroll the Security Engine in the CrowdSec Console:

Accept it in the Console:

Detecting is cool, blocking is better.

To complete our setup, we need the ability to block bad IPs and requests before they reach Apache. We will install the Nginx bouncer (or Remediation Component) for this. The Remediation Component (also known as bouncer) can block IPs when instructed by CrowdSec. As simple as this:

 # apt install crowdsec-nginx-bouncer

What matters in the installation output is that:

 cscli is /usr/bin/cscli
cscli/crowdsec is present, generating API key
API Key : 
Restart nginx to enable the crowdsec bouncer : sudo systemctl restart nginx

The Remediation Component installation detects a running CrowdSec on the same machine, and in this case, it will self-configure.

Testing 🙂

From a 3rd-party machine, let’s try to trigger our newly deployed CrowdSec. What we’re going to do here is try to access some well-known backdoors. The crowdsecurity/http-backdoors-attempt should have our back, and will ban any IP trying to access more than one backdoor in a short period:

 $ for i in "b37.php" "1337.php" "AK-74.php" ; do curl X.X.X.X/${i}  ; done

On our reverse-proxy logs, we can see in CrowdSec and Nginx:

 ==> /var/log/nginx/access.log <==
X.X.X.X - - [23/May/2025:07:12:25 +0000] "GET /b37.php HTTP/1.1" 404 277 "-" "curl/8.5.0"
X.X.X.X - - [23/May/2025:07:12:25 +0000] "GET /1337.php HTTP/1.1" 404 277 "-" "curl/8.5.0"

==> /var/log/crowdsec.log <==
time="2025-05-23T07:12:25Z" level=info msg="Ip X.X.X.X performed 'crowdsecurity/http-backdoors-attempts' (2 events over 63.14369ms) at 2025-05-23 07:12:25.309969591 +0000 UTC"

==> /var/log/nginx/access.log <==
X.X.X.X - - [23/May/2025:07:12:25 +0000] "GET /AK-74.php HTTP/1.1" 404 277 "-" "curl/8.5.0"

==> /var/log/crowdsec.log <==
time="2025-05-23T07:12:25Z" level=info msg="(ec2fbc52890f61faf47a24f8edffa3d3V1V7BFIbLCGxqPIO/crowdsec) crowdsecurity/http-backdoors-attempts by ip X.X.X.X (FR/5410) : 4h ban on Ip X.X.X.X"

We can see, after trying to reach out for two backdoors, our attacker’s IP gets banned for 4 hours (crowdsecurity/http-backdoors-attempts by ip X.X.X.X (FR/5410) : 4h ban on Ip X.X.X.X).

So now, if we try to reach out to the web server again:

 $ curl -I X.X.X.X/foobar 
HTTP/1.1 403 Forbidden
Server: nginx/1.24.0 (Ubuntu)
Date: Fri, 23 May 2025 07:39:16 GMT
Content-Type: text/html
Connection: keep-alive
cache-control: no-cache

More importantly, this request doesn’t reach the actual web server and gets stopped at the reverse-proxy level:

 # grep foobar /var/log/apache2/*.log
#

Going further: Web Application Firewall

However, this approach has a limit: CrowdSec reads logs and acts based on their content, which means that you somehow react to an attack that has already happened. We want to intercept malicious requests “on the fly” so that they never reach Apache. This is the job of the WAF:

Let’s follow the guide found here – https://doc.crowdsec.net/docs/next/appsec/quickstart/nginxopenresty :

We install the appsec collection. They contain the WAF rules

 $ sudo cscli collections install crowdsecurity/appsec-virtual-patching crowdsecurity/appsec-generic-rules

2. We enable the AppSec/WAF acquisition, which allows CrowdSec to expose a service to which Nginx can post validation requests.

 # cat > /etc/crowdsec/acquis.d/appsec.yaml << EOF
appsec_config: crowdsecurity/appsec-default
labels:
  type: appsec
listen_addr: 127.0.0.1:7422
source: appsec
EOF

3. Next, we restart CrowdSec

 # systemctl restart crowdsec

4. Now, we instruct our Nginx Remediation Component to rely on CrowdSec for the WAF feature:

 # cat >> /etc/crowdsec/bouncers/crowdsec-nginx-bouncer.conf << EOF
APPSEC_URL=http://127.0.0.1:7422
EOF

5. Finally, we restart Nginx

 # systemctl restart nginx

Testing the WAF

So now, we can try to trigger the WAF:

 $ curl -I X.X.X.X/.env
HTTP/1.1 403 Forbidden
Server: nginx/1.24.0 (Ubuntu)
Date: Fri, 23 May 2025 12:18:24 GMT
Content-Type: text/html
Connection: keep-alive
cache-control: no-cache

And indeed in CrowdSec’s logs, we see:

 ==> /var/log/crowdsec.log <==
time="2025-05-23T12:18:24Z" level=info msg="AppSec block: crowdsecurity/vpatch-env-access from X.X.X.X (127.0.0.1)"

==> /var/log/nginx/error.log <==
2025/05/23 12:18:24 [alert] 25451#25451: *443 [lua] crowdsec.lua:637: Allow(): [Crowdsec] denied 'X.X.X.X' with 'ban' (by appsec), client: X.X.X.X, server: _, request: "HEAD /.env HTTP/1.1", host: "X.X.X.X"

==> /var/log/nginx/access.log <==
X.X.X.X - - [23/May/2025:12:18:24 +0000] "HEAD /.env HTTP/1.1" 403 0 "-" "curl/8.5.0"

28^H^H10 days later: Measuring efficiency

Ten days later, let’s check our web server and see if CrowdSec has been doing its job!

First of all, let’s log in to our Console account (https://app.crowdsec.net) and confirm that our reverse proxy seems to do its job at detecting and blocking attacks:

In terms of its ability to block incoming requests, we can look at the difference in processed requests by our Web Server versus our Reverse Proxy:

💡See the spike on the 5th of June? A handful of malicious IPs intensively scanned our website. Luckily, the CrowdSec WAF and blocklist stopped all of the nefarious activities, hence this spike is only visible on the reverse proxy, but not on the web server itself.

In blue, we can see the number of requests processed by the Reverse Proxy (Nginx) and, in red, the number of requests that make it to our backend web server (Apache). We can view this in percentage, and our reverse proxy effectively stops around 75% of requests in normal usage.

Where are these blocks coming from? Again, we can head to the CrowdSec Console, in the “Remediation Metrics” component, to understand the origin. We can see that ~90% of the requests were blocked by the WAF and/or Security Engine, while ~10% were blocked by the Community Blocklist itself:

Wrapping this up

By setting up our reverse proxy with the CrowdSec WAF and Security Engine in front of our application, we are blocking a massive amount of requests before they even reach our application. These blocks come from various sources: The Security Engine detects and blocks behaviors, banning offending IP addresses for an extended period to avoid them consuming our resources and limiting the chances they manage to breach our application. The WAF stops malicious requests “on the fly” before they can reach our backend web server, and the community blocklist completes this by preemptively blocking malicious IPs.