Protect Your Applications with AWS WAF and CrowdSec: Part I

In the first part of this series, I will show you how to install and configure the AWS WAF Remediation Component and use it to protect a simple Nginx web server running behind an Application Load Balancer.

Feel free to jump to the second part of the series if you want to see a more “modern” version of this setup and learn how to protect a serverless application with the AWS WAF and CrowdSec.

Prerequisites

To follow along with this tutorial, I recommend you familiarize yourself with the technologies used:

• Basic understanding of CrowdSec — check out the CrowdSec Academy for a crash course
• Basic knowledge of AWS services — in this tutorial, I will use CloudFront, API Gateway, Lambda, and AWS WAF
• Knowledge of Terraform in order to test the setup

Target infrastructure

AWS WAF is a managed Web Application Firewall offering that allows you to inspect requests and block them (or display a CAPTCHA) based on the criteria you choose. It can inspect requests to:

• API Gateway REST API
• CloudFront distributions
• Application Load Balancer
• AppSync GraphQL API

The CrowdSec Remediation Component (aka bouncer) takes advantage of this to:

• Either block or display a CAPTCHA to IPs or ranges for which CrowdSec has a decision
• Either block or display a CAPTCHA to countries for which CrowdSec has a decision

CrowdSec for Windows will also be able to detect network scans that attempt to get past the Windows firewall.

The diagram below shows our target architecture for this setup:

Installing the Remediation Component

First things first — let’s install the CrowdSec Security Engine and the Remediation Component.

Add the CrowdSec repository to your system. For this article, I am using an Ubuntu 20.04 server.

curl -s https://install.crowdsec.net | sudo sh

If you are using another OS, please refer to the documentation on how to install CrowdSec on your distribution.

Now let’s install the CrowdSec Security Engine:

sudo apt install crowdsec

This command will install the Security Engine and automatically detect supported services running. In this case, I already have an Nginx server installed, so CrowdSec will automatically monitor its logs and enable the Nginx collection.

Finally, install the AWS WAF Remediation Component:

sudo apt install crowdsec-aws-waf-bouncer

This will install the Remediation Component and automatically register it with the local API. You can check that the Remediation Component registered itself properly by running cscli bouncers list.

root@ip-172-31-29-160:~# cscli bouncers list
---------------------------------------------------------------------------------------------------------------------------------------------------------
NAME                IP ADDRESS  VALID  LAST API PULL         TYPE                      VERSION                                                          
---------------------------------------------------------------------------------------------------------------------------------------------------------
 AWS-WAF-1647938971  127.0.0.1   ✔      2022-04-26T09:25:02Z  crowdsec-aws-waf-bouncer  v0.1.3-debian-pragmatic-92d8fc9061f2b1b602ba4d836fcb112c5f11d4fd 
---------------------------------------------------------------------------------------------------------------------------------------------------------
root@ip-172-31-29-160:~#

Configuring AWS WAF

To use the Remediation Component, you must have created a web Access Control List (ACL) in AWS WAF and associated it with one (or more) AWS resources.

In this article, I will associate my ACL with an Application Load Balancer (ALB) proxying traffic to an EC2 instance running a simple Nginx server.

I just need to create a new rule using the AWS console and associate it with our ALB:

As I am adding the ACL to a regional resource — in this case, an ALB — the web ACL must live in the same region.

Configuring the Remediation Component

First, I need to grant the Remediation Component permissions to interact with the AWS WAF APIs.

For this article, the Remediation Component is running on an EC2 instance, so I create an instance role with the following permissions and associate it to the instance:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "VisualEditor0",
      "Effect": "Allow",
      "Action": [
        "wafv2:DeleteIPSet",
        "wafv2:DeleteRuleGroup",
        "wafv2:CreateRuleGroup",
        "wafv2:UpdateWebACL",
        "wafv2:GetIPSet",
        "wafv2:UpdateRuleGroup",
        "wafv2:GetWebACL",
        "wafv2:GetRuleGroup",
        "wafv2:CreateIPSet",
        "wafv2:UpdateIPSet",
    		"wafv2:TagResource"
      ],
      "Resource": [
        "arn:aws:wafv2:*:*:*/webacl/*/*",
        "arn:aws:wafv2:*:*:*/ipset/*/*",
        "arn:aws:wafv2:*:*:*/rulegroup/*/*"
      ]
    },
    {
      "Sid": "VisualEditor1",
      "Effect": "Allow",
      "Action": [
        "wafv2:ListRuleGroups",
        "wafv2:ListWebACLs",
        "wafv2:ListIPSets"
      ],
      "Resource": "*"
    }
  ]
}

Of course, in a real-world deployment, you will want to restrict the permission of the Remediation Component to the ACLs and rule groups it will manage.

If you are not running the Remediation Component on EC2 (or do not want to use instance roles), it also supports authentication with an access key (it will look in $HOME/.aws/ for the default profile or the profile configured in the Remediation Component).

Before starting the Remediation Component, I need to configure it with the name of the ACL I created previously and a few other parameters:

api_key: 926c1f30880dcc691375504e856988bd
api_url: "http://127.0.0.1:8080/"
update_frequency: 10s
daemon: true
log_media: file
log_dir: /var/log/
log_level: info
waf_config:
  - web_acl_name: web-acl-article
  fallback_action: ban
  rule_group_name: crowdsec-rule-group-eu-west-1
  scope: REGIONAL
  region: eu-west-1
  ipset_prefix: crowdsec-ipset-eu-west-1
  

Let me explain a few things in the waf_config section:

• web_acl_name: This is the name of my web ACL, in which the Remediation Component will add its rule group.
• fallback_action: If the Remediation Component receives a decision with an unknown type, use that instead.
• rule_group_name: This is the name of the rule group in which the Remediation Component will add its rules.
• scope: Defines whether I am creating rules for a CloudFront distribution or not.
• region: Shows the AWS region in which everything will be created.

When you’re done editing the configuration, just restart the Remediation Component by running systemctl restart crowdsec-aws-waf-bouncer for it to create a new rule group in your web ACL and the required IP sets.

It may take a minute for the Remediation Component to be fully up and running.

I am also modifying the CrowdSec profiles.yaml to apply a CAPTCHA for all HTTP-related scenarios instead of a classic IP ban.

The profiles.yaml file tells CrowdSec what to do when it receives an alert (i.e., a scenario has been matched enough time for its bucket to overflow).

I add the following at the top of /etc/crowdsec/profiles.yaml:

name: captcha_http_scenarios
filters:
- Alert.Remediation ==	true &&	Alert.GetScenario() startsWith "crowdsecurity/http-"
decisions:
- type: captcha
  duration: 4h
on_success: break
—

This instructs the Security Engine to create a captcha decision valid for four hours if the name of the scenario that triggered the alert starts with crowdsecurity/http-.

Restart (or reload) the Security Engine for the changes to take effect.

Testing the Remediation Component

To test the Remediation Component, I simply run Nikto, a very noisy web scanner, in order to trigger decisions:

nikto -h lb-aws-waf-article-1898138181.eu-west-1.elb.amazonaws.com

The Security Engine will detect the attack and trigger a decision:

root@ip-172-31-29-160:~# cscli decisions list
+---------+----------+------------------+----------------------------+---------+---------+------------------+--------+-------------------+----------+
|   ID    |  SOURCE  |   SCOPE:VALUE    |           REASON           | ACTION  | COUNTRY |        AS        | EVENTS |    EXPIRATION     | ALERT ID |
+---------+----------+------------------+----------------------------+---------+---------+------------------+--------+-------------------+----------+
| 5167576 | crowdsec | Ip:42.42.42.42 | crowdsecurity/http-crawl-non-static | captcha | FR      | 12322 Free SAS |     43 | 3h59m7.904366287s |     1157 |
+---------+----------+------------------+----------------------------+---------+---------+------------------+--------+-------------------+----------+

The Remediation Component will get the decision at its next poll cycle (by default, every 10s) and update the WAF configuration to display a CAPTCHA to the user.

Note: AWS may take some time to propagate the new configuration, but in our experience, it takes about 30 seconds in most cases.

If you solve the CAPTCHA, you will gain access to the website:

The Remediation Component also supports decisions at a country level.

First, let’s delete the current decision: 

root@ip-172-31-29-160:~# cscli decisions delete -i 42.42.42.42

Next, let’s add a ban decision that will apply to all French IPs:

root@ip-172-31-29-160:~# cscli decisions add --scope country --value FR --type ban

If I visit my website from a French IP, I will get a 403:

Summing up

In this article, I showed you how to configure the AWS WAF Remediation Component to protect an application running behind an ALB and demonstrated its ability to block both IPs and countries. You can also use this Remediation Component to protect any application running behind CloudFront, a REST API Gateway, or an AppSync GraphQL API.

In this tutorial, I showed you how to run the Remediation Component on an EC2 instance, but you can also easily run it in a container, for example, with AWS Fargate.

You can find the repository for the AWS Remediation Component here and the dedicated documentation here. 

In part two of this AWS WAF protection series, I will show you how to protect applications in a serverless environment.