How to write CrowdSec parsers & scenarios - the Asterisk VoIP use case
Tutorial: how to write a CrowdSec parser to process Asterisk logs and write a scenario to detect common attacks (user enumeration, brute force ..) in Asterisk.
Introduction
In this tutorial, we are going to see how we can write a CrowdSec parser to process Asterisk logs and then how to write a CrowdSec scenario to detect common attacks (user enumeration, brute force ...) on this service.
Requirements
In order to write the CrowdSec parser and scenario, we will need the following:
- Some samples of Asterisk logs (authentication failed because of invalid username, bad password …)
- A CrowdSec instance running (possibly not on a production server)
- Some knowledge about Grok patterns
- Some knowledge about the YAML file format
Setting up the environment
To set up the environment development there are two options:
- Install CrowdSec on your laptop or on a non-production server from the repositories
https://docs.crowdsec.net/docs/getting_started/install_crowdsec/
- Create the test environment from the tarball package
https://docs.crowdsec.net/docs/contributing/contributing_test_env/
For this tutorial, we are going to use the first option.
Be sure that the crowdsecurity/linux is installed (and more precisely, the crowdsecurity/syslog-logs parser, all this can be seen with cscli hub list) because as the first "parser" in the chain, it will direct the logs to the right parser - in our case asterisk, but more on this later.
Writing the parser
Here are the Asterisk logs that we want to parse (to detect brute force and user enumeration): login failed because of an incorrect username and because of an incorrect password.
About Grok patterns, you can use this website to debug your grok pattern.
Tip: A lot of basic patterns are provided by CrowdSec, don’t reinvent the wheel :)
Invalid username
Log sample
Sample of login failed because of an incorrect username:
Grok pattern
In this log, we can see the “InvalidAccountID” in the Security Event field which means that the user “netadmin” (in the AccountID field) doesn’t exist. Here we are mostly interested in capturing the targeted username, the source IP address and port, and the timestamp of the event. We also capture the asterisk session ID and the targeted IP address in case we need them for the next scenarios.
So here is the grok corresponding to this line:
Note: the NOTDQUOTE grok pattern is embedded in CrowdSec patterns and means “everything except a double quote”
Which outputs the following fields:
Invalid password
Log sample
Sample of login failed because of invalid password:
Grok pattern
In this log line we can see the “ChallengeResponseFailed”, which means that the user `6001` (in the AccountID field) exists but the password was incorrect. Here we are mostly interested in capturing the targeted username, the source IP address and port, and the timestamp of the event. We also capture the asterisk session ID and the targeted IP address in case we need them for the next scenarios.
So here is the grok for this log line (for invalid password):
Note: the NOTDQUOTE grok pattern is embedded with CrowdSec patterns and means “everything except a double quote”
Which outputs the following fields:
Writing the parser
Now that we have our two interesting log samples and their associated grok, we can start to write the parser.
Let’s start with the beginning and the easiest part.
Here we specify the name of the parser (<AUTHOR>/<NAME>), a short description, a filter (we want to parse only logs where the program is asterisk) and the behavior where the log parsing is successful (here, next_stage means that this parser is enough for this log line, and it can move directly to the next stage: enrichment)
Now we are going to define two nodes in our parser, one for the failed authentication because of the invalid username and another one for the invalid password.
So here we define the two nodes, where we say “apply this grok on this field”, where message is always the log line of the service (without the syslog header in case of a syslog log).
Now we need to set some statics, mostly to define what parsed fields we want to keep track of: fields that are in the Meta dictionary of the evt object are going to be kept in the final alerts, while others are discarded.
Note: the evt.Parsed object contains all the fields that you captured with the grok and can be used in the statics to populate the evt.Meta object
So here is what is set with the statics:
- in evt.Meta.log_type, we set the value “asterisk_failed_auth”: this will be used in scenarios to trigger relevant events
- in evt.StrTime directly (notice the target key instead of meta) we set the timestamp of the event (this is mostly used when running CrowdSec in replay mode)
- in evt.Meta.target_user we set the username that we captured with our Grok (notice the expression key, which evaluates the content of the given object)
- in evt.Meta.session_id we set the captured session ID
- in evt.Meta.asterisk_service we set the captured service
And now to finish the parser, we can apply global statics (they will be applied whenever a node of the parser succeeds)
Here is what is set with the statics:
- in evt.Meta.service, we set the value “asterisk”
- in evt.Meta.source_ip we set the IP address that we have captured in our grok (the remote IP, not the local one)
We can use those 2 statics as global because each grok must capture at least an IP address (to allow CrowdSec to block it later) and if one of the nodes match, it means that the service of the log is asterisk.
We could also have put the static about the username and the log_type in the global static, but we want to be able to add more nodes in the future that will not be about a failed authentication or will not capture a username (log_type can be something else than a failed authentication and we might not be able to capture a username in other logs, whereas the source_ip is always needed to block attacks on this service).
This is what our final parser looks like:
Test the parser
Now that we have our parser, we can put it in /etc/crowdsec/parsers/s01-parse/asterisk-logs.yaml
We can write our two log lines in a file called asterisk.log, and run CrowdSec this way:
This will produce the following output (if everything went well):
Note: The green/red lights indicate if a line was picked up by a given parser. The +X ~Y and -Z in brackets indicate a summary of the changes made by a given parser and adding -v would display individual changes made by parsers.
Writing the scenario
Note: Since we are using a private IP address in our logs, don’t forget to remove the crowdsecurity/whitelists parser (sudo cscli parsers remove crowdsecurity/whitelists), or else your new scenario won’t catch anything.
Log sample
Now that we have a working parser, we need more logs to detect a user enumeration or a brute force of password:
Writing the scenario
To detect user enumeration and brute force attempts, we are going to create two different scenarios with the following name:
- crowdsecurity/asterisk_user_enum
- crowdsecurity/asterisk_bf
The naming convention for scenarios is “<author>/<scenario_name>”.
Before writing the scenario, you can view all the available fields in the evt Object (to use them later in the scenario filter, group by and distinct key) by running sudo cscli explain --file asterisk.log --type asterisk -v.
Here is the output example for one line (for the invalid username log line):
Asterisk brute force
Now that we have our parser we can write our scenarios.
Let’s start with the basics:
Here we specify:
- the type of scenario (leaky or trigger), more here
- the name of the scenario
- a short description of the scenario
Now we can specify on what type of event we want to match:
And then group the scenarios by IP addresses:
So for now, we want to match on events with a log_type equal to ‘asterisk_failed_auth’ and group by IP address.
Now we need to define the capacity of the scenario (how many events should match before the scenario is triggered) and the leak speed (this is the rate for an event to be leaked from the leaky bucket, more here) and a black hole (a duration for which a scenario will be "silenced" after being triggered (for the same IP address). This is intended to limit/avoid spam of scenarios that might be very rapidly triggered.):
Here we say that if an IP address is doing more than 5 failed authentication in less than 20 seconds, the scenario will be triggered and will then be silent for 1m (only for the same IP).
And we can add some labels to our scenario:
So this how the entire scenario looks like:
Asterisk user enumeration
Now, let’s add one more trick: we want to detect a single IP performing failed authentication on many different users. This is what is commonly referred to as user enumeration (and might be seen in the case of credential stuffing for example).
This scenario is quite similar to the brute force scenario, except that we want to have a distinct clause on the username:
Note: The distinct clause ensures that the event will only “enter” the bucket if no event with this value is already stored in the bucket. An attacker hammering the same username won’t trigger the scenario.
Test the scenario
To test the scenario we need to create the scenarios in the right folder.
Create /etc/crowdsec/scenarios/asterisk_user_enum.yaml and /etc/crowdsec/scenarios/asterisk_bf.yaml and copy respectively the configurations that we made before.
Paste the logs from the brute force into a file called asteris_bf.log and run sudo cscli explain --file asterisk_bf.log --type asterisk:
This screenshot doesn’t show all the logs, but we can see that they match the wanted behaviors.
If your log is not triggered by your scenario, you can run the cscli command with a -v flag to know which fields are parsed and why it doesn’t match with your scenario:
sudo cscli explain --file asterisk_bf.log --type asterisk -v:
Closing words
We hope this article helped those of you that might struggle with parser and scenario creation. Of course, this parser and the scenarios are going to end up in the hub, and this article is purely for educational purposes.
Happy hunting!
.png)
