Acquisition

Info

Please note that the /etc/crowdsec/config/acquis.yaml should be auto generated by the wizard in most case.

The acquisition configuration specifies lists of files to monitor and associated "labels".

The type label is mandatory as it's later used in the process to determine which parser(s) can handle lines coming from this source.

Acquisition can be found in /etc/crowdsec/config/acquis.yaml, for example :

Acquisition example

filenames:
  - /var/log/nginx/access-*.log
  - /var/log/nginx/error.log
labels:
  type: nginx
---
filenames:
  - /var/log/auth.log
labels:
  type: syslog

Testing and viewing acquisition

At startup

At startup, you will see the monitored files in /var/log/crowdsec.log :

...
time="30-04-2020 08:57:25" level=info msg="Opening file '/var/log/nginx/http.access.log' (pattern:/var/log/nginx/http.access.log)"
time="30-04-2020 08:57:25" level=info msg="Opening file '/var/log/nginx/https.access.log' (pattern:/var/log/nginx/https.access.log)"
time="30-04-2020 08:57:25" level=info msg="Opening file '/var/log/nginx/error.log' (pattern:/var/log/nginx/error.log)"
time="30-04-2020 08:57:25" level=info msg="Opening file '/var/log/auth.log' (pattern:/var/log/auth.log)"
time="30-04-2020 08:57:25" level=info msg="Opening file '/var/log/syslog' (pattern:/var/log/syslog)"
time="30-04-2020 08:57:25" level=info msg="Opening file '/var/log/kern.log' (pattern:/var/log/kern.log)"
...

At runtime

cscli allows you to view Crowdsec metrics info via the metrics command. This allows you to see how many lines are coming from each source, and if they are parsed correctly.

You can see those metrics with the following command:

 metrics

cscli metrics example

##  metrics
...
INFO[0000] Acquisition Metrics:                         
+------------------------------------------+------------+--------------+----------------+------------------------+
|                  SOURCE                  | LINES READ | LINES PARSED | LINES UNPARSED | LINES POURED TO BUCKET |
+------------------------------------------+------------+--------------+----------------+------------------------+
| /var/log/nginx/http.access.log           |         47 |           46 |              1 |                     10 |
| /var/log/nginx/https.access.log          |         25 |           25 | -              |                     18 |
| /var/log/kern.log                        |     297948 |       297948 | -              |                  69421 |
| /var/log/syslog                          |     303868 |       297947 |           5921 |                  71539 |
| /var/log/auth.log                        |      63419 |        12896 |          50523 |                  20463 |
| /var/log/nginx/error.log                 |         65 |           65 | -              | -                      |
+------------------------------------------+------------+--------------+----------------+------------------------+
...

Info

All these metrics are actually coming from Crowdsec's prometheus agent. See prometheus directly for more insights.