Acquisition
Info
Please note that the /etc/crowdsec/config/acquis.yaml
should be auto generated by the wizard in most case.
The acquisition configuration specifies lists of files to monitor and associated "labels".
The type
label is mandatory as it's later used in the process to determine which parser(s) can handle lines coming from this source.
Acquisition can be found in /etc/crowdsec/config/acquis.yaml
, for example :
Acquisition example
filenames:
- /var/log/nginx/access-*.log
- /var/log/nginx/error.log
labels:
type: nginx
---
filenames:
- /var/log/auth.log
labels:
type: syslog
Testing and viewing acquisition¶
At startup¶
At startup, you will see the monitored files in /var/log/crowdsec.log
:
...
time="30-04-2020 08:57:25" level=info msg="Opening file '/var/log/nginx/http.access.log' (pattern:/var/log/nginx/http.access.log)"
time="30-04-2020 08:57:25" level=info msg="Opening file '/var/log/nginx/https.access.log' (pattern:/var/log/nginx/https.access.log)"
time="30-04-2020 08:57:25" level=info msg="Opening file '/var/log/nginx/error.log' (pattern:/var/log/nginx/error.log)"
time="30-04-2020 08:57:25" level=info msg="Opening file '/var/log/auth.log' (pattern:/var/log/auth.log)"
time="30-04-2020 08:57:25" level=info msg="Opening file '/var/log/syslog' (pattern:/var/log/syslog)"
time="30-04-2020 08:57:25" level=info msg="Opening file '/var/log/kern.log' (pattern:/var/log/kern.log)"
...
At runtime¶
cscli allows you to view Crowdsec metrics info via the metrics
command.
This allows you to see how many lines are coming from each source, and if they are parsed correctly.
You can see those metrics with the following command:
metrics
cscli metrics example
## metrics
...
INFO[0000] Acquisition Metrics:
+------------------------------------------+------------+--------------+----------------+------------------------+
| SOURCE | LINES READ | LINES PARSED | LINES UNPARSED | LINES POURED TO BUCKET |
+------------------------------------------+------------+--------------+----------------+------------------------+
| /var/log/nginx/http.access.log | 47 | 46 | 1 | 10 |
| /var/log/nginx/https.access.log | 25 | 25 | - | 18 |
| /var/log/kern.log | 297948 | 297948 | - | 69421 |
| /var/log/syslog | 303868 | 297947 | 5921 | 71539 |
| /var/log/auth.log | 63419 | 12896 | 50523 | 20463 |
| /var/log/nginx/error.log | 65 | 65 | - | - |
+------------------------------------------+------------+--------------+----------------+------------------------+
...
Info
All these metrics are actually coming from Crowdsec's prometheus agent. See prometheus directly for more insights.