Using rsyslog to monitor files

This feature is quite new to me but it is really neat for monitoring applications that does not write to syslog.
But you still want to pass on the data in "realtime" without to much fuzz.

Create a file under /etc/rsyslog.d/ that does the magic.

cd /usr/local/bro/logs/
logs=$(find . | awk -F\/ '{print $3}'|awk -F\. '{print $1}'|sort -u)

echo $ModLoad imfile # > bro.conf
for i in $logs
do
echo '$InputFileName /usr/local/bro/logs/current/'$i >> bro.conf
echo '$InputFileTag bro_'$i >> bro.conf
echo '$InputFileStateFile stat-bro_'$i >> bro.conf
echo '$InputFileSeverity info' >> bro.conf
echo '$InputFileFacility local7' >> bro.conf
echo '$InputRunFileMonitor' >> bro.conf
done

echo 'local7.* @your_syslog_server_to_use_for_analyzing' >> bro.conf

An example file

$InputFileSeverity info
$InputFileFacility local7
$InputRunFileMonitor
$InputFileName /usr/local/bro/logs/current/ssl.log
$InputFileTag bro_ssl:
$InputFileStateFile stat-bro_ssl
$InputFileSeverity info
$InputFileFacility local7
$InputRunFileMonitor
$InputFileName /usr/local/bro/logs/current/smtp.log
$InputFileTag bro_smtp:
$InputFileStateFile stat-bro_smtp
$InputFileSeverity info
$InputFileFacility local7
$InputRunFileMonitor
$InputFileName /usr/local/bro/logs/current/smtp_entities.log
$InputFileTag bro_smtp_entities:
$InputFileStateFile stat-bro_smtp_entities
$InputFileSeverity info
$InputFileFacility local7
$InputRunFileMonitor
$InputFileName /usr/local/bro/logs/current/notice.log
$InputFileTag bro_notice:
$InputFileStateFile stat-bro_notice
$InputFileSeverity info
$InputFileFacility local7
$InputRunFileMonitor
$InputFileName /usr/local/bro/logs/current/ssh.log
$InputFileTag bro_ssh:
$InputFileStateFile stat-bro_ssh
$InputFileSeverity info
$InputFileFacility local7
$InputRunFileMonitor
$InputFileName /usr/local/bro/logs/current/ftp.log
$InputFileTag bro_ftp:
$InputFileStateFile stat-bro_ftp
$InputFileSeverity info
$InputFileFacility local7
$InputRunFileMonitor
local7.* @127.0.0.1

 

The script assumes that your bro installation has been up and running for a couple of days to most types of logfiles. Edit your syslogserver and copy the file to /etc/rsyslog.d/ and restart rsyslog. Depending on where your suslogserver is, you should se the data flow, if not start digging with tcpdump.

tcpdump -n -r filename udp and dst port 514