Assumptions vs. Reality in DevOps

ELK GeoIP

Ever set up a cloud machine using Amazon EC2 or any other cloud provider? If not, it's big fun. After your machine is up an running wait some minutes and attackers start their work by brute forcing root ssh login.

So I thought it would be a good idea to know where they come from. I started my Ubuntu 14.04 machine and set up fail2ban, which logs to /var/log/fail2ban.log. Read more on fail2ban setup.

A simple

cat /var/log/fail2ban.log*|grep Ban|awk '{print $1,$2,$7;}' | sed 's/ /T/' | sed 's/,/./' >> fail2ban.log

extracts all banned IP addresses to fail2ban.log, which looks like

2015-06-05T00:37:20.887 186.121.210.50
2015-06-05T01:12:02.366 182.100.67.114
2015-06-05T02:20:53.002 218.65.30.107
2015-06-05T02:23:13.149 186.147.233.125
2015-06-05T04:08:53.423 119.97.184.14
2015-06-05T05:59:14.905 43.255.188.146
2015-06-05T07:02:10.099 66.210.34.180

2015-06-05T07:18:58.651 58.218.211.166

Truely, this list gets quite long over time. Next download and install ELK stack. Elasticsearch and Logsatsh are running out of the box. Edit Kibanas config file kibana.yml and set Elasticsearch URL pointing to your local ES instance:

elasticsearch_url: "http://localhost:9200"

Next create a fail2ban directory on same level as ELK resides and put your fail2ban.log in. Last step: Create grok filter for Logstash named auth.conf and save it to ELK level directory:

input {

   file {

     type => "fail2ban"

     path   => [ "/home/papa/Projekte/es/fail2ban/fail2ban.log*" ]

     start_position => "beginning"

     discover_interval => 1

   }

}

filter {

   if [type] == "fail2ban"

   {

     grok {

       match => [ "message", "%{TIMESTAMP_ISO8601:bandate} %{IPORHOST:ip}" ]

     }

    geoip {

      source => "ip"

    }

   }

}

output {

  elasticsearch {

    host => localhost

  }

}

You're fine now just starting up the ELK:

./elasticsearch-1.5.2/bin/elasticsearch >> es.log &

./logstash-1.5.0/bin/logstash -f ./auth.conf >> ls.log &

./kibana-4.0.2/bin/kibana >> kib.log &

Simply put these 3 line to a run.sh file for reuse.

Point your favourite web browser to localhost:5601, which should come up with an unconfigured Kibana. Check Use event times to create index names and set thee Time-field name to bandate.

Switch to Visualize and create new Tile Map from new search. Bucket type is like Geo Coordinates, Aggregation GeoHash, field geoip.location. Don't forget to click apply. Maybe you'll need to adjust time settings to get your data displayed in Kibana. By default last 15mins are selected.

Your're done! Finally you should get somwthing like this displayed for your banned IPs trying to ssh your server.

But maybe you do not want to know about banned IPs but to accepted logins. Take a look on your /var/log/auth.log and adjust auth.conf accordingly.