Reply
Emerging Member
Posts: 41
Registered: ‎10-23-2015
Kudos: 24

USG Firewall Log Live Visulaztion using rsyslog server and sumologic

[ Edited ]

Hey All,

 

So I saw this thread about analyzing USG firewall logs a couple weeks ago and it got me thinking over thanksgiving break on the easiest and cheapest way to quickly visualize firewall logs. A lot of you may already know about this or have similiar / better setups. I'm not an I.T. person by trade, I just happen to be the one running the tech side of my companys two small offices on the side, and I was looking for the most simple solution I could find.

 

I did a bunch of research and came across projects link splunk, kibana and the ELK stack, grafana, etc. I dabbled in all of these but none of them were easy to setup, easy to maintain, and easy to customize. I then came across SumoLogic, which is a cloud hosted log visualizer. They offer a free tier for people transmitting less than 500mb/day of data, which I should be well under.

 

The way it works is you install a collector on the machine where the log file is located, and then it transmits the data straight to sumologic. You can set up the collector to grab syslog directly, but I found it easier to log the syslog entries from my unifi controller to a seperate file, and then tell the collector to watch that local file.

 

From there you can parse the data and visualize it anyway you want. Graphs, lists, GeoIP lookup, etc. After a day of tinkering here is what my first dashboard looks like

 

Screen Shot 2017-11-27 at 3.24.48 PM.png

 

The first thing I did was follow the guide @allank so wonderfully put together to setup a rsyslog server here: https://community.ubnt.com/t5/UniFi-Routing-Switching/Analyzing-USG-firewall-logs-for-attack-visibil... I set mine up on a $5/mo + 4$/mo block storage instance on DigitalOcean

 

Once it was setup and the logs were streaming in, I then created my account on SumoLogic, and wen't through the setup wizard of installing the collector

 

Choose Streaming Data

 

Screen Shot 2017-11-27 at 3.42.01 PM.png

Choose Other Sources at the bottom

Screen Shot 2017-11-27 at 3.42.20 PM.png

Choose Local File

 

Screen Shot 2017-11-27 at 3.42.37 PM.png

Choose New Collector, and then copy and paste the provided code on your rsyslog server. It will take a minute, but once it's installed the continue button will be available

 Screen Shot 2017-11-27 at 3.42.49 PM.png

 

Then you can point the collector at the unifi log file on your rsyslog server

 

 **Note the file path in here is wrong if you followed the rsyslog tutorial above**

it should be /data/log/unifi/unifi.log

 

Screen Shot 2017-11-27 at 3.43.34 PM.png

It will take a few minutes to configure itself, transmit, and index the data, but soon enough you'll see your syslog messages come pouring in

 

I recomend reading up on the various syntax and comands you can use in sumo logic as there's plenty you can do. I learn best by looking at examples, so here's a few of the search's I performed on the data and then pinned to my dashboard. Keep in mind you'll need to change the source command at the top of each one to you're own source category (What you labeled the category when configuring the collector)

 

These may or may not be the most efficent way to parse the data, but it's what I got working. I'm not a programmer or data visulizer by anymeans

 

Top Blocked IP's

**NOTE** parsing is case sensitive, for some reason this forum de-capilized my source code. src=* should be all capital letters. same with parsing any other data

Be sure to change the first source_ip to your own I.P. address

 

_sourceCategory="GCNY" 
| parse "src=* " as source_ip
| where source_ip != "X.X.X.X" **Put your own static IP here**
| where source_ip != "192.168.0.1"
| lookup latitude, longitude, country_code, country_name, region, city, postal_code, area_code, metro_code from geo://default on ip = source_ip
| where source_ip != "192.168.0.1"
| count by source_ip, latitude, longitude, country_code, country_name, region, city, postal_code, area_code, metro_code
| top 10 source_ip, country_name by _count

Screenshot of Config

Screen Shot 2017-11-27 at 4.05.31 PM.png

 

Total Blocked Requests

I made it ignore a bunch of block requests coming from 224* IP's. I suspect this is something I should look into, but that's for another day.

 

_sourceCategory="GCNY" 
| parse "DST=* " as destination_ip
| where destination_ip != "224.0.0.251"
| where destination_ip != "224.0.0.1"
| parse "WAN_LOCAL-*]" as firewall_rule
| where firewall_rule matches "4000-D" 
| count firewall_rule

Screenshot of Config

Screen Shot 2017-11-27 at 4.04.21 PM.png

 

 Blocked Over Time
Viewed as an Area Chart under Aggregates

_sourceCategory="GCNY" 
| parse "DST=* " as destination_ip
| where destination_ip != "224.0.0.251"
| where destination_ip != "224.0.0.1"
| parse "WAN_LOCAL-*]" as firewall_rule
| where firewall_rule matches "4000-D" 
| timeslice 5m
| count by _timeslice, firewall_rule

Screenshot of Config

Screen Shot 2017-11-27 at 4.03.23 PM.png

GeoIP Lookup of Blocked IPs

**NOTE** parsing is case sensitive, for some reason this forum de-capilized my source code. src=* should be all capital letters. same with parsing any other data

SumoLogic uses a built-in geo IP lookup service. Super handy.

Replace X.X.X.X with your own IP

Viewed as Map in Aggregate

 

_sourceCategory="GCNY" 
| parse "src=* " as remote_ip
| where remote_ip != "X.X.X.X" or "192.168.0.1" or "224.*.*.*"
| lookup latitude, longitude, country_code, country_name, region, city, postal_code, area_code, metro_code from geo://default on ip = remote_ip
| count by latitude, longitude, country_code, country_name, region, city, postal_code, area_code, metro_code
| sort _count

Screenshot of Config

Screen Shot 2017-11-27 at 4.01.24 PM.png

 

Top Blocked TVP Ports:

This one was a little more involved. SumoLogic allows you to lookup data and fields from hosted CSV files, and match parsed data against it. I wanted a database of the protocols and descriptions of each port, so I grabbed a list of TCP ports and a list of UDP ports and created a google sheet that looks like this

 

Screen Shot 2017-11-27 at 4.11.42 PM.png

etc...

 

I then created a public link to it, and discovered that if you add /gviz/tq?tqx=out:csv after the unique key in the google link, it will share it as a CSV file. More info here: https://megalytic.com/support/using-google-sheets-to-host-editable-csv-files/

 

Then we can perform lookup's on that list within a SumoLogic search function. Here's my code for TCP ports, parses the destination port and matches it against the tcpport field in the google doc, then grabs the corresponding service and description data

 

_sourceCategory="GCNY" 
| parse "DST=* " as destination_ip
| where destination_ip != "224.0.0.251"
| where destination_ip != "224.0.0.1"
| parse "WAN_LOCAL-*]" as firewall_rule
| where firewall_rule matches "4000-D"
| parse "PROTO=* " as protocol
| where protocol = "TCP" 
| parse "DPT=* " as dest_port
| lookup service from https://docs.google.com/spreadsheets/d/1SkjeN_mZGgziTyP5JOZx_w6E2CE0oCxLhmhSGcUk7FU/gviz/tq?tqx=out:csv on dest_port=tcpport
| lookup description from https://docs.google.com/spreadsheets/d/1SkjeN_mZGgziTyP5JOZx_w6E2CE0oCxLhmhSGcUk7FU/gviz/tq?tqx=out:csv on dest_port=tcpport
| count by dest_port, service, description
| top 10 dest_port by _count, service, description

Screenshot of config

Screen Shot 2017-11-27 at 4.16.51 PM.png

 

Blocked UDP Ports

And then did the same thing by creating another CSV file for UDP ports

_sourceCategory="GCNY" 
| parse "DST=* " as destination_ip
| where destination_ip != "224.0.0.251"
| where destination_ip != "224.0.0.1"
| parse "WAN_LOCAL-*]" as firewall_rule
| where firewall_rule matches "4000-D"
| parse "PROTO=* " as protocol
| where protocol = "UDP" 
| parse "DPT=* " as dest_port
| lookup service from https://docs.google.com/spreadsheets/d/14ogIzDDKu8jII2AiTuzOlz1CCVtMy_lgfZ2y3HfNhYE/gviz/tq?tqx=out:csv on dest_port=udpport
| lookup description from https://docs.google.com/spreadsheets/d/14ogIzDDKu8jII2AiTuzOlz1CCVtMy_lgfZ2y3HfNhYE/gviz/tq?tqx=out:csv on dest_port=udpport
| count by dest_port, service, description
| top 10 dest_port by _count, service, description

Screenshot of Config

Screen Shot 2017-11-27 at 4.18.33 PM.png

 

 

After that, I added all of these searches to my SumoLogic dashboard and arranged them how you saw in the beginning. You can then turn on live mode and see live updates, quickly change the time range of all the widgets, share the dashboard with a public URL, etc.

 

There's a ton of apps and pre-configured dashboards for all sorts of data. If you search for and add the Threat Intel Quick Analyse dashboard, you can search all of your logs for suspicious IP's and activity. This shows that over the last 24-hours I may have been targeted with an SSH scanner

Screen Shot 2017-11-27 at 4.22.54 PM.png

 

Anyways, that's it. Again, there may be better ways to parse or structure this data, but this is what I got working. Enjoy!

 

Member
Posts: 102
Registered: ‎01-01-2017
Kudos: 11
Solutions: 1

Re: USG Firewall Log Live Visulaztion using rsyslog server and sumologic

nice work. wondering if you can help me set up mine.

 

i have a syslog server setup using kiwi. The logs get saved to a file and SumoLogic picks up the log file. 

 

In live view i can see the syslog messages. Using your snippets of code to create the dashboard however it doesnt populate any data.

 for example if i use

_sourceCategory="GCNY" 
| parse "src=* " as source_ip
| where source_ip != "X.X.X.X" **Put your own static IP here**
| where source_ip != "192.168.0.1"
| lookup latitude, longitude, country_code, country_name, region, city, postal_code, area_code, metro_code from geo://default on ip = source_ip
| where source_ip != "192.168.0.1"
| count by source_ip, latitude, longitude, country_code, country_name, region, city, postal_code, area_code, metro_code
| top 10 source_ip, country_name by _count

i change sourceCategory="GCNY" to my category name

change the x.x.x.x to my WAN IP

change the 192.168.0.1 to my local subnet

 

 

Emerging Member
Posts: 41
Registered: ‎10-23-2015
Kudos: 24

Re: USG Firewall Log Live Visulaztion using rsyslog server and sumologic

I sometimes had trouble getting it to look in the right category for data, I would try this

 

If you go to your collection tab, you can launch a search in a specific dataset by clicking this button where your collector is listed

 

Screen Shot 2017-11-28 at 1.03.24 PM.png

 

The other thing I would look at is to make sure your messages that are coming in have "src=* " as a string to parse, do you have firewall logging enabed on your firewall rules? The default firewall rules don't have logging enabled, and to enable them you have to configure your config.json file on your controller

 

Look in the link I posted to the first tutorial on how to enable firewall logging

Member
Posts: 102
Registered: ‎01-01-2017
Kudos: 11
Solutions: 1

Re: USG Firewall Log Live Visulaztion using rsyslog server and sumologic

thanks mattlowe01

 

ye i edited the pre-defined rules using a json to add logging. i can confirm this works as my kiwi syslog server and prtg syslog both see the logs and i can see the rule WAN_LOCAL-4000-D  so i can see it dropping alot of traffic. 

 

Also when i look at my collectors and click that little symbol i see my logs coming in they have src=x.x.x.x (what ever IP is trying to access my network) 

Emerging Member
Posts: 41
Registered: ‎10-23-2015
Kudos: 24

Re: USG Firewall Log Live Visulaztion using rsyslog server and sumologic

[ Edited ]

Capitalize "src=* "

_sourceCategory="GCNY" 
| parse "src=* " as source_ip

Edit: for some reason the forum un-capitlizes all uppercase letters...

 

src should be all capitals

Member
Posts: 102
Registered: ‎01-01-2017
Kudos: 11
Solutions: 1

Re: USG Firewall Log Live Visulaztion using rsyslog server and sumologic

never mind managed to sort it. 

 

must of been tired doing it earlier. i had to use capitals for SRC

 

so parse "src=* "   it didnt like lower case

Emerging Member
Posts: 41
Registered: ‎10-23-2015
Kudos: 24

Re: USG Firewall Log Live Visulaztion using rsyslog server and sumologic

[ Edited ]

Yeah... just realized that. I'm gonna make a note in the main post. Does anyone know how to make the forum not un-capitilize letters?

 

edit: seems it only un-capitlizes src, not dst or anythign else. odd

Member
Posts: 102
Registered: ‎01-01-2017
Kudos: 11
Solutions: 1

Re: USG Firewall Log Live Visulaztion using rsyslog server and sumologic

thanks for sharing all this. 

 

apparently my unifi firewall has blocked over 5000 attemptes in the last 6 hours. thats a scary amount

New Member
Posts: 2
Registered: ‎07-15-2016

Re: USG Firewall Log Live Visulaztion using rsyslog server and sumologic

One of the things I noticed for the GEO Block rule is that you are not specifying which Firelog rule you are looking at.  So in my case, for a period of time, it seemed to have logged not just drops but accepts, and therefore my GeoIP Blocked IP count's went through the roof because it was using accept's as well and counting them.  I updated the rules to this, anyone see any issue?

 

I basically check:

 

1) Is the source not my router/home network

2) Is the source a response to a drop rule

3) Is the event not related to the weird 224.*.*.* traffic <maybe related to VPN traffic? >

 

 

_sourceCategory="Whatever"
| parse "src=* " as remote_ip
| where remote_ip != "my external IP" 

| where remote_ip != "192.168.0.1"

| where remote_ip != "192.168.1.1"
| parse "WAN_LOCAL-*]" as firewall_rule
| where firewall_rule matches "4000-D"
| parse "DST=* " as destination_ip
| where destination_ip != "224.0.0.251"
| where destination_ip != "224.0.0.1"
| where destination_ip != "224.*.*.*"
| lookup latitude, longitude, country_code, country_name, region, city, postal_code, area_code, metro_code from geo://default on ip = remote_ip
| count by latitude, longitude, country_code, country_name, region, city, postal_code, area_code, metro_code
| sort _count

Highlighted
New Member
Posts: 28
Registered: ‎06-07-2017
Kudos: 19
Solutions: 1

Re: USG Firewall Log Live Visulaztion using rsyslog server and sumologic

[ Edited ]

Thanks, this has been kind of fun to play with.  I added a rule 4000 to my WAN LOCAL to drop, without logging, any of the DST=224.0.0.251 traffic that would have been dropped and logged.  This cleans a lot of clutter out of the raw logs that I had no need to see.  The SumoLogic queries change to referencing "WAN-LOCAL-*" matching "4001-D" now instead of "4000-D".  The SRC and Destination IP filtering gets smaller too.

 

I don't have any destination 224.0.0.1, or I'd add that to the Address Group too.

USG-Firewall-Group-Address-mDNS.png

 

USG-Firewall-Group-Port-mDNS.png

 

USG-Firewall-WAN-LOCAL.png

 

Reply