Its been a
while since I made a blog entry, figured I’d stop being lazy and give back to
the community again! Recently I upgraded my firewall and decided to go the custom
route. My flavor of choice was PFSense, as it has robust features. After months
of using it, I couldn’t be happier, but there was one issue I didn’t like, I couldn’t
configure the firewall to block bot/port scans. A few google
searches produced nothing. People mentioned using Snort, but as I started playing
with those tools, I thought there must be a simpler way. To give you some
background on my motivations and why I do this: when I first started building
cloud servers, I was shocked by the amount of traffic that is constantly
bombarding your firewall. I saw quite a bit of repeat offenders. So on my cloud
instances, I used UFW + fail2ban to block anything that touched ports outside
my “allowed ports”.
FYI - this
can be super frustrating when you “forget” that you have this set and start
playing with something, like a web server on a different port. Depending on
your cloud provider, you could be screwed (I somehow blocked everything including
localhost once and had no console, YIKES!). You may need to connect via VPN in
case of emergency. I also recommend taking an audit for several days just to make sure that legit services are not reconfigured, as you do not want to block them.
My home
network has nowhere near as many public facing services as my cloud servers, but it does have some
and that makes me nervous because it is where literally EVERYTHING resides. Most of the traffic you see is bot scans and usually only a few ports. From time to time, you'll get someone with some talent who is curious. Its that person I'm concerned about, because when they find
your only open port[s] and begins to test, all they have to do is wait for you to forget to patch something and then
they’re able to use a near-day to get a foothold. The thought of this makes me wake up in cold sweats! BTW - explaining this to people outside the community makes feel as if I should be wearing aluminum foil.
Like with my cloud servers, blocking an IP from touching any port outside your allowed range makes it SIGNIFICANTLY harder to scan and see whats open. For the record, it can still be done. You could use something like TOR to funnel the port scan through, then use the TOR management port to change IP’s for each port. However that would only give you around 7k ports as there are only about that many TOR endpoints. So you'd eventually get an nmap top 1k but not a full port scan, as there are not 65535 exit nodes. I did read a writeup way back that said sneaker proxies (yes, as in "shoe" proxies) have access to large ranges of unique RFC 1918 IP address if you REALLY were diligent on getting a full scan (*cough* compromised machines with RAT's installed to proxy connections though *cough*).
Like with my cloud servers, blocking an IP from touching any port outside your allowed range makes it SIGNIFICANTLY harder to scan and see whats open. For the record, it can still be done. You could use something like TOR to funnel the port scan through, then use the TOR management port to change IP’s for each port. However that would only give you around 7k ports as there are only about that many TOR endpoints. So you'd eventually get an nmap top 1k but not a full port scan, as there are not 65535 exit nodes. I did read a writeup way back that said sneaker proxies (yes, as in "shoe" proxies) have access to large ranges of unique RFC 1918 IP address if you REALLY were diligent on getting a full scan (*cough* compromised machines with RAT's installed to proxy connections though *cough*).
Anyhow, I didn’t
want to go with my cloud solution and “corrupt” PFSense with something like
fail2ban, as PFSense is a premium firewall and toolset. After doing some digging into how PFSense
firewall does it's logging, I realized that my “catch all REJECT” rule located at the bottom of
my WAN interface ruleset had the option for logging. I turned that on and located the log location
on the filesystem. PFSense logs in CSV, which makes it super easy to pull and
filter logs. I dumped my “/var/log/filter.log” and a novel worth of information
dropped out:
Thinking on my feet, I logged onto one of my cloud servers, installed netcat and attempted to connect to a port that was not on my “approved” list, then grepped for the cloud server IP address in my PFSense logs:
Thinking on my feet, I logged onto one of my cloud servers, installed netcat and attempted to connect to a port that was not on my “approved” list, then grepped for the cloud server IP address in my PFSense logs:
grep 'IP.ADDRESS'
/var/log/filter.log
Wow, that was pretty simple, I had found it in the first go, this was going to be a good morning.
It produced a wealth of information, as the ip address was associated with
other rules I had created. So I went back into the PFSense firewall config for
that specific rule to see if there was some kind of label for it, and conveniently
enough, there was!
This picture depicts where to go to find the rule number:
This picture depicts where to go to find the rule number:
A note: You’ll
notice that I do not have my rule set to “drop”. I recently move to the school
of thought that it is better to let the person or bot know right away that the port is
blocked instead of forcing them to wait many more milliseconds to time out. Reject is nearly immediate where as they'll continue to wait for a packet to come back, so it creates far fewer states on your router. There is a really good writeup on what I am talking about here
here (http://www.chiark.greenend.org.uk/~peterb/network/drop-vs-reject).
It will save you some resources!
ANOTHER note on this: I still use “drop” on certain cloud servers, especially during red team engagements I do this because I want full anonymity. Sometimes security minded people, or people like me with an aluminum foil hat on staring at their logs all day, will sometimes counter scan you. With blocking, if the said person goes to do a counter scan and nothing seems to be there, they may assume "Hmmm, maybe it was a docker image that spun up for a scan or maybe they are behind a firewall that is doing NAT, and they may just forget about you. If you want want that level of anonymity, I totally understand and uses that level sometimes. For me and my home network, resources are king. I cant stop building things and because of that am now would be in the run for a home-enterprise sized network of the year. So I need to save resources where I can.
ANOTHER note on this: I still use “drop” on certain cloud servers, especially during red team engagements I do this because I want full anonymity. Sometimes security minded people, or people like me with an aluminum foil hat on staring at their logs all day, will sometimes counter scan you. With blocking, if the said person goes to do a counter scan and nothing seems to be there, they may assume "Hmmm, maybe it was a docker image that spun up for a scan or maybe they are behind a firewall that is doing NAT, and they may just forget about you. If you want want that level of anonymity, I totally understand and uses that level sometimes. For me and my home network, resources are king. I cant stop building things and because of that am now would be in the run for a home-enterprise sized network of the year. So I need to save resources where I can.
“Tracking ID”
showed up in the logs when I grepped for the number associated with it. Now I
had to find a way to pull the IP information, get them into some kind of crude
file or database and make sure that there were no duplicates (I later realized
that PFSense would handle duplicates for me, but my script does it also). I initially
created a bash script that pulled the information. It was stupid simple and took
me maybe 10 mins to write. However, I felt it was very “flimsy” and I was not
at all confident with letting a flimsy, half-assed shell script handle my banned
IP’s list. For the record, there are more "flimsy" bash scripts running than I'd like keeping me afloat. But this is one that could lock me out completely. What do I mean by "flimsy? I guess I'll share:
Yeah, I didn’t like it either. So I warmed up my python skills and created something that made me feel warm and fuzzy inside:
Here's a direct link to my github in case blogger gets the spacing wrong:
https://github.com/s0meguy1/PFBlock/blob/master/blacklist.py
By no means would I call this a world-renowned python script, but it made me feel much better, and I could later do things like add timestamps, error handling (which I did none of here) or whatever came up as things started breaking. The sky’s the limit with Python.
#!/usr/local/bin/bash
grep 'RULEID' /var/log/filter.log |cut -d ',' -f 19|sort|uniq >> /root/Initial.txt
cat /root/Initial.txt|sort|uniq > /root/IPBlacklist.txt
cp /root/IPBlacklist /root/Initial.txt
grep 'RULEID' /var/log/filter.log |cut -d ',' -f 19|sort|uniq >> /root/Initial.txt
cat /root/Initial.txt|sort|uniq > /root/IPBlacklist.txt
cp /root/IPBlacklist /root/Initial.txt
Yeah, I didn’t like it either. So I warmed up my python skills and created something that made me feel warm and fuzzy inside:
#!/usr/local/bin/python2.7
import os
import time
DBsave = "/root/IPBlacklist.txt" #Change if you don’t like the location
def Filter(data):
count = 0
path = "/var/log/filter.log"
f = open(path,'r')
olddata = []
unsorted = []
with open(data) as old:
for odata in old:
if len(odata.strip()) != 0:
olddata.append(odata.rstrip("\n"))
sorted = []
with open(path) as text:
for line in text:
if 'RULEID' in line: #change this line based on your rule number
unsorted.append(line.split(',')[18])
nextdata = list(set(unsorted))
nextolddata = list(set(olddata))
for nextline in nextdata:
if nextline not in nextolddata:
count += 1
sorted.append(nextline)
if count == 0:
exit()
else:
for fileline in sorted:
file = open(data, 'a+')
file.write(fileline + "\n")
file.close()
if not os.path.exists(DBsave):
    with open(DBsave, 'w'): pass
Filter(DBsave)
import os
import time
DBsave = "/root/IPBlacklist.txt" #Change if you don’t like the location
def Filter(data):
count = 0
path = "/var/log/filter.log"
f = open(path,'r')
olddata = []
unsorted = []
with open(data) as old:
for odata in old:
if len(odata.strip()) != 0:
olddata.append(odata.rstrip("\n"))
sorted = []
with open(path) as text:
for line in text:
if 'RULEID' in line: #change this line based on your rule number
unsorted.append(line.split(',')[18])
nextdata = list(set(unsorted))
nextolddata = list(set(olddata))
for nextline in nextdata:
if nextline not in nextolddata:
count += 1
sorted.append(nextline)
if count == 0:
exit()
else:
for fileline in sorted:
file = open(data, 'a+')
file.write(fileline + "\n")
file.close()
if not os.path.exists(DBsave):
    with open(DBsave, 'w'): pass
Filter(DBsave)
Here's a direct link to my github in case blogger gets the spacing wrong:
https://github.com/s0meguy1/PFBlock/blob/master/blacklist.py
By no means would I call this a world-renowned python script, but it made me feel much better, and I could later do things like add timestamps, error handling (which I did none of here) or whatever came up as things started breaking. The sky’s the limit with Python.
This was great
and all, but I realized I had no way to get this newly created list into PFSense.
Luckily, I had installed PFBlockerNG, which I HIGHLY recommend! I also use it as
a quasi-pihole, and block entire countries or regions from accessing my home
network (don't worry, google hosts this blog. I wanted a larger audience than my local city). PFBlocker has the option for custom FQDN whitelists, but also IPV4 (and IPV6)
whitelists. This feature is usually used to pull from URL’s which contain
large lists of IP addresses that people are constantly updating, which I also do, But I decided
to see if it would read from the filesystem, which it did.
To do this,
within the PFSense web app (with PFBlocker installed), head to:
Firewall
-> PFBlocker -> IPv4
From there,
you add a new list, give it an alias (alias might be optional? I can’t remember),
then under “IPv4 Lists” you can add the blacklist from the absolute path of the
txt file you created in the python script (Ea /location/of/your/sript.txt –
the DBsave variable). I set my list to update every hour.
Fantastic! But
one more thing, you must add the python script to cron so that it can be set to
run and update. As I said before, this could be done within the python script, however I was too lazy
to start down that path when I could just have cron do it for me. Also it probably would eat more resources to do that. I decided
since the PFSense list was updating every hour, that I would have cron do it every 30
minutes just to be safe, so I ran “crontab -e” and added:
*/30 * * * *
/usr/local/bin/python2.7 /root/blacklist.py
*** you may
want to verify your version of python is in /usr/local/bin/
And that’s it!
You are now preventing unwanted harassment from returning to your network!
I made a promise that I'd start writing more as I created things. So if you're interested in what is next, come back and checkout my blog, or follow me on twitter:
@s0meguy1
I'll be posting this there as well. Good luck!
I made a promise that I'd start writing more as I created things. So if you're interested in what is next, come back and checkout my blog, or follow me on twitter:
@s0meguy1
I'll be posting this there as well. Good luck!
