by WebKeyDesign | May 9, 2014 | Linux, Networking, Software
SARG Reports are a good compliment to Squid Proxy and since there is a package that is available for installation in pfsense, it makes good sense to setup SARG Reports. The downsides to SARG Reports is that the reports do take up space and over time this can be significant. This posting is about a problem I encountered on pfsense 2.1 and the latest SARG package.
For some unknown reason the reports stopped generating. Upon checking my System Log this is the issue I found:
php: /pkg_edit.php: The command 'export LC_ALL=C && /usr/pbi/sarg-amd64/bin/sarg -d `date +%d/%m/%Y`-`date +%d/%m/%Y`' returned exit code '1',
the output was 'SARG: Cannot get the modification time of input log file /var/log/squid/access.log (No such file or directory). Processing it anyway SARG: File not found: /var/log/squid/access.log'
I am using the 64-bit version of pfsense, so hence the sarg-amd64. If you are using 32-bit, it will state instead sarg-i386.
The solution is to edit the sarg.conf file that is located in one of these locations, depending on your pfsense build:
/usr/pbi/sarg-amd64/etc/sarg/sarg.conf
/usr/pbi/sarg-i386/etc/sarg/sarg.conf
You will need to verify that the access_log line is correct:
#access_log /usr/local/squid/var/logs/access.log
In my case, removing the # sign and specifying the correct path to my Squid access.log corrected the problem.
If you have issues with SARG Reports, it is best to do the following:
- Under the Status Menu – click SARG Reports.
- On the General tab click Save
- Next click on the Users tab and click Save
- Click Schedule and create your schedule or if you have one already open it up and click Save.
- You can go back to the Schedule and Force Update to see if SARG Reports are working now.
I also schedule SARG Reports in Cron to run at 11:50pm every night instead of midnight.
50 23 */1 * *
by WebKeyDesign | May 7, 2014 | cPanel, Networking, Tech Notes
In this post, I discuss how to setup your own domain with Zoho Mail, a hosted email solution from Zoho. Similar to Google Apps, Zoho provides a set of online business tools including office apps, project management, and contact management. At the time of this post the hosted email package has a free option as well as higher tiers for users who need more options. The biggest selling point for Zoho is that none of their apps have advertisements, and so if you are bothered by other webmail solutions that feature ads, Zoho seems to be a good alternative solution. Other than webmail access, the other reasons to use Zoho is that it works with desktop email clients, smartphones, and tablets.
My Requirements For An Email Solution
In order of importance, here is what I was looking for in an email solution.
Integrates with your Domain name
- I wanted an email solution that would work with my current personal domain that my family uses.
iPhone and iPad Support
- Everyone in the family has an iPhone, iPod, or iPad that they can use for email.
Apple Mail Support
- Oh, we do use our Mac computers every now and then, so we need desktop mail.
IMAP
- I want to keep email on the server, and not worry about losing it. I can also manage my email from my smartphone when I have time. This is really convenient.
Cost
- Free if possible, but am willing to pay for a good solution on a yearly basis.
Ad-free
- Not having my kids bombarded with advertisements is a good thing.
Zoho’s Instructions
Step 1: Verify domain ownership
Please verify your domain ownership. This is required to prevent imposters from using domains to send malicious messages. You can follow either the CNAME method or HTML method for verification.
Step 2: Add / import users to your organization
As the administrator of your organization, you have a Control Panel link in your user interface. In the Control panel, click User Details on the left list of options and click Add User option on the top. You can also import a list of users by selecting the Import User option.
Step 3: Migrating data to Zoho Mail
We recommend you to test migration for 2 users before pointing MX records.
Step 4: Point MX records to Zoho
Point the Mail Exchanger (MX) records to Zoho to start receiving mails to your inbox.
Changing My eMail
Steps 1 and 2 were pretty easy. Since I only have 4 users accounts, it took a few minutes to setup my four users in the Control panel. I skipped Step 3, I don’t really keep a lot of personal email. My current email was stored in Apple Mail and I was fine leaving it there. The interesting step is number 4. The Internet works via DNS. The domain naming system allows everyone and everything to find each other on the network. This is accomplished by DNS having different types of records to point requests to the right place. In the case of most personal domains, you have two different parties involved. The first is a domain registrar who takes care of your domain registration. The second is your hosting provider, usually for shared hosting this is a cPanel type hosting provider. When someone tries to email you at user@some_domain_name.net, a lookup is made to the root DNS authority for the .net domains, your registrar is what adds your domain to the DNS authority servers. The root server then sends you to your nameservers specified. Your nameservers are at your cPanel hosting provider. The final step in the lookup is to see what your cPanel nameserver has for what is called the MX records. The MX records have the server that processes your email and ideally where the email is going to go.
Your current hosting provider has MX records for its own email processing. What needs to be done, is to remove the current MX records and replace them with MX records that point to Zoho’s servers. This way only your email, emails sent to user@some_domain_name.net, will route to Zoho, but everything else will still be at your current hosting provider. To do this most cPanel hosts make it easy to do this now. Log into your cPanel control panel and scroll down to the Email section. You want to click open the MX Entry icon.
You will first choose your domain name that you want to change the MX records for. This is your main domain. First add the MX entries as Zoho instructs, then remove your current record for your host. When the changes are complete, it should look similar to this:
The DNS changes take a matter of minutes to a couple of hours to propagate to the rest of the internet.
by WebKeyDesign | Feb 1, 2013 | Networking
After several months of running pfSense as my home router solution, I now feel that my current Squid Proxy configuration is stable enough to recommend. I have been running the current Squid 3 package that is available in pfSense without many issues. The configuration is pretty simple. Primarily I found that running proxies, including Squid, in transparent mode is just too much trouble for home networks. Transparent mode never quite works right with iOS devices and other media devices, that in the end is not worth the hassle. Instead I manually specify a web proxy in my preferred browsers: Firefox, Safari, and IE. for the iPhone, you can specify a proxy for the wireless connection you are using. It is kind of a pain to have to remember to input a proxy, but you only do it one time and it is easier to troubleshoot one application or device at a time then trying to troubleshoot transparent mode and bring down all usage while you work things out.
I have a small home user network with around 20 different devices all communicating to the router. There are about six computers, and the rest of the devices are your typical smart phones, iPods, game consoles, and media players. The Squid setup is there to speed up web browsing and downloads for the computers and web browsers on the phones and iPods.
There are some basic configuration recommendations on the pfSense Wiki, but once you get past that, there is not much out there as to what settings to use. In general Squid uses two resources: disk space and memory. For my configuration I have settled on a 32 disk cache setting, meaning I have set aside 32GB of disk space to cache to disk. For memory, Squid utilizes memory in two different ways. The first to hold an index of the disk cache and the rest for the rest of Squid functions. To calculate the memory usage, the rule is that you need 10 MB for each Gigabyte of space you are caching.
- cache_mem 640 MB
- maximum_object_size_in_memory 4096 KB
- minimum_object_size 0 KB
- maximum_object_size 4194304 KB
- cache_dir ufs /var/squid/cache 32768 16 256
Using the parameters above, I have decided to utilize 640 MB for cache memory, which is twice the rule. The largest cached object in memory is 4 MB, meaning anything larger than 4 MB will not be cached in memory and will have to rely on disk cache. Lastly the largest file on disk that can be cached is equal to 4 GB.
With this configuration and typically 3 to 8 devices connecting to the proxy, at the most the disk cache grows by a couple of GB a week. More importantly, RAM for the router peaks at 58% in use, which leaves plenty of room for other pfSense functions. The current router has 4GB of RAM and is an Intel Atom CPU based system.
by WebKeyDesign | Aug 28, 2012 | Networking
At this time, I have installed pfSense a few times and have been running pfSense for about three months. The experience has been very trying at times and so hopefully my personal notes will help others in deciding if pfSense is right for you. Note that I am trying to use pfSense as a home router and caching proxy, and not as a typical business class network firewall, which is what its actual intent.
With the hardware setup, I had to now install the pfSense software and configure it. The SuperMicro MBD-X7SPA-HF-O has an internal USB port that you can use if you want to install pfSense or any operating system to, but in my case, I was going to use a 60GB SSD drive instead as my main operating system drive. There are multiple ways to do the installation, 1) Make a bootable USB drive and boot from that, 2) CD-ROM installation, or 3) Use IPMI to control the system remotely and install from an ISO image. Since my box has no actual CD-ROM drive, I ended up using IPMI, which works quite well, once you get some experience with it. Once I booted via the ISO image (through IPMI), the installer formatted the SSD drive and copied the necessary files. The rest of the configuration could be done through IPMI, or you can simply hookup a keyboard and monitor to the box, like I did.
Rather than cover the installation process in more depth, I will refer you instead to Overclockers, whose guide covers the installation in more depth than I could ever hope to. It was actually the Overclockers guide and this SmallNetBuilder article that inspired me to build my own pfSense box in the first place.
Helpful Setup Tips
The first thing of note is that you should specify “em0” as the LAN connection. This is contrary to the Overclockers guide. The reason is that the IPMI only works through the first Network Port and I would like to be able to access IPMI internally on my LAN only. You will specify “em1” as the WAN connection. (In the BIOS you can specify a specific IP address for the IPMI setup and take note of the MAC Address; you can then setup a DHCP reservation for the IPMI address.)
For DHCP reservations, it is helpful to have a list of MAC addresses and the IP numbers that you are going to give those devices. This way you can add them all at once during the setup and be done with it. Most of the devices in my home now have reserved IP addresses.
You will want to configure the Admin Web Interface to HTTPS and to a different port number. Basement PC Tech has an excellent guide on how to add a certificate to the Admin interface.
You can add FreeIPMI to pfSense. Reference this post on the pfSense Forum on how to install it.
Since this setup will be used as a home router, UPnP is essential for devices such as the PS3 and XBox 360. The best walk through for setting this up is found on Cqrite.com.
Lastly you will want to install your packages. In my case, Squid 2.79, RRD Summary, Sarg, and Cron were the only packages I needed.
Unexpected Problems
My home setup is pretty straight forward, but for unknown reasons, I had major issues with my Sony Blu-ray Player, Apple TV 2.0, and Playon (software for media streaming from a Windows box). I could not quite figure out exactly why all of these devices broke. Essentially media streaming failed to work period on the Blu-Ray Player. What was strange was that the PS3 was fine. It did not quite make sense why NetFlix worked fine on the PS3, but not on the Apple TV or Blu-ray Player. For a while I thought it was some combination of Squid in transparent mode, however after setting up Traffic Shaping in pfSense, it just decided to work one day. I even did a clean install and started from scratch, and I could never get it to work again. I ended up restoring my original configuration to fix it again. I narrowed down the problem to port 443, in which I could not authenticate with the streaming services.
One package that I wanted to try out was HAVP, which acts as an HTTP Proxy with an Anti-Virus scanner. This proved to be problematic for me. With HAVP my internet downloads were slower and unpredictable. HAVP requires lots of CPU speed, and in the end I found that it was not worth it for me. Plus, I could never get NetFlix streaming to work!
pfSense, The Home Router
pfSense was my Summer project this year and it was very interesting. The hardware I put together makes for a very fast home router and you certainly do notice the difference between pfSense and say your typical $200 router. The Squid proxy works well, in fact it is the best proxy solution that I can think of. However, Squid is still a proxy and proxies in general are difficult and can cause minor issues. The firewall aspect of pfSense is superb, I feel that this is the most secure system you can have for the price. The problem I ran into with pfSense is that it is not really meant for home networking. In a sense pfSense is not your typical plug-n-play device, so this makes it really hard for a non-networking person to work with. I find myself conflicted, because I really like everything that pfSense delivers, but feel that third party firmware such as DD-WRT is a better solution for home networking. The firewall is easier to work with on third party firmwares. The other area where pfSense proves too hard is Traffic Shaping. The QOS interfaces of Linux based routers (DD-WRT, NetGear, Tomato, etc) are much easier to work with and while pfSense actually has more options for managing traffic the setup requires more tweaking.
Up until now, my criticisms of pfSense are based on difficulties due to complexity, but one area where pfSense does come up lacking I think, is documentation. The pfSense.org website has a wiki and a forum, but some of the documentation on the wiki is incomplete and so the better resource ends up being the forum. There is an official book: pfSense: The Definitive Guide. The book is definitely a must if you are serious about working with pfSense. There is also a pfSense Cookbook, but that book is rather worthless as it basically covers screenshots of pfSense 2.0, but without any real context as to how the system actually works, so I would not recommend that book. What would work for pfSense is if there was a guide for home networking, that put together screenshots and recommendations for a typical home network router configuration.
Overall, pfSense can work as a home network router, but it does take time to become proficient with it and it is not as simple as your typical home router. After working with it for a few months, I find it to be a great solution for content filtering in schools. It is very affordable and if you are willing to spend the time, it can be a great solution for your security needs.
by WebKeyDesign | Aug 13, 2012 | Networking
Over the years, my home office has become a museum of sorts for wireless routers. There on a shelf sits my old reliable Linksys WRT54GS with upgraded antennas, next to it is a Linksys WRT350N, and lastly a Netgear WNR3500L. My current router sits in the office as well, that is a Netgear WNDR3800. The WNDR3800 is less than a year old and performs quite well, but then Western Digital announced they were getting into the wireless router business and announced the My Net N900, which is interesting since it includes 7 Gigabit Ethernet ports! But alas, I could not justify upgrading to a new wireless router in less than a year. With each router upgrade I have looked at three features: speed of actual CPU, internal memory, and DD-WRT compatibility. Home routers are essentially computers that route network traffic. The faster their CPU and more memory they have the faster they can operate. On slower ISP connections, you do not notice it as much, but once you upgrade your internet connection and add more devices to your home network, the more your router’s performance becomes impacted. Hence the WNDR3800 works better than the slower WNR3500L it replaced. I am mostly talking about the wired connections, since wireless speeds can vary and I tend to prefer wired connections. The more I thought about it, the more I came to the conclusion that I was really trying to upgrade the CPU and memory in my router; the wireless radio was adequate and the built-in 4-port switch was already being supplanted by a dedicated 8-port switch. In general wireless routers are a good value. They combine a wireless radio, a network switch, and routing capabilities for around $150 or less. The WD N900 looks like an even better value, given the 7-ports, but in my case, I wanted to separate the three main functions. Hence my search for the perfect home router began. (more…)
by WebKeyDesign | Mar 17, 2011 | Networking
Like most technical people, I find myself watching less and less television. There are simply not enough hours in the day to do my regular work, spend time with the kids, walk the dog, and watch TV. However, I still like to watch the occasional South Park episode on SouthParkStudios or turn on Hulu to watch a few episodes of Parks And Recreation and 30 Rock. For a time, I encountered a lot of issues with SouthParkStudios not streaming right. After doing some research, I found that Flash streaming is problematic if you have multiple computers at home. The solution is to modify your router settings. In DD-WRT routers go to NAT/QOS: Port Triggering and add port 1935. This will allow SouthParkStudios videos to work on all computers in your home network.
This fix should also work for Hulu Plus videos as well. However, the other problem that Hulu Plus has is that it defaults to 720 HD most of the time. If your internet connection cannot handle this speed, it is best to log into Hulu, go to your Account: Settings and under Player Settings change the Playback Quality. For me 480p works fine.
