MostlyChris

Thoughts that are my own.

Adding Mod_deflate to Apache

mod_deflate allows apache to compress files and deliver them to browsers that can use compressed files. This saves on bandwidth and renders page loads faster. It’s a fairly simple process to do this. You need to have write access to the apache config files and need to be able to create a new conf file. I am doing this on a Redhat server with Apache 2.2.3.

The first step is to make sure that mod_deflate.so is loaded in apache. You should see a line in httpd.conf similar to the following:

LoadModule deflate_module modules/mod_deflate.so

If that line does not exist, then you’ll need to get the mod_deflate object file. It is generally included with apache so all you need to do is locate it on your server and load it using the correct path for where it is.

The next step is to create a config file to house the mod_deflate configuration. On my server the conf.d directory is where all such files are stored. Apache then loads files in that directory if they end in ‘.conf’. I created a file called mod_deflate.conf.

In my configuration, I chose to compress all files except images, files such as gzip, tar, etc. that are already compressed and PDF files (they are already fairly compressed). The configuration for that looks like this:

SetOutputFilter DEFLATE
SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ \
    no-gzip dont-vary
SetEnvIfNoCase Request_URI \
    \.(?:exe|t?gz|zip|bz2|sit|rar)$ \
    no-gzip dont-vary
SetEnvIfNoCase Request_URI \.pdf$ no-gzip dont-vary

I then skip browsers that have problems with deflate.

BrowserMatch ^Mozilla/4 gzip-only-text/html
BrowserMatch ^Mozilla/4\.0[678] no-gzip
BrowserMatch \bMSIE !no-gzip !gzip-only-text/html

Finally, I set some logging to test with.

DeflateFilterNote Input input_info
DeflateFilterNote Output output_info
DeflateFilterNote Ratio ratio_info
LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate
CustomLog /var/log/httpd/deflate_log deflate

Make sure you restart apache after this so the new configuration file is used. Check your deflate_log to see that it is working.


Originally posted Aug 2010

InnoDB Shows Disabled

I recently ran across a replication error in mysql that looks a bit like the following:

Error: ‘Incorrect information in file… … .frm on query’

If you run the command:

mysql -e 'show engines'

You will see that the InnoDB engine is DISABLED. Since mysql ships with InnoDB enabled by default, this is a strange error.

What it turns out to be, at least in my case, is a corrupted ib_logfile. The fix is fairly simple.

Stop mysql.

/etc/init.d/mysql stop

Move or delete the ib_logfile0 and ib_logfile1 files.

mv ib_logfile0 ib_logfile0.bak

mv ib_logfile1 ib_logfile1.bak

Restart mysql.

/etc/init.d/mysql start

That’s it. Now issuing the show engines command results in a YES for InnoDB.


Originally posted Oct 2010

Nginx

There isn’t much information I can present in this post that hasn’t been hashed out somewhere else on the web. I’m just noting the fact that I have now converted two servers to running strictly nginx and shutting down apache.

I did this for a couple of reasons. First, on one of my servers, the load was approaching eight for a four core CPU and things were starting to crawl. This is a very busy server that runs terrabytes of web traffic each month. So far I have seen the load cut in half by running nginx. I have only run this for a couple of days of normal traffic so I’m waiting to see how it averages out. The config is fairly basic with some caching turned on.

http {

    ...

    open_file_cache max=1000 inactive=300s;
    open_file_cache_valid    30s;
    open_file_cache_min_uses 2;
    open_file_cache_errors   off;

    ...

The second reason I did this was for learning. Nginx is becoming more prevelant and I want to make sure I know how it works.

I installed nginx on the second server for more training and to see what it does to Wordpress. I have a couple of Wordpress sites on the server and they are quite slow.

I have used varnish for caching with an apache backend, but I needed to have it serve ssl content which varnish does not do natively. Nginx fits the bill here.

Try it out and of course YMMV.

Apache Server Hits

I still use these one liners on a regular basis and thought I’d port them over to my new blog.

Use this little one liner to grab the number of hits from the apache access logs.

DATE=`date +%d.%b.%Y`':';for i in `lsof -p $(ps faxwwwu|awk '$1 ~ /^root$/ && $11 ~ /httpd$/ {print $2|"head -1"}' )| awk '$9 ~ /access_log$/ {print $9| "sort -u"}'`; do echo "——-"$i"——"$DATE;awk '$4 ~ /^.'$DATE'/ {hit[substr($4,2,2)"\t"substr($4,14,2)"."substr($4,17,1)]++;ip[$1]++} END { for (i in hit) { print hit[i]"\t"i|"sort -k 3 -g"}; print ""}' $i;done

You can also grab a sorted list of IP addresses accessing sites by doing using this one liner.

DATE=`date +%d.%b.%Y`;for i in `lsof -p $(ps faxwwwu|awk '$1 ~ /^root$/ && $11 ~ /httpd$/ {print $2|"head -1"}' )| awk '$9 ~ /access_log$/ {print $9| "sort -u"}'`; do echo "——-"$i"——"$DATE;awk '$4 ~ /^.'$DATE'/ {hit[substr($4,2,2)"\t"substr($4,14,2)"."substr($4,17,1)]++;ip[$1]++} END { for (i in hit) { print hit[i]"\t"i|"sort -k 3 -n"}; print "";  for (i in ip) {print ip[i]"\t"i|"sort -n"}}' $i;done