Friday, October 23, 2009

Real time log file watcher

Have you ever wanted to be notified - in real-time - of an event in a log file?

This is a very efficient and effective way to do it.. all you need is inotifywait (in inotify-tools package, and of course inotify enabled kernel which is pretty standard these days).

while inotifywait -qq -e modify $LOGFILE; do
x=`tail -n1 $LOGFILE | grep "something"`
if [ "$x" != "" ]; then
  echo $x | mailx -s "Something happened!"

Wednesday, October 21, 2009

Automatic Proxy Howto

For business laptops, moving from home to the office - proxy settings can be a pain. You can use browser plugins like "foxy proxy" - but the best solution - bar none - is automatic proxy configuration.

To get automatic proxy configuration working (at the office where proxy is required), you need a webserver (eg. apache2), a name server (eg. bind9), and a DHCP server (eg. dhcp3).

Apache2 config
* Put javascript into setProxy.pac in you webserver root directory (change it for your domains/proxy)

function FindProxyForURL(url, host)
if (isPlainHostName(host) ||
dnsDomainIs(host, "") ||
dnsDomainIs(host, ""))
return "DIRECT";
else return "PROXY; DIRECT";

* Link wpad.dat to it (ln -s setProxy.pac wpad.dat)
* add the following to apache mime.types file (on ubuntu just /etc/mime.types):

 application/x-ns-proxy-autoconfig dat
 application/x-javascript-config pac

* restart apache  (/etc/init.d/apache2 restart)

Bind9 config
* add the following to /var/cache/bind/

* restart named (don't forget to change serial number) - /etc/init.d/bind9 restart

In other words, "wpad" has to resolve to your apache server... so client can "ping wpad"

DHCP config
* add the following to /etc/dhcp3/dhcpd.conf:
 option wpad code 252 = text;
 option wpad " ";
 (last space is MANDATORY - IE strips last char)

* restart dhcp

That should do it!

This works with KDE 4 - w/ Google Chrome (set in KDE network settings).  And with Firefox 3.0.14 with "Automatic proxy" option selected.

KDE 4 - Network Manager and KDE Wallet

Quite annoyingly the KDE 4.x Network Manager *requires* the KDE Wallet to be enabled to store passwords for secure networks (WEP/WPA).

If KDE Wallet is enabled - then it requires a password... so you login, and are immediately prompted for KDE Wallet password.

The only way I found to get around this is to set the KDE wallet password to be blank.

Click on KDE Wallet in tray, click kdewallet, goto File -> Change Password, and leave both blank.

IF a system is set to autologin, then I can understand the whole KDE wallet idea - but for users explicitly logging in with a password, KDE Wallet should be accessed transparently.

Monday, October 19, 2009

Linux performance tuning - I/O wait

Have a 64-bit ubuntu linux (hardy) system running 30 KDE desktop sessions (using nomachine's NX) on an IBM bladeserver HS21 with 2 quad-core Xeon's and 16G of RAM.  Many times a day, users would experience "freezes" within either firefox or evolution that would last up to about 15 seconds.. window completely unresponsive - then would resume execution.  This using the stock 2.6.24-24-generic ubuntu kernel with voluntary prempting.

After much performance monitoring with sar, and associating the timing of the freezes to sar data, we determined that I/O wait seemed to be the culprit.  Real memory usage {used - (cached+buffered)} only ever peaked around 12G, and CPUs were largely idle.

Disk is 10k rpm 300GB SAS 2.5" drive.. in RAID 1 (mirrored) using HS21 onboard LSI controller.

I/O Wait is primarily disk I/O but could be network I/O as well, so we decided to tune disk and network.

Filesystems is using ext3 and mounted with noatime.

Finding this link , I tried adding the mount options: noatime,nobh,data=writeback,commit=90

NOTE:  If on root filesystem, you must add "rootflags=nobh,data=writeback,commit=90" in grub.

After a reboot, and once full load was back on system, I realized a substantial reduction in average IO wait time (from ~50ms to ~25ms), and overall average % disk utilization (from ~15% to ~8% - statistics from iostat).

This was great improvement but there were still slight 1-2s freezing incidents at this point, so I additionally tweaked the network...

For network performance tuning I ran across this link and added the following to /etc/sysctl.conf:

net.ipv4.tcp_rmem="4096 87380 8388608"
net.ipv4.tcp_wmem="4096 87380 8388608"

Together, these have stopped the "freezing" altogether.  There are still periods of slowness due to load, but now they are few and far between.

Friday, October 9, 2009

Locking Firefox 3 preferences in Ubuntu

1. Create a file called loadcustom.js in /usr/lib/firefox-3.0.x/defaults/preferences/, where 3.0.x will be something like 3.0.11 or 3.0.14 depending on your current version.
Put the following in /usr/lib/firefox-3.0.x/defaults/preferences/loadcustom.js

// tell firefox to load customized config file
pref("general.config.obscure_value", 0);
pref("general.config.filename", "firefox.cfg");

2. Create a file called firefox.cfg in /usr/lib/firefox-3.0.x/ with the following content (the first line in both files must start with a comment):

// Lock specific preferences in Firefox so that users cannot edit them
lockPref("app.update.enabled", false);
lockPref("network.proxy.http", "");
lockPref("network.proxy.http_port", 8080);
lockPref("network.proxy.type", 1);
lockPref("network.proxy.no_proxies_on", "localhost,,");
lockPref("network.proxy.share_proxy_settings", true);
lockPref("browser.startup.homepage", "");

3. Restart Firefox, and the preferences should be locked down. You should be able to use this to lock down any setting in about:config

Tuesday, October 6, 2009

32-bit firefox on 64-bit ubuntu hardy

Problem: External applet which requires Java 1.5 on 64-bit system.. no firefox plugin for Java 1.5 in 64-bit exists to date (Oct. 6, 2009 - and not likely to happen!)

Solution: Run 32-bit firefox in chroot environment


sudo apt-get install debootstrap schroot

Edit /etc/schroot/schroot.conf and add the following:

description=Ubuntu 8.04 Hardy for i386

$ sudo mkdir -p /srv/chroot/hardy_i386
$ sudo debootstrap --variant=buildd --arch i386 hardy /srv/chroot/hardy_i386

Will take a while.. downloads base packages

$ schroot -c hardy_i386 -u root

apt-get install ubuntu-minimal

Edit /etc/apt/sources.list and add "universe multiverse" to end of the line

apt-get update
apt-get install firefox sun-java5-plugin

In order to get NIS working, I had to install nis package and edit /etc/nsswitch.conf accordingly.

IF you don't want your "real" /home mounted in chroot environment, then edit /etc/schroot/setup.d/10mount and comment out (#) line with /home.

You'll have to create /home/username (mkdir /home/username; chown username /home/username)

Now exit.. and start again as user:

$ schroot -c hardy_i386

firefox -no-remote

NOTE: -no-remote is need in case 64-bit firefox is running already

Goto "about:plugins" to see if Java plugin is available.