Real time log file watcher

Have you ever wanted to be notified - in real-time - of an event in a log file?

This is a very efficient and effective way to do it.. all you need is inotifywait (in inotify-tools package, and of course inotify enabled kernel which is pretty standard these days).

#!/bin/bash
LOGFILE=/var/log/smdr.log
while inotifywait -qq -e modify $LOGFILE; do
x=`tail -n1 $LOGFILE | grep "something"`
if [ "$x" != "" ]; then
  echo $x | mailx -s "Something happened!" you@abc.org
fi
done

Automatic Proxy Howto

For business laptops, moving from home to the office - proxy settings can be a pain. You can use browser plugins like "foxy proxy" - but the best solution - bar none - is automatic proxy configuration.

To get automatic proxy configuration working (at the office where proxy is required), you need a webserver (eg. apache2), a name server (eg. bind9), and a DHCP server (eg. dhcp3).

Apache2 config
* Put javascript into setProxy.pac in you webserver root directory (change it for your domains/proxy)

function FindProxyForURL(url, host)

{

if (isPlainHostName(host) ||

dnsDomainIs(host, ".yourdomain1.ca") ||

dnsDomainIs(host, ".yourdomain2.ca"))

return "DIRECT";

else return "PROXY yourproxy.yourdomain.ca:8080; DIRECT";

}

* Link wpad.dat to it (ln -s setProxy.pac wpad.dat)
* add the following to apache mime.types file (on ubuntu just /etc/mime.types):

 application/x-ns-proxy-autoconfig dat

 application/x-javascript-config pac

* restart apache  (/etc/init.d/apache2 restart)

Bind9 config
* add the following to /var/cache/bind/db.yourdomain.ca

wpad IN CNAME apachehost.yourdomain.ca.

* restart named (don't forget to change serial number) - /etc/init.d/bind9 restart

In other words, "wpad" has to resolve to your apache server... so client can "ping wpad"

DHCP config
* add the following to /etc/dhcp3/dhcpd.conf:

 option wpad code 252 = text;

 option wpad "http://apachehost.yourdomain.ca/setProxy.pac ";

 (last space is MANDATORY - IE strips last char)

* restart dhcp


That should do it!

This works with KDE 4 - w/ Google Chrome (set in KDE network settings).  And with Firefox 3.0.14 with "Automatic proxy" option selected.

KDE 4 - Network Manager and KDE Wallet

Quite annoyingly the KDE 4.x Network Manager *requires* the KDE Wallet to be enabled to store passwords for secure networks (WEP/WPA).

If KDE Wallet is enabled - then it requires a password... so you login, and are immediately prompted for KDE Wallet password.

The only way I found to get around this is to set the KDE wallet password to be blank.

Click on KDE Wallet in tray, click kdewallet, goto File -> Change Password, and leave both blank.

IF a system is set to autologin, then I can understand the whole KDE wallet idea - but for users explicitly logging in with a password, KDE Wallet should be accessed transparently.

Linux performance tuning - I/O wait

Have a 64-bit ubuntu linux (hardy) system running 30 KDE desktop sessions (using nomachine's NX) on an IBM bladeserver HS21 with 2 quad-core Xeon's and 16G of RAM.  Many times a day, users would experience "freezes" within either firefox or evolution that would last up to about 15 seconds.. window completely unresponsive - then would resume execution.  This using the stock 2.6.24-24-generic ubuntu kernel with voluntary prempting.

After much performance monitoring with sar, and associating the timing of the freezes to sar data, we determined that I/O wait seemed to be the culprit.  Real memory usage {used - (cached+buffered)} only ever peaked around 12G, and CPUs were largely idle.

Disk is 10k rpm 300GB SAS 2.5" drive.. in RAID 1 (mirrored) using HS21 onboard LSI controller.

I/O Wait is primarily disk I/O but could be network I/O as well, so we decided to tune disk and network.

Filesystems is using ext3 and mounted with noatime.

Finding this link , I tried adding the mount options: noatime,nobh,data=writeback,commit=90

NOTE:  If on root filesystem, you must add "rootflags=nobh,data=writeback,commit=90" in grub.

After a reboot, and once full load was back on system, I realized a substantial reduction in average IO wait time (from ~50ms to ~25ms), and overall average % disk utilization (from ~15% to ~8% - statistics from iostat).

This was great improvement but there were still slight 1-2s freezing incidents at this point, so I additionally tweaked the network...

For network performance tuning I ran across this link and added the following to /etc/sysctl.conf:

net.ipv4.tcp_tw_reuse=1
net.ipv4.tcp_fin_timeout=30
net.ipv4.tcp_keepalive_time=1800
net.core.wmem_max=8388608
net.core.rmem_max=8388608
net.ipv4.tcp_rmem="4096 87380 8388608"
net.ipv4.tcp_wmem="4096 87380 8388608"

Together, these have stopped the "freezing" altogether.  There are still periods of slowness due to load, but now they are few and far between.

Locking Firefox 3 preferences in Ubuntu

Based on http://kirksblog.steffensenfamily.com/archives/24

1. Create a file called loadcustom.js in /usr/lib/firefox-3.0.x/defaults/preferences/, where 3.0.x will be something like 3.0.11 or 3.0.14 depending on your current version.

Put the following in /usr/lib/firefox-3.0.x/defaults/preferences/loadcustom.js

// tell firefox to load customized config file
pref("general.config.obscure_value", 0);
pref("general.config.filename", "firefox.cfg");

2. Create a file called firefox.cfg in /usr/lib/firefox-3.0.x/ with the following content (the first line in both files must start with a comment):

// Lock specific preferences in Firefox so that users cannot edit them
lockPref("app.update.enabled", false);
lockPref("network.proxy.http", "127.0.0.1");
lockPref("network.proxy.http_port", 8080);
lockPref("network.proxy.type", 1);
lockPref("network.proxy.no_proxies_on", "localhost, 127.0.0.1, 192.168.1.0/24");
lockPref("network.proxy.share_proxy_settings", true);
lockPref("browser.startup.homepage", "http://www.desiredhomepage.com/");

3. Restart Firefox, and the preferences should be locked down. You should be able to use this to lock down any setting in about:config

32-bit firefox on 64-bit ubuntu hardy

Problem: External applet which requires Java 1.5 on 64-bit system.. no firefox plugin for Java 1.5 in 64-bit exists to date (Oct. 6, 2009 - and not likely to happen!)

Solution: Run 32-bit firefox in chroot environment

Following https://help.ubuntu.com/community/DebootstrapChroot

sudo apt-get install debootstrap schroot

Edit /etc/schroot/schroot.conf and add the following:

[hardy_i386]

description=Ubuntu 8.04 Hardy for i386

location=/srv/chroot/hardy_i386

personality=linux32

root-users=bob

run-setup-scripts=true

run-exec-scripts=true

type=directory

users=alice,bob,charlie

$ sudo mkdir -p /srv/chroot/hardy_i386

$ sudo debootstrap --variant=buildd --arch i386 hardy /srv/chroot/hardy_i386 http://archive.ubuntu.com/ubuntu/

Will take a while.. downloads base packages

$ schroot -c hardy_i386 -u root

apt-get install ubuntu-minimal

Edit /etc/apt/sources.list and add "universe multiverse" to end of the line

apt-get update

apt-get install firefox sun-java5-plugin

In order to get NIS working, I had to install nis package and edit /etc/nsswitch.conf accordingly.

IF you don't want your "real" /home mounted in chroot environment, then edit /etc/schroot/setup.d/10mount and comment out (#) line with /home.

You'll have to create /home/username (mkdir /home/username; chown username /home/username)

Now exit.. and start again as user:

$ schroot -c hardy_i386

firefox -no-remote

NOTE: -no-remote is need in case 64-bit firefox is running already

Goto "about:plugins" to see if Java plugin is available.