Serenade to a Cuckoo(o)

WinTV 1600 model 1199 and VirtualBox on linux

2011-09-17T10:24:00.000-04:00

Had a working VirtualBox WinXP guest on an Ubuntu Natty host.

Installed a WinTV 1600 video capture card, and after that, the VirtualBox WinXP guest would not boot.. got the message:

Failed to load VMMR0.r0 VERR_NO_MEMORY

After digging thru the logs, and noticing vmalloc errors - I decided to blacklist the cx18 modules (for capture card), and reboot. VirtualBox worked now, but then if I removed the blacklist on the cx18 modules, and tried to load the module - I got the same vmalloc errors.

Determined its a shortage of virtual memory in the linux kernel. Default is 128M.

Certain device drivers (modules) - in my case, a video capture card - require chunks of this virtual memory for device mapping.

Virtualbox obviously uses the same mechanism. You can have 16GB of RAM, but if you only have 128M of vm allocated, you are going to run into the same issue.

Solution is to pass the "vmalloc=<size>" parameter to your kernel on boot. Preferably in 64M chunks.

In ubuntu, edit /etc/default/grub, and add "vmalloc=192M" to the GRUB_CMDLINE_LINUX_DEFAULT list.

eg. GRUB_CMDLINE_LINUX_DEFAULT="vmalloc=192M quiet splash"

sudo update-grub

reboot

Now they play nicely together - and the overall system performance is much snappier.

Using curl with eXist to perform XQuery

2011-05-09T16:34:00.000-04:00

Simple trick to use HTTP POST with curl to do an XQuery - (I struggled for a bit with the Content-type)

curl --header "Content-type: text/xml" --data "@x.xml" http://hostname:8080/exist/rest/db/

where x.xml is something like:

<?xml version="1.0" encoding="UTF-8"?>
<query xmlns="http://exist.sourceforge.net/NS/exist" start="1" max="20">
<text>
<![CDATA[
some XQuery here...
]]>
</text>
<properties>
<property name="indent" value="yes"/>
</properties>
</query>

ext4 performance and the barrier option

2011-04-11T15:07:00.000-04:00

We recently implemented a 6-disk SAS RAID 1+0 /home ext4 filesystem on our application server to improve performance, and found that we still had a disk I/O bottleneck, with processor I/O wait averaging in the 3 to 4% range.

I came across this LWN article http://lwn.net/Articles/283161/ that talks about a 30% performance drop when the barrier option is enabled on an ext4 filesystem.

As of the time of the LWN article, barrier was not enabled by default on an ext4 filesystem. But somewhere along the kernel release line, barrier=1 (on) became the default for newly created ext4 filesystems.

Doing a "cat /proc/mounts" on my ubuntu 10.04 distro, I found that my ext4 filesystems indeed had the option barrier=1. So on April 3rd, I added barrier=0 to my fstab, and rebooted. The performance boost was quite dramatic.

Using measures of "average wait time", "% utilization" and "%IO wait", here is the before and after stats, measured and averaged on a typical work week 8am to 5pm...

With barrier=1:

Average wait time: 90 ms
Average Disk utilization %: 40%
Average CPU IO wait %: 3.5%

With barrier=0:

Average wait time: 2 ms
Average Disk utilization %: 2%
Average CPU IO wait %: 0.40%

Graphs depict the change more dramatically (comparing April 1st to April 4th):

Average Wait Time

Average % Utilization

Load Average

NOTE: This is not a "very busy" April 1st compared to a "very light" April 4th. Both are with a 30 user load, approx. 2300 processes. And every other day of the week in the comparison show the same dramatic improvement.

Of course one can argue that we are sacrificing data integrity for performance - but we do have a UPS backed system, and crashes are (thus far) non-existent.

Finding the WWN on an HS22 SAS connectivity module

2011-03-15T11:16:00.005-04:00

Recently obtained an IBM HS22 blade server with a SAS connectivity module, and was attempting to map a RAID volume in a disk storage module to this blade.. when I realized the AMM did not report the SAS ID (WWN) as it normally does with my HS21 blades.

I figured it was a firmware issue, so I upgraded the AMM firmware - but still nothing.

I found this link:http://www-947.ibm.com/support/entry/portal/docdisplay?brand=5000020&lndocid=MIGR-5083823 which scared me - seemed to indicate I had the wrong hardware.

I poked around the LSI BIOS and noted every SAS ID I could find.. and tried using those to map the volume, all to no avail.

It didn't help that the AMM reported the SAS module part number as 49Y8009, whereas I had ordered 43W4068.. thought there was a shipping error or something, but it turns out the 49Y8009 is just the internal IBM part number.

Thinking it was still related to firmware.. I upgraded the uEFI and the IMM firmware on the HS22. Again, this helped nothing.

In a desperate final attempt, I decided to delve into the SAS RAID controller itself.. accessed the web interface, selected "Monitor SAS Module", and voila! There, next to "Blade Slot Connection 2" was a SAS ID that was never presented anywhere else. I mapped the volume using this WWN, and now all is good.

What a nightmare to find though.. found forum posts with no answers: https://www.ibm.com/developerworks/forums/thread.jspa?messageID=14418964&#14418964

Someone from IBM, please update your documentation!!

Aggregating row data using user-defined aggregate in Informix IDS

2011-02-25T15:44:00.001-05:00

Implementing mysql group_concat() aggregate in Informix IDS 11.5 - thanks to Jonathan Leffler.

As DBA do:

CREATE FUNCTION gc_init(dummy varchar(255)) RETURNING LVARCHAR;
RETURN '';
END FUNCTION;

CREATE FUNCTION gc_iter(result LVARCHAR, value VARCHAR(255))
RETURNING LVARCHAR;
IF result = '' THEN
RETURN TRIM(value);
ELSE
RETURN result || ',' || TRIM(value);
END IF;
END FUNCTION;

CREATE FUNCTION gc_comb(partial1 LVARCHAR, partial2 LVARCHAR)
RETURNING LVARCHAR;
RETURN partial1 || ',' || partial2;
END FUNCTION;

CREATE FUNCTION gc_fini(final LVARCHAR) RETURNING LVARCHAR;
RETURN final;
END FUNCTION;

CREATE AGGREGATE group_concat
WITH (INIT = gc_init, ITER = gc_iter,
COMBINE = gc_comb, FINAL = gc_fini);

Then you need to grant access to the functions to users that need it:

grant execute on function gc_init to appsql;
grant execute on function gc_iter to appsql;
grant execute on function gc_comb to appsql;
grant execute on function gc_fini to appsql;

Example Use:

select ord_id, group_concat(product)
from ord
group by ord_id

xscreensaver and empathy presence notification

2010-11-25T15:44:00.001-05:00

xscreensaver doesn't support d-bus - and therefore when screensaver activates, empathy (IM client) doesn't change status to "Away", like it does with gnome-screensaver.

The xscreensaver man page includes a nice perl script which you can use to accomplish the same thing. So here is "watchx.pl" script:

#!/usr/bin/perl

my $blanked = 0;
open (IN, "xscreensaver-command -watch |");
while () {
     if (m/^(BLANK|LOCK)/) {
         if (!$blanked) {
             system "/usr/local/bin/empathy-away.sh";
             $blanked = 1;
         }
     } elsif (m/^UNBLANK/) {
         system "/usr/local/bin/empathy-avail.sh";
         $blanked = 0;
     }
}

Then, using dbus-send, you can set status in empathy.. I use the following scripts "empathy-away.sh":

#!/bin/bash
USER=`whoami`
service=`qdbus | grep $USER | sed -e "s/ //g"`
path=`echo $service | sed -e "s/\./\//g"`
dbus-send --dest=$service /$path org.freedesktop.Telepathy.Connection.Interface.SimplePresence.SetPresence string:"away" string:"Away from keyboard"

and "empathy-avail.sh":

#!/bin/bash
USER=`whoami`
service=`qdbus | grep $USER | sed -e "s/ //g"`
path=`echo $service | sed -e "s/\./\//g"`
dbus-send --dest=$service /$path org.freedesktop.Telepathy.Connection.Interface.SimplePresence.SetPresence string:"available" string:"Available"

NOTE: This is designed for one empathy account.. and that account name containing the users login name. Works on ubuntu lucid. Also - that dbus-send command is all on one line.

And to have it launch at login... create the following "watchx.desktop" file and place it in /etc/xdg/autostart:

[Desktop Entry]
Type=Application
Exec=/usr/local/bin/watchx.pl
Hidden=false
NoDisplay=false
X-GNOME-Autostart-enabled=true
Name[en_US]=Watch screen saver
Name=Watch screen saver
Comment[en_US]=Watch screen saver
Comment=Watch screen saver

NETCONSOLE to the rescue

2010-08-03T11:51:00.001-04:00

On a headless server, when you get a kernel panic and the system is totally locked up, and there is no sync of kernel log, its extremely frustrating trying to debug.

Discovered a wonderful kernel module/feature called NETCONSOLE. From netconsole.txt - "This module logs kernel printk messages over UDP allowing debugging of problem where disk logging fails and serial consoles are impractical."

Using this usefull howto as a basis I got this working on ubuntu 10.04, which already has netconsole built as a module.

On the source system where you want to trap the messages:

modprobe netconsole netconsole=<sport>@<saddr>/<dev>,<dport>@<daddr>/<dmac>

where sport is the source port, saddr is the source address, dev is interface, dport is destination port, daddr is destination address and dmac is destination mac address.

On the remote end, install netcat, and do:

netcat -u -l -p<dport>  >> /var/log/netconsole

So easy.. and so useful..

Note: This is different than remote syslogd - which will capture kernel messages over the network - since netconsole is at the kernel level and thus more capable of capturing panics.

Migrating from Palm Treo 680 to Nokia E71 OTA with Funambol

2010-05-07T13:50:00.000-04:00

Lets say you want to be able to migrate contacts, calendars, tasks and notes from your Treo to a Nokia E71 (or other newer S60 device), and you don't want to go the "Windows/Palm desktop - export/Outlook import -> Nokia Suite -> device" route. I don't use Windows, and I wanted an OTA (over-the-air) sync solution.

This turned out to be quite simple, thanks to Funambol and SyncML

There are 3 parts to this - (1) funambol server setup, (2) Funambol client on E71, and (3) Synthesis SyncML client on Palm Treo

Funambol Server
https://www.forge.funambol.org/download/

Simply download, install and launch
-you must have an internet accessible interface

Next, launch the Funambol Admin Tool : /opt/Funambol/admin/bin/funamboladmin
-login.. open up server properties, and set the URL: http://xx.xx.xx.xx:8080/funambol/ds
  (where xx.xx.xx.xx is your public IP)
-click on save

(you may also want to change the admin password)

Done!

Funambol Client on Nokia E71
-you can download from funambol.org or via www.ovi.mobi
-install and configure 3 things
  -username
  -password
  -URL (use same as established on server obviously)

note: by default, the funambol server does auto-provisioning, so the username/password above is going to be "created" on the server, so choose whatever you wish.

-perform initial sync

Synthesis SyncML client
You can download 30-day trial here:
http://www.synthesis.ch/dl_client.php?lang=e&lay=desk&bp=CPDA&pv=PALM#dlds_CPDA1

-start client, goto Options -> Settings...
-set SyncML Server URL (as per above)
-set username/password (as per above)
-click "options" button
  -Server version -> SyncML 1.2 /OMA
  -OK
-select databases to sync - and set sync modes to "slow"

NOTE: database names are set for a specific "SyncML Server", you need to change them using "more.." buttons

Contacts: server path: card
Events: server path: event
Tasks: server path: task
Notes: server path: note

-perform sync - this will push everything OTA to funambol server

Final Step

-re-sync your Nokia to grab data from funambol server OTA

Trixbox 2.8 CE install via USB

2010-04-12T13:35:00.000-04:00

Bought an Acer Aspire R1600 for a trixbox install. Nice small form factor, but no cdrom, so the only install options are via USB or network.

I've used unetbootin for converting ISOs to make bootable USBs before.. never with a problem. But with trixbox.. not so simple - things "seem" to install, but the system ends up being very broken, as the install scripts won't complete properly. There is hard-coded reference to cdrom device in the install config files (anaconda/kickstart). In 2010, its almost inconceivable that there is no install from USB option in trixbox (or asterisknow). I've been administering linux systems (many distros) since 1996, and I've never had such a difficult time with an install.

I could not find any definitive solution anywhere (even tried the trixbox 2.6 usb install howto, which didn't work - plus its ludicrous that you should need Windows in order to install linux), so I struggled through dozens of reboots until I *finally* got a working trixbox system.

Unfortunately the Aspire R1600 has some proprietary nvidia NIC, which is not supported by the forcedeth 0.60 module which is supplied with trixbox. I had to find a 0.62 version of kmod-forcedeth which installed on trixbox. But this is no fault of trixbox, but of Acer for making such a stupid decision on NIC.

Ok, so this is how I did it. If anyone has a simpler solution, please comment.

1) Use unetbootin ( http://unetbootin.sourceforge.net/ ) with trixbox 2.8 ce ISO to create bootable USB drive
2) remove the trixbox directory from USB drive (its not needed, and I only had 1G, so needed room)
3) copy the trixbox 2.8 ce ISO to the USB drive
4) on USB drive, edit isolinux/ks.cfg comment out "cdrom" line

5) place USB drive in target system, and configure BIOS to boot from USB, and boot
6) when grub menu presents, scroll down to "default", and press [Tab] to edit command line
- change "ks=cdrom:/isolinux/ks.cfg" to "ks=hd:sdb1:/isolinux/ks.cfg" and press enter to boot

NOTE: this should work for most.. if you have one drive as sda.. USB drive typcially becomes sdb

7) installer will proceed, but may try to read from sdb1 prior to kernel detecting device.. it will prompt for kickstart file.. you can just retry.. check Alt-F4 for sdb detection.. (Alt-F1 to get back). Next it will ask for location of ISO... select /dev/sdb1 and continue with install

For me there was complaint about kmod-dahdi, but in the end.. all install scripts executed fine, and the system was completely operational.

NOTE: I tried the same type of solution with AsteriskNOW, to no avail.. there was a fatal error as it tried to re-mount a mounted filesystem, and installation aborted.

Extracting OpenOffice.org Calc cells from the command line

2010-03-22T16:39:00.001-04:00

In theory it should be simple... get cell value from ODS file, sheet x, row y, column z.

With OpenDocument format, content.xml stores spreadsheet data with the following schema:

<table:table>
<table:table-row>
    <table:table-cell> ....

So a simple XPath expression like //table:table[1]/table:table-row[5]/table:table-cell[3] should give me "Sheet1:C5".

With xmlstarlet command would be:

xmlstarlet sel -N office="urn:oasis:names:tc:opendocument:xmlns:office:1.0" -N table="urn:oasis:names:tc:opendocument:xmlns:table:1.0" -t -v "//table:table[1]/table:table-row[5]/table:table-cell[3]" content.xml

BUT.. in order to save space, blank rows and columns are not stored.. instead an attribute is used to indicate how many blank rows or columns, so you can see something like <table:table-row table:number-rows-repeated=10/>.

This breaks the easy XPath expression above, and makes it difficult to find specific co-ordinates.

I've developed an XSLT transformation that will take content.xml and expand it to be indexable.

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
   xmlns:office="urn:oasis:names:tc:opendocument:xmlns:office:1.0"
   xmlns:table="urn:oasis:names:tc:opendocument:xmlns:table:1.0"
   xmlns:text="urn:oasis:names:tc:opendocument:xmlns:text:1.0">

<xsl:output method = "xml" indent = "yes" encoding = "UTF-8" omit-xml-declaration = "no"/>

<xsl:template match="table:table-cell">

        <xsl:choose>
        <xsl:when test="@table:number-columns-repeated">
          <xsl:variable name="numcols" select="number(@table:number-columns-repeated)"/>
          <xsl:choose>
          <xsl:when test="$numcols > 1000">
             <xsl:call-template name="processcol">
           <xsl:with-param name="currentCol" select="0"/>
           <xsl:with-param name="totalCols" select="1"/>
             </xsl:call-template>
          </xsl:when>
          <xsl:otherwise>
             <xsl:call-template name="processcol">
           <xsl:with-param name="currentCol" select="0"/>
           <xsl:with-param name="totalCols" select="$numcols"/>
             </xsl:call-template>
          </xsl:otherwise>
          </xsl:choose>
        </xsl:when>
        <xsl:otherwise>
          <xsl:copy-of select="."/>
        </xsl:otherwise>
        </xsl:choose>
</xsl:template>

<xsl:template match="table:table-row">

        <xsl:choose>
        <xsl:when test="@table:number-rows-repeated">
      <xsl:variable name="numrows" select="number(@table:number-rows-repeated)"/>
          <xsl:choose>
          <xsl:when test="$numrows > 3000">
             <xsl:call-template name="processrow">
           <xsl:with-param name="currentRow" select="0"/>
           <xsl:with-param name="totalRows" select="1"/>
              </xsl:call-template>
          </xsl:when>
          <xsl:otherwise>
             <xsl:call-template name="processrow">
           <xsl:with-param name="currentRow" select="0"/>
           <xsl:with-param name="totalRows" select="$numrows"/>
            </xsl:call-template>
          </xsl:otherwise>
          </xsl:choose>
   </xsl:when>
        <xsl:otherwise>
          <xsl:call-template name="processrow">
       <xsl:with-param name="currentRow" select="0"/>
       <xsl:with-param name="totalRows" select="1"/>
          </xsl:call-template>
        </xsl:otherwise>
        </xsl:choose>

</xsl:template>

<xsl:template match="@*|*">
   <xsl:copy>
     <xsl:apply-templates select="@*|node()"/>
   </xsl:copy>
</xsl:template>

<xsl:template name="processrow">
   <xsl:param name="currentRow"/>
   <xsl:param name="totalRows"/>
        <xsl:choose>
   <xsl:when test="$currentRow < $totalRows">
<xsl:copy>
<xsl:apply-templates select="table:table-cell"/>
</xsl:copy>
        <xsl:call-template name="processrow">
       <xsl:with-param name="currentRow" select="$currentRow + 1"/>
       <xsl:with-param name="totalRows" select="$totalRows"/>
        </xsl:call-template>
   </xsl:when>
   </xsl:choose>
</xsl:template>

<xsl:template name="processcol">
   <xsl:param name="currentCol"/>
   <xsl:param name="totalCols"/>
        <xsl:choose>
   <xsl:when test="$currentCol < $totalCols">
<xsl:copy-of select="."/>
        <xsl:call-template name="processcol">
       <xsl:with-param name="currentCol" select="$currentCol + 1"/>
       <xsl:with-param name="totalCols" select="$totalCols"/>
        </xsl:call-template>
   </xsl:when>
   </xsl:choose>
</xsl:template>

</xsl:stylesheet>

This can be called simply using : xmlstarlet tr /usr/local/bin/expand.xslt content.xml > output.xml

and then queried using: xmlstarlet sel -N office="urn:oasis:names:tc:opendocument:xmlns:office:1.0" -N table="urn:oasis:names:tc:opendocument:xmlns:table:1.0" -t -v "//table:table[1]/table:table-row[5]/table:table-cell[3]" output.xml

NOTE: Since this is using recursive XSLT, I put a cap on 3000 repeated rows, and 1000 repeated columns. So this will work *unless* you have a spreadsheet with over 3000 blank rows (and you want to query row 3005), OR over 1000 blank columns (and you want to query column 1001) .

Putting this all together in a bash script we have:

#!/bin/bash

# $1 ods file
# $2 sheet #
# $3 row #
# $4 column #

tmpdir=/tmp/ods.$$
mkdir $tmpdir
cp "$1" $tmpdir
cd $tmpdir
IFS="|"
filename=`basename "$1"`

unzip -qq "$tmpdir/$filename"

xmlstarlet tr /usr/local/bin/expand.xslt content.xml > output.xml

xmlstarlet sel -N office="urn:oasis:names:tc:opendocument:xmlns:office:1.0" -N table="urn:oasis:names:tc:opendocument:xmlns:table:1.0" -t -v "//table:table[$2]/table:table-row[$3]/table:table-cell[$4]" output.xml

cd - &>/dev/null

rm -rf $tmpdir

Invocation looks like this:

oo_extract_cell.sh Test.ods 1 5 3

PXE chaining to modernize legacy PXE NICs

2010-02-17T16:04:00.000-05:00

Ok - so we have faulty/legacy PXE implementation in some onboard NICs, that don't support "next-server"... Here is how to fix...   VERY cool! It basically replaces the faulty PXE with a modern gPXE.

Following these instructions.. build undionly.kpxe and place in local /tftpboot/

http://etherboot.org/wiki/pxechaining

Then do something like the following in dhcpd.conf:

group {
        allow booting;
        option root-path "192.168.202.4:/opt/ltsp/i386";

        host ws042 {
                hardware ethernet 00:06:29:DF:2C:52;
                fixed-address ws042.dainty.ca;
                option host-name "ws042";

                if substring (option vendor-class-identifier, 0, 9) = "PXEClient" {
                        if exists user-class and option user-class = "gPXE" {
                                next-server 192.168.202.4;
                                filename "/ltsp/i386/pxelinux.0"; #secondary PXEclient
                        } else {
                        filename "/tftpboot/undionly.kpxe"; #first boot.. loads gPXE
                        }
                } else {
                        next-server 192.168.202.4;
                        filename "/ltsp/i386/nbi.img"; #normal boot image
                 }

        }
}

Real time log file watcher

2009-10-23T08:34:00.000-04:00

Have you ever wanted to be notified - in real-time - of an event in a log file?

This is a very efficient and effective way to do it.. all you need is inotifywait (in inotify-tools package, and of course inotify enabled kernel which is pretty standard these days).

#!/bin/bash
LOGFILE=/var/log/smdr.log
while inotifywait -qq -e modify $LOGFILE; do
x=`tail -n1 $LOGFILE | grep "something"`
if [ "$x" != "" ]; then
echo $x | mailx -s "Something happened!" you@abc.org
fi
done

Automatic Proxy Howto

2009-10-21T11:17:00.000-04:00

For business laptops, moving from home to the office - proxy settings can be a pain. You can use browser plugins like "foxy proxy" - but the best solution - bar none - is automatic proxy configuration.

To get automatic proxy configuration working (at the office where proxy is required), you need a webserver (eg. apache2), a name server (eg. bind9), and a DHCP server (eg. dhcp3).

Apache2 config
* Put javascript into setProxy.pac in you webserver root directory (change it for your domains/proxy)

function FindProxyForURL(url, host)

{

if (isPlainHostName(host) ||

dnsDomainIs(host, ".yourdomain1.ca") ||

dnsDomainIs(host, ".yourdomain2.ca"))

return "DIRECT";

else return "PROXY yourproxy.yourdomain.ca:8080; DIRECT";

}

* Link wpad.dat to it (ln -s setProxy.pac wpad.dat)
* add the following to apache mime.types file (on ubuntu just /etc/mime.types):

application/x-ns-proxy-autoconfig dat

application/x-javascript-config pac

* restart apache (/etc/init.d/apache2 restart)

Bind9 config
* add the following to /var/cache/bind/db.yourdomain.ca

wpad IN CNAME apachehost.yourdomain.ca.

* restart named (don't forget to change serial number) - /etc/init.d/bind9 restart

In other words, "wpad" has to resolve to your apache server... so client can "ping wpad"

DHCP config
* add the following to /etc/dhcp3/dhcpd.conf:

option wpad code 252 = text;

option wpad "http://apachehost.yourdomain.ca/setProxy.pac ";

(last space is MANDATORY - IE strips last char)

* restart dhcp

That should do it!

This works with KDE 4 - w/ Google Chrome (set in KDE network settings). And with Firefox 3.0.14 with "Automatic proxy" option selected.

KDE 4 - Network Manager and KDE Wallet

2009-10-21T09:56:00.000-04:00

Quite annoyingly the KDE 4.x Network Manager *requires* the KDE Wallet to be enabled to store passwords for secure networks (WEP/WPA).

If KDE Wallet is enabled - then it requires a password... so you login, and are immediately prompted for KDE Wallet password.

The only way I found to get around this is to set the KDE wallet password to be blank.

Click on KDE Wallet in tray, click kdewallet, goto File -> Change Password, and leave both blank.

IF a system is set to autologin, then I can understand the whole KDE wallet idea - but for users explicitly logging in with a password, KDE Wallet should be accessed transparently.

Linux performance tuning - I/O wait

2009-10-19T16:09:00.007-04:00

Have a 64-bit ubuntu linux (hardy) system running 30 KDE desktop sessions (using nomachine's NX) on an IBM bladeserver HS21 with 2 quad-core Xeon's and 16G of RAM. Many times a day, users would experience "freezes" within either firefox or evolution that would last up to about 15 seconds.. window completely unresponsive - then would resume execution. This using the stock 2.6.24-24-generic ubuntu kernel with voluntary prempting.

After much performance monitoring with sar, and associating the timing of the freezes to sar data, we determined that I/O wait seemed to be the culprit. Real memory usage {used - (cached+buffered)} only ever peaked around 12G, and CPUs were largely idle.

Disk is 10k rpm 300GB SAS 2.5" drive.. in RAID 1 (mirrored) using HS21 onboard LSI controller.

I/O Wait is primarily disk I/O but could be network I/O as well, so we decided to tune disk and network.

Filesystems is using ext3 and mounted with noatime.

Finding this link , I tried adding the mount options: noatime,nobh,data=writeback,commit=90

NOTE: If on root filesystem, you must add "rootflags=nobh,data=writeback,commit=90" in grub.

After a reboot, and once full load was back on system, I realized a substantial reduction in average IO wait time (from ~50ms to ~25ms), and overall average % disk utilization (from ~15% to ~8% - statistics from iostat).

This was great improvement but there were still slight 1-2s freezing incidents at this point, so I additionally tweaked the network...

For network performance tuning I ran across this link and added the following to /etc/sysctl.conf:

net.ipv4.tcp_tw_reuse=1
net.ipv4.tcp_fin_timeout=30
net.ipv4.tcp_keepalive_time=1800
net.core.wmem_max=8388608
net.core.rmem_max=8388608
net.ipv4.tcp_rmem="4096 87380 8388608"
net.ipv4.tcp_wmem="4096 87380 8388608"

Together, these have stopped the "freezing" altogether. There are still periods of slowness due to load, but now they are few and far between.

Locking Firefox 3 preferences in Ubuntu

2009-10-09T14:17:00.006-04:00

Based on http://kirksblog.steffensenfamily.com/archives/24

1. Create a file called loadcustom.js in /usr/lib/firefox-3.0.x/defaults/preferences/, where 3.0.x will be something like 3.0.11 or 3.0.14 depending on your current version.

Put the following in /usr/lib/firefox-3.0.x/defaults/preferences/loadcustom.js

// tell firefox to load customized config file
pref("general.config.obscure_value", 0);
pref("general.config.filename", "firefox.cfg");

2. Create a file called firefox.cfg in /usr/lib/firefox-3.0.x/ with the following content (the first line in both files must start with a comment):

// Lock specific preferences in Firefox so that users cannot edit them
lockPref("app.update.enabled", false);
lockPref("network.proxy.http", "127.0.0.1");
lockPref("network.proxy.http_port", 8080);
lockPref("network.proxy.type", 1);
lockPref("network.proxy.no_proxies_on", "localhost, 127.0.0.1, 192.168.1.0/24");
lockPref("network.proxy.share_proxy_settings", true);
lockPref("browser.startup.homepage", "http://www.desiredhomepage.com/");

3. Restart Firefox, and the preferences should be locked down. You should be able to use this to lock down any setting in about:config

32-bit firefox on 64-bit ubuntu hardy

2009-10-06T13:49:00.004-04:00

Problem: External applet which requires Java 1.5 on 64-bit system.. no firefox plugin for Java 1.5 in 64-bit exists to date (Oct. 6, 2009 - and not likely to happen!)

Solution: Run 32-bit firefox in chroot environment

Following https://help.ubuntu.com/community/DebootstrapChroot

sudo apt-get install debootstrap schroot

Edit /etc/schroot/schroot.conf and add the following:

[hardy_i386]

description=Ubuntu 8.04 Hardy for i386

location=/srv/chroot/hardy_i386

personality=linux32

root-users=bob

run-setup-scripts=true

run-exec-scripts=true

type=directory

users=alice,bob,charlie

$ sudo mkdir -p /srv/chroot/hardy_i386

$ sudo debootstrap --variant=buildd --arch i386 hardy /srv/chroot/hardy_i386 http://archive.ubuntu.com/ubuntu/

Will take a while.. downloads base packages

$ schroot -c hardy_i386 -u root

apt-get install ubuntu-minimal

Edit /etc/apt/sources.list and add "universe multiverse" to end of the line

apt-get update

apt-get install firefox sun-java5-plugin

In order to get NIS working, I had to install nis package and edit /etc/nsswitch.conf accordingly.

IF you don't want your "real" /home mounted in chroot environment, then edit /etc/schroot/setup.d/10mount and comment out (#) line with /home.

You'll have to create /home/username (mkdir /home/username; chown username /home/username)

Now exit.. and start again as user:

$ schroot -c hardy_i386

firefox -no-remote

NOTE: -no-remote is need in case 64-bit firefox is running already

Goto "about:plugins" to see if Java plugin is available.

Manually removing ubuntu or debian package

2009-07-22T08:40:00.004-04:00

Ran into an issue where a package was only partially removed from an ubuntu system - have no clue why.. but it was in a loop.. "apt-get -f install" would try to remove packagex and fail due to not existant post-remove script.

Thanks to this post you can manually remove an ubuntu/debian package doing the following:

First try:

dpkg -P packagex

Likely this will still fail so do:

dpkg -L packagex

this will give you a list of files to delete when the next step is done.

Then go to the directory:

/var/lib/dpkg/info

and delete the file:

packagex.postrm

and do:

apt-get remove --purge packagex
OR
apt-get -f install

to finish up, remove all files you found in the 'dpkg -L' step, and directories specific to packagex.

Then run apt-get update

rsync and directory renaming

2009-07-06T10:48:00.003-04:00

rsync is an extremely useful tool - but its one shortcoming is when it comes to file and/or directory renaming. I am using it to mirror file systems using "rsync -xHau --delete /home remotesystem:/somedir".

If a file or directory gets renamed on the source host, the entire file or directory gets re-copied.

For files, there is the --fuzzy option, and there is a patch for "detect-renamed" - but files are less of a problem for me.

Directory renames are not addressed by any patch or option that I can find, and often results in a much higher transfer volume (especially if a high-level directory is renamed).

I've developed a strategy that works great in my scenario.. essentially, after every rsync of a file system, a .rsyncname file is created in every directory (to a configurable max depth) simply holding the name of the directory itself. Before each rsync, all the .rsyncname files are compared to the directory name - and if different, a rename is triggered on the remote system. This requires top-down processing so I opted for a breadth-first search algorithm (cannot rename a leaf node if higher node needs renaming).

There are 3 scripts involved, which I called rsyncstorename, rsyncbasename, and rsyncrename, all of which I put in /usr/local/bin on the "source" host. (not required on the mirror host)

rsyncstorename:

#!/bin/bash

dir=$*
x=`cat "$dir/.rsyncname"`
y=`basename "$dir"`
if [ "$x" != "$y" ]; then
echo $y > "$dir/.rsyncname"
fi

This is used by rsyncbasename, and not called directly. The comparison adds a little overhead, but it saves on rsync time overall. Originally it was just creating the file every run - but each rsync sent every .rsyncname file to remote host. With this method - its one-time cost.

rsyncbasename:

#!/bin/bash

#$1 = path
#$2 = max depth

find $1 -maxdepth $2 -mount -type d -exec /usr/local/bin/rsyncstorename {} \;

Very simple.. pass the file system path and the desired max depth (eg. "rsyncbasename /home 4"), but is called from rsyncrename script. Doesn't cross file-system boundaries.

rsyncrename:

#!/bin/bash

#$1 = path
#$2 = max depth
remotepath=/mnt/disk
remotehost=somehost

r ()
{
let level=$3+1
if [ $level -gt $4 ]; then return 0; fi
cd "$1"
for d in *; do
if [ -d "$d" ]; then
if [ -L "$d" ]; then
echo "skipping symbolic link"
else
if [ -f "$d/.rsyncname" ]; then
x=`cat "$d/.rsyncname"`
y=`basename "$d"`
if [ "$x" != "$y" ]; then
echo renaming $remotepath$2/$x to $remotepath$2/$d
ssh $remotehost mv "$remotepath$2/$x" "$remotepath$2/$d"
fi
fi
fi
fi;
done
for d in *; do
if [ -d "$d" ]; then
(r "$d" "$2/$d" $level $4)
fi;
done
}

r "$1" "$1" 0 "$2"
rsync -xHau --delete $1 $remotehost:$remotepath`dirname $1`
/usr/local/bin/rsyncbasename $1 $2

Arguments are again path and depth (eg. "rsyncrename /home 4"). Change remotehost and remotepath to whatever is required. NOTE: Script expects file system mirror on remotehost:remotepath - ie. if you wish to rsync a sub-directory, eg. /home/userA, then remotehost:remotepath/home must exist. This script combines the rsync and the renaming. Note the sequence, renaming, rsync, then rsyncbasename. Will only work on 2nd iteration, since first time no renaming will occur and .rsyncname files will be created.

BTW.. bash based breadth-first-search algorithm grabbed from here.

NOTE: This method does not handle if a directory branch is relocated (eg. "mv /home/userA/mail /home/userB/mail2") .. rsync will handle this, but everything in the branch will be copied.

4GL comma split function

2009-06-26T15:56:00.003-04:00

not really.. but if you want to iterate over a delimited string via 4gl, you can do the following:

(parse_str function adapted from http://www.tek-tips.com/faqs.cfm?fid=2364 )

# This function searches string for an instance of char_delim
# starting at position cnt and returns the string ending at the next
# char_delim or the end of string
######################################################################
function parse_str(char_delim, cnt, string)
######################################################################
define
char_delim CHAR(1), # character delimiter
cnt SMALLINT, # postion in string to start counting
string CHAR(300), # string to search
ret_string CHAR(40), # string returned
i, # index
x, # counter
string_len SMALLINT # string length

LET string_len = LENGTH(string)

IF string_len = 0 # got a null string
THEN
RETURN "",0
END IF

IF string[cnt] = char_delim
THEN
LET cnt = cnt + 1 # get off the delimiter
END IF
IF cnt > string_len
THEN # return when the end of the string is reached
RETURN "",0
END IF

LET x = 1
FOR i = cnt TO string_len
IF string[i] = char_delim
THEN
EXIT FOR
END IF
LET ret_string[x] = string[i] # save a character
LET x = x + 1
END FOR

return ret_string, i # return the positon ended
end function

function split(f_orig)
define f_orig,f_str char(100), x, f_pos smallint
let x = length(f_orig)
let f_pos = 1
while (f_pos > 0 and f_pos < x)
call parse_str(",",f_pos,f_orig) returning f_str, f_pos
display f_str, f_pos
end while
end function

main
call split("some,comma,delimited,string")
end main

Aubit4GL: connecting to mysql and informix concurrently

2009-06-26T08:04:00.003-04:00

Since database connections seems to be compile time, and you have to set A4GL_SQLTYPE in order to compile against database X - I wonder how you can open concurrent connections.. I cannot find any documentation on how to do this - but according to http://aubit4gl.sourceforge.net/aubit4gldoc/manual/html/features_summary.html it maybe that you have to use ODBC connections to each instead of native??

It seems that "open session" and "use session" can be used to dynamically switch at runtime. But compile time seems to be the kicker.. especially if you use "define x like table.column" in 4gl, unless it allows "define x like db.table.column" or searches both databases.

Aubit4GL: mysql connection specification

2009-06-25T10:46:00.003-04:00

Since this is not well documented on Aubit4gl site, posting here:

To connect to mysql database:

Make sure your mysql server is not bound to localhost - even local connections are via hostname
Make sure you can connect to mysql server.. grant at least select privs using "grant select on db.* to 'test'@'hostname' identified by 'test';" NOTE: localhost privs are not enough - connection thru hostname.
Do "export MYSQL_SERVER=hostname"
Set/export the following environment variables:

A4GL_SQLUID=test #whatever connection user is
A4GL_SQLPWD=test #whatever connection password is
A4GL_LEXDIALECT=INFORMIX
A4GL_LEXTYPE=C
A4GL_SQLTYPE=mysql

And now take the simple.4gl program:

database test # of whatever your database name is

main
display "Hello World"
end main

Compile with : aubit 4glpc simple.4gl -o simple.4ae

If it compiles - database connection is ok!

Thanks to Mike's comment below - using .aubit4gl.acl to store database:user:pw DOES work.. just be sure to "unset A4GL_SQLUID" and "unset A4GL_SQLPWD".

Logitech Quickcam Pro 9000 on ubuntu hardy - no go

2009-06-25T07:44:00.000-04:00

On initial plugin:

[ 102.851571] usb 3-1: new full speed USB device using uhci_hcd and address 2
[ 44.040462] usb 3-1: device descriptor read/64, error -71
[ 102.933128] usb 3-1: device descriptor read/64, error -71
[ 102.939599] usb 3-1: new full speed USB device using uhci_hcd and address 3
[ 102.945009] usb 3-1: device descriptor read/64, error -71
[ 102.970417] usb 3-1: device descriptor read/64, error -71
[ 102.983105] usb 3-1: new full speed USB device using uhci_hcd and address 4
[ 103.020497] usb 3-1: device not accepting address 4, error -71
[ 103.025529] usb 3-1: new full speed USB device using uhci_hcd and address 5
[ 103.060941] usb 3-1: device not accepting address 5, error -71
[ 116.499098] usb 5-5: new high speed USB device using ehci_hcd and address 5
[ 116.627482] usb 5-5: configuration #1 chosen from 1 choice

and

lsusb -d 046d:0990 -v | grep bcdDevice
bcdDevice 0.08

which according to http://www.quickcamteam.net/documentation/faq/how-many-versions-of-the-quickcam-pro-9000-are-there says its third generation with buggy firmware.

If I launch cheese - system hangs completely.

According to http://linux-uvc.berlios.de/, linux 2.6.26 includes linux uvc driver natively.. but hardy is 2.6.24.

Downloaded the driver from http://linuxtv.org/hg/~pinchartl/uvcvideo/ (gz link) and then..

tar zxvf uvcvideo-bff77ec33116.tar.gz
cd uvcvideo-bff77ec33116/
make # and go have a coffee
sudo make install
reboot

dmesg now shows:

[ 105.937744] usb 5-5: new high speed USB device using ehci_hcd and address 3
[ 106.197507] usb 5-5: configuration #1 chosen from 1 choice
[ 107.127908] Linux video capture interface: v2.00
[ 107.135021] usbcore: registered new interface driver snd-usb-audio
[ 45.915452] uvcvideo: Found UVC 1.00 device (046d:0990)
[ 45.932166] input: UVC Camera (046d:0990) as /devices/pci0000:00/0000:00:1d.7/usb5/5-5/5-5:1.0/input/input10
[ 45.962220] usbcore: registered new interface driver uvcvideo
[ 45.962226] USB Video Class driver (v0.1.0)

cheese shows no output
skype neither
dmesg now shows

[ 177.592621] uvcvideo: Failed to set UVC probe control : -110 (exp. 26).
[ 177.976920] uvcvideo: Failed to set UVC probe control : -110 (exp. 26).
[ 178.401788] uvcvideo: Failed to set UVC probe control : -110 (exp. 26).
[ 178.807251] uvcvideo: Failed to set UVC probe control : -110 (exp. 26).
[ 179.193500] uvcvideo: Failed to set UVC probe control : -110 (exp. 26).
[ 179.743456] uvcvideo: Failed to set UVC probe control : -110 (exp. 26).
[ 180.154183] uvcvideo: Failed to set UVC probe control : -110 (exp. 26).
[ 180.655020] uvcvideo: Failed to set UVC probe control : -110 (exp. 26).

skype reports

Skype V4L2: Could not find a suitable capture format
Skype V4L2: Could not find a suitable capture format

Aborting effort for now..

mysqldump to informix DS

2009-06-23T13:01:00.000-04:00

Problem: Need to access mysql 5.0 data from 4js BDL (I4gl)

Solution attempts:

generate 4js dynamic VM with mysql support.. no mysql 5.0 support (only 4.x) and not forward compatible
try aubit4gl with mysql support.. no go.. see this
call script to unload mysql table and then load into informix

On solution attempt 3 - this proved more difficult than I first imagined.

I've been spoiled with Informix's very easy "unload to". Mysql simply has no equivalent. There is the "SELECT INTO OUTFILE", but (1) you need to "grant file on *.* to 'user'@'whateverhost'" (besides select permission of course), and (2) can be run from a remote host, but the output file is local to the mysql server instance.. and while you can use NFS, its a pain.

Also there is mysqldump - which I thought might do it.. but alas, if you use the --tab option it uses "SELECT INTO OUTFILE" internally anyway - plus you need to "grant lock tables on db.* to 'user'@'whateverhost'".

Ok - so had to revise my strategy - since I am running this script on remote host... this generates ansi "insert" statements that I will try to execute against informix:

mysqldump -h mysqlhost -usomeuser -psomepassword --compatible=ansi --no-create-info --compact dbname tablename > x.sql

Kinda works.. doesn't like text fields (clob datatype in informix).. and date formating could be an issue (easily fixed with informix DBDATE environment variable).. but certainly not as easy as I imagined.

BladeCenter S vitals via SNMP - installing MIBs

2009-06-23T08:18:00.000-04:00

Thanks to this post:

http://blog.barfoo.org/2009/02/10/monitoring-the-ibm-bladecenter-chassis-with-nagios/

I was able to use SNMP to get all kinds of info from my BladeCenter S - and specifically have it monitored with nagios. The original poster stated that the OIDs weren't documented anywhere.. but that is what the mib file is.. you just need to install it..

Here is how to install the MIBs from the IBM firmware (mmblade.mib) on ubunutu (requires snmp package):

copy mmblade.mib (contained in the firmware download) to /usr/share/snmp/mibs/BLADE-MIB.txt

(if you want alerts.. copy mmalerts.mib to /usr/share/snmp/mibs/BLADESPPALT-MIB.txt)

You now have access to over 10,000 settings.. you can get a complete list by doing:

snmpwalk -v1 -c public -Of ipaddress-of-bladecenterS -m BLADE-MID enterprises.ibm

and then for example, to find the front panel temperature:

snmpwalk -v1 -c public -Of ipaddress-of-bladecenterS -m BLADE-MIB enterprises.ibm.ibmAgents.netfinitySupportProcessorAgent.
bladeCenterSnmpMIB.monitors.temperature.ambientTemp.
frontPanelTemp.0

which, although quite verbose.. is more human readable than

snmpwalk -v1 -c public -Of ipaddress-of-bladecenterS
.1.3.6.1.4.1.2.3.51.2.2.1.5.1.0

Aubit4GL and libmysqld

2009-06-22T15:06:00.000-04:00

Had a frustrating time trying to build aubit4gl on ubuntu with mysql support.

/usr/bin/ld: /usr/lib/mysql/libmysqld.a(net_serv.o): relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC
/usr/lib/mysql/libmysqld.a: could not read symbols: Bad value

This is from libmysqlclient15-dev package. Turns out its not even an ubuntu issue.. its actual mysql bug:

http://bugs.mysql.com/bug.php?id=39288

I did see somewhere that it was fixed in libmysqlclient16-dev - which is not available in hardy (8.04) which I need.

I stumbled upon this link and was able to build a relocatable libmysqld.. but then I ran it this:

gcc -shared -L/lib -o /opt/aubit4gl/src/plugins-1.10RC_62/libSQL_mysqldb.so db-mysql.o -L/usr/local/lib -L/usr/local/mysql -L/usr/lib/mysql -L/usr/lib -L/opt/aubit4gl/src/lib -lmysqld -laubit4gl-1.10RC_62 -Wl,-Bsymbolic-functions -L/usr/lib/mysql -lmysqld -lwrap -lrt
/usr/bin/ld: cannot find -lwrap

even though libwrap is in /lib .. arrrg ok.. so I did the following:

cd lib/libsql/mysql
gcc -shared -L/lib -o /opt/aubit4gl/src/plugins-1.10RC_62/libSQL_mysqldb.so db-mysql.o -L/usr/local/lib -L/usr/local/mysql -L/usr/lib/mysql -L/usr/lib -L/opt/aubit4gl/src/lib -lmysqld -laubit4gl-1.10RC_62 -Wl,-Bsymbolic-functions -L/usr/lib/mysql -lmysqld
cd ../../..
make

(ie. I just removed -lwrap and -lrt and it went ok) and proceeded to compile the remainder of aubit4gl. Now to test against mysql