WinTV 1600 model 1199 and VirtualBox on linux

Had a working VirtualBox WinXP guest on an Ubuntu Natty host.

Installed a WinTV 1600 video capture card, and after that, the VirtualBox WinXP guest would not boot.. got the message:

Failed to load VMMR0.r0 VERR_NO_MEMORY

After digging thru the logs, and noticing vmalloc errors - I decided to blacklist the cx18 modules (for capture card), and reboot. VirtualBox worked now, but then if I removed the blacklist on the cx18 modules, and tried to load the module - I got the same vmalloc errors.

Determined its a shortage of virtual memory in the linux kernel. Default is 128M.

Certain device drivers (modules) - in my case, a video capture card - require chunks of this virtual memory for device mapping.

Virtualbox obviously uses the same mechanism. You can have 16GB of RAM, but if you only have 128M of vm allocated, you are going to run into the same issue.

Solution is to pass the "vmalloc=<size>" parameter to your kernel on boot.  Preferably in 64M chunks.

In ubuntu, edit /etc/default/grub, and add "vmalloc=192M" to the GRUB_CMDLINE_LINUX_DEFAULT list.

eg. GRUB_CMDLINE_LINUX_DEFAULT="vmalloc=192M quiet splash"

sudo update-grub

reboot

Now they play nicely together - and the overall system performance is much snappier.

Using curl with eXist to perform XQuery

Simple trick to use HTTP POST with curl to do an XQuery - (I struggled for a bit with the Content-type)

curl --header "Content-type: text/xml" --data "@x.xml" http://hostname:8080/exist/rest/db/

where x.xml is something like:


<?xml version="1.0" encoding="UTF-8"?>
<query xmlns="http://exist.sourceforge.net/NS/exist" start="1" max="20">
<text>
  <![CDATA[
  some XQuery here...
  ]]>
</text>
<properties>
  <property name="indent" value="yes"/>
</properties>
</query>

ext4 performance and the barrier option

We recently implemented a 6-disk SAS RAID 1+0 /home ext4 filesystem on our application server to improve performance, and found that we still had a disk I/O bottleneck, with processor I/O wait averaging in the 3 to 4% range.

I came across this LWN article http://lwn.net/Articles/283161/ that talks about a 30% performance drop when the barrier option is enabled on an ext4 filesystem.

As of the time of the LWN article, barrier was not enabled by default on an ext4 filesystem.  But somewhere along the kernel release line, barrier=1 (on) became the default for newly created ext4 filesystems.

Doing a "cat /proc/mounts" on my ubuntu 10.04 distro, I found that my ext4 filesystems indeed had the option barrier=1. So on April 3rd, I added barrier=0 to my fstab, and rebooted.  The performance boost was quite dramatic.

Using measures of "average wait time", "% utilization" and "%IO wait", here is the before and after stats, measured and averaged on a typical work week 8am to 5pm...


With barrier=1:

Average wait time:   90 ms
Average Disk utilization %:   40%
Average CPU IO wait %:  3.5%

With barrier=0:

Average wait time:  2 ms
Average Disk utilization %:   2%
Average CPU IO wait %:  0.40%


Graphs depict the change more dramatically (comparing April 1st to April 4th):

Average Wait Time

Average % Utilization

Load Average

NOTE: This is not a "very busy" April 1st compared to a "very light" April 4th.  Both are with a 30 user load, approx. 2300 processes.  And every other day of the week in the comparison show the same dramatic improvement.

Of course one can argue that we are sacrificing data integrity for performance - but we do have a UPS backed system, and crashes are (thus far) non-existent.

Finding the WWN on an HS22 SAS connectivity module

Recently obtained an IBM HS22 blade server with a SAS connectivity module, and was attempting to map a RAID volume in a disk storage module to this blade..  when I realized the AMM did not report the SAS ID (WWN) as it normally does with my HS21 blades.

I figured it was a firmware issue, so I upgraded the AMM firmware - but still nothing.

I found this link:http://www-947.ibm.com/support/entry/portal/docdisplay?brand=5000020&lndocid=MIGR-5083823  which scared me - seemed to indicate I had the wrong hardware.

I poked around the LSI BIOS and noted every SAS ID I could find.. and tried using those to map the volume, all to no avail.

It didn't help that the AMM reported the SAS module part number as 49Y8009, whereas I had ordered 43W4068.. thought there was a shipping error or something, but it turns out the 49Y8009 is just the internal IBM part number.

Thinking it was still related to firmware.. I upgraded the uEFI and the IMM firmware on the HS22.  Again, this helped nothing.

In a desperate final attempt, I decided to delve into the SAS RAID controller itself..  accessed the web interface, selected "Monitor SAS Module", and voila!  There, next to "Blade Slot Connection 2" was a SAS ID that was never presented anywhere else.  I mapped the volume using this WWN, and now all is good.

What a nightmare to find though..   found forum posts with no answers:  https://www.ibm.com/developerworks/forums/thread.jspa?messageID=14418964&#14418964

Someone from IBM, please update your documentation!!

Aggregating row data using user-defined aggregate in Informix IDS

Implementing mysql group_concat() aggregate in Informix IDS 11.5 - thanks to Jonathan Leffler.

As DBA do:

CREATE FUNCTION gc_init(dummy varchar(255)) RETURNING LVARCHAR;
    RETURN '';
END FUNCTION;

CREATE FUNCTION gc_iter(result LVARCHAR, value VARCHAR(255))
    RETURNING LVARCHAR;
    IF result = '' THEN
        RETURN TRIM(value);
    ELSE
        RETURN result || ',' || TRIM(value);
    END IF;
END FUNCTION;

CREATE FUNCTION gc_comb(partial1 LVARCHAR, partial2 LVARCHAR)
    RETURNING LVARCHAR;
    RETURN partial1 || ',' || partial2;
END FUNCTION;

CREATE FUNCTION gc_fini(final LVARCHAR) RETURNING LVARCHAR;
    RETURN final;
END FUNCTION;

CREATE AGGREGATE group_concat
    WITH (INIT = gc_init, ITER = gc_iter,
          COMBINE = gc_comb, FINAL = gc_fini);

Then you need to grant access to the functions to users that need it:

grant execute on function gc_init to appsql;
grant execute on function gc_iter to appsql;
grant execute on function gc_comb to appsql;
grant execute on function gc_fini to appsql;


Example Use:

select ord_id, group_concat(product)
from ord
group by ord_id