Category: Computer

Remote system upgrade (with grub and bmc-watchdog)

IPMI is a very very powerful tool for system administrators, especially those telecommuting ones. It’s serial over LAN (SOL) support eliminates the need to personally sit in front of a server to do any pre-network operations, including reconfiguring the BIOS settings. However, it does require (A) an additional IP address to access the IPMI network interface from the Internet; or, when no additional IP can be allocated, (B) the access to a second server on the same LAN (not necessarily with administrator privilege). When either (A) or (B) is available, you can theoretically do anything remotely including fresh installation of an operation system (starting, for example, with a network boot and/or a remote drive).

Unfortunately, one of my recent situation allowed neither (A) nor (B). So, the first installation had no option but to be done by on-site personnel. But, once a networked system was up and running with a working grub boot manager, I could remotely install a new system on an unused (or a large enough swap) partition and test it out with the “boot once” support of grub. On a Debian based system with grub-2, this involves

  • changing the value of “GRUB_DEFAULT” in /etc/default/grub to “saved”,
  • running “update-grub”,
  • editing /boot/grub/grub.cfg to make an entry for the new system (if it was not discovered correctly by grub-probe),
  • running “grub-reboot” for the entry, and
  • rebooting the machine.

However, in most cases, you are bound to make some mistakes in the new system and fail to recover network contact to the server until an on-site person can hit the reset button of the machine for you.

Lucky for me, the BMC of the IPMI on the server did have a working watchdog timer. Therefore, I could setup the timer with enough time and start it before rebooting the machine. That way, if the new system worked, I could login to the server through the Internet and stopped the timer. But, if the new system got stuck, the watchdog would do a hard reset on the machine after the time ran out and returned to the original working system… no more waiting for on-site personnel. The actual command I used to setup the timer is bmc-watchdog from freeipmi:

  • bmc-watchdog -s -u 4 -p 0 -a 1 -F -P -L -S -O -i 900

One can consult the man page for the meaning of these options. Simply, this sets up 15 minutes on the timer for a hard reset, which can be checked with

  • bmc-watchdog -g

started with

  • bmc-watchdog -r

and stopped with

  • bmc-watchdog -y

(While, theoretically, one can achieve the same result with ipmitool, it did not work for me on the specific system.)

Portable library for C++ GUI programming

It has been for a few occasions that I find my self wanting to port my GUI programs (most of which were written with gtkmm) to other platforms for the enjoyment of my friends. However, while most GUI toolkits claim to be portable to all platforms (Linux, Windows, and Mac), they generally require installation of multiple shared libraries by the users.

This is a major show stopper for most of my friends and is sufficient to kill any interest in them on the first mention. Therefore, the only viable mean is for me to make and send them statically-linked, monolithic executables that they can happily click on to start the shows. (They generally don’t mind waiting a few minutes to download a bloated binary, as long as it remains a single step.)

My recent survey of the GUI library landscape brought my attention to GLUT. While it still requires installation on Windows, I can easily find static versions of FreeGLUT library for MingW that can be used to cross-compile, on Linux, statically-linked executables that can run independently on Windows. Furthermore, the GLUT framework, which, according to this, “is installed with every install of Mac OS X”.

However, GLUT library only provides facilities for managing windows and handling user inputs. It is by no means a GUI toolkit and you will have to draw all user-interactive elements by yourself (in OpenGL). I do find a GUI library, GLUI, that is built on and should be as portable as GLUT. However, after porting a couple programs to GLUI, I failed to find it enjoyable for me to break up the C++-elegant logic of gtkmm and redo my work in a less polished API.

What follows is the birth of gltk, it is an implementation of the gtkmm API on GLUT. I actually started with adding the libsigc++‘s signal-slot API to GLUI since the original callback mechanism only supported single static callback function and I needed more flexibility to port my programs. But, the hack soon proliferated into the entire source tree, and I decided it would be much more enjoyable for me to start something entirely from scratch.

After a somewhat persisting part-time effort that lasted more than a month, I have just made the first release of the library. It’s usable for a simple application that only needs some buttons, checkboxes, sliders, single line text labels or entries to control calculations. For myself, this represent over 80% of the applications that I would have considered porting. I am feeling pretty happy about it and I hope some others will also find it useful.

Project Homepage: http://gltk.ccdw.org/

Permutation

The following spaghetti prints all permutations of a string (“abcd”). I wrote it as an example to show the benefit of structured programming in my class.

#include <iostream>
using namespace std;
int main()
{
        char c[] = "abcd\n";
        int size = sizeof(c) - 2;
        int n[size + 1];
        n[size] = 1;
        int idx = size;
        char t;
new_round:
        for (int i = 0; i < idx; i ++) n[i] = i + 1;
        cout << c;
start_shift:
        idx = 0;
        t = c[0];
shift_next:
        if (n[idx]) goto shift_done;
        idx ++;
        c[0] = c[idx];
        c[idx] = t;
        t = c[0];
        goto shift_next;
shift_done:
        n[idx] --;
        if (idx == size) return 0;
        if (n[idx] == 0) goto start_shift;
        goto new_round;
}

The structured version is as follows:

#include <iostream>
using namespace std;
int main()
{
        char c[] = "abcd\n";
        int size = sizeof(c) - 2;
        int n[size + 1];
        n[size] = 1;
        int idx = size;
        char t;
        do {
                if (n[idx]) {
                        for (int i = 0; i < idx; i ++) n[i] = i + 1;
                        cout << c;
                }
                idx = 0;
                t = c[0];
                while (n[idx] == 0) {
                        idx ++;
                        c[0] = c[idx];
                        c[idx] = t;
                        t = c[0];
                }
                n[idx] --;
        } while (idx < size);
        return 0;
}

The idea is to rotate a string by n times, where n is the length of the string. While, before each rotation, rotate the n-1 substring at the front n-1 times. And while, before each rotation, rotate the n-2 substring at the front n-2 times. And so on… We can see that this is more elegantly done recursively:

#include <iostream>
using namespace std;
char c[] = "abcd\n";
int size = sizeof(c) - 2;
void rotate(int l)
{
        char ch = c[l - 1];
        for (int i = l - 1; i; i --) c[i] = c[i - 1];
        c[0] = ch;
}
void perm(int l)
{
        if (l == 1) cout << c;
        else for (int i = 0; i < l; i ++) {
                perm(l - 1);
                rotate(l);
        }
}
int main()
{
        perm(size);
        return 0;
}

The above thinking counts on the string to have all distinct chars. When this isn’t the case, a different approach is to consider the lexical order of the permutations. From a given permutation, we simply need to figure out the next in the lexical order until the order is “maximized”. It turns out that this is in the C++ STL. My reimplementation is as follows:

#include <iostream>
using namespace std;
void swap(char & a, char & b)
{
        char c = a;
        a = b;
        b = c;
}
// increase the order of string
bool incr(char * str, size_t len)
{
        size_t i = 1;
        while (i < len && str[i - 1] >= str[i]) i ++;
        if (i == len) return false; // no kink
        // found a kink
        size_t j = i - 1;
        while (j > 0 && str[j - 1] < str[i]) j --; // size kink
        swap(str[i], str[j]); // shave kink
        // reverse rest
        for (i --, j = 0; j < i; i --, j ++) swap(str[i], str[j]);
        return true;
}
int main()
{
        char c[] = "aabbc\n";
        int size = sizeof(c) - 2;
        do cout << c; while (incr(c, size));
}

Building driver for AverMedia A827 on Linux kernel 2.6.38

Vendor: AVerMedia
Product: AVerTV Hybrid Volar HX
Model: A827

This is a Analog+DVB-T USB TV receiver with official support for Linux up until 2009-11-26. The latest driver can be downloaded from the product page above.

The “Normal” automatic process of installing the driver fails for 2.6.38 kernel. Using the “Expert” process to extract the driver source code to a selected location allows one to compile the driver manually and track down the problems. First, the function calls for mutex initialization are gone and should be replaced with those of semaphore initialization. Second, the Teletext support is also gone and should be eliminated from the driver source code. These fixes are summarized in this patch.

However, compiling the driver with “make” results in the WARNINGs that the symbols “param_array_get” and “param_array_set” are undefined. This is due a prebuilt object file, “aver/osdep_dvb.o_shipped”, that was built with an older version of kernel. Building this file requires some internal header files from kernel source tree that are not normally available in, e.g., linux-headers or kernel-headers packages. Provided that the kernel source tree for building the running kernel is installed/available in the system, the shipped object file can be removed. After this, the “make” command can complete without incidence. Installing the drivers (“averusbh826d.ko”  “h826d.ko”) to the module directory, e.g., “/lib/modules/2.6.38/kernel/drivers/media/dvb/dvb-usb” and updating the module dependency with “depmod -a”, the adapter seems to work normally afterwards.

Setup isatap router on debian

The ifupdown package on Debian does not support isatap as a mode for v4tunnel. Therefore, one can not simply create a single entry in /etc/network/interfaces to make it work. Anyhow, following are the steps I took to set it up.

  1. Install iproute and radvd
  2. Add “net.ipv6.conf.all.forwarding=1” to /etc/sysctl.conf
  3. Add /etc/radvd.conf containing:
    interface is0
    {
        AdvSendAdvert on;
        UnicastOnly on;
        AdvHomeAgentFlag off;
        prefix 2002:aaaa:bbbb:1::/64
        {
            AdvOnLink on;
            AdvAutonomous on;
            AdvRouterAddr off;
        };
    };

    (replace “2002:aaaa:bbbb:1” with the prefix of your ipv6 subnet)

  4. Since I have my default address connected to a 6to4 tunnel on my eth0 already, I need to add an additional ip4 address to eth0. In /etc/network/interfaces I add the following post-up, and pre-down scripts to eth0:
    post-up ip tunnel add is0 mode isatap local 192.168.1.12 ttl 64
    post-up ip link set is0 up
    post-up ip addr add 2002:aaaa:bbbb:1::5efe:192.168.1.12/64 dev is0
    post-up ip addr add 192.168.1.12/32 dev eth0
    pre-down ip addr del 192.168.1.12/32 dev eth0
    pre-down ip link set is0 down
    pre-down ip tunnel del is0

    (again replace “2002:aaaa:bbbb:1“)

  5. Restart the computer or do the following:
    $ sysctl -p
    $ ifdown eth0
    $ ifup eth0
    $ invoke-rc.d radvd start

On the client side, I just installed isatapd, added

ISATAP_ROUTERS=”192.168.1.12″

to /etc/default/isatapd, and restarted with “invoke-rc.d isatapd restart“. Then, everything works!

Non-interactive ssh password auth

There are situations where only password authentication is allowed on a remote SSH server. And, it may be desirable to have non-interactive password entry even under interactive terminal sessions, e.g., running git, rsync, etc. through ssh. However, OpenSSH makes this difficult by requiring interactive keyboard entry whenever there is an associated tty to the process. While it is possible to bypass this with an expect script, the easiest solution is sshpass.

Processing command-line arguments in C++

I just released arg as a standalone library under LGPL. It’s a command-line parser for C++ programs. The aim is to keep the programming effort in adding command-line processing to C++ programs at a minimum. There are several considerations in this direction:

  1. simple and intuitive syntax
  2. requiring no redundancy
  3. localized codes
  4. extensibility
  5. completeness

The simple example as given on the arg homepage,

#include <arg.hh>
#include <iostream>
int main(int argc, char ** argv)
{
        arg::Parser p;
        int n;
        p.add_opt(‘n’).stow(n);
        p.parse(argc, argv);
        std::cout << n << ‘n’;
        return 0;
}

should be very close to the minimum as far as 1. goes.

Programming is for a programmer to describe what he wants the computer to do. Per point 2., he should not be asked to provide the same information multiple times. (Well, maybe except in situations where multiple confirmations are required: “Launch the missile. Please have the missile launched. Yes, I really want you to launch the missile! Launch the *&^%$ missile!!!”; Computer: “Aborted on Error: missile != *&^%$ missile”.)

When working on an item, e.g., adding a new command-line option, the programmer won’t be asked to go to multiple places in the codes if 3. is observed. While common and frequent usages should be supported and simplified in the library, new and novel applications will ask for 4. Finally, some rare, special, and/or tricky applications will demand 5. in the arsenal.

synchronize work spaces on different machines

While working on a set of files on different machines, it’s a common problem to keep things in sync. A real solution is to use a revision control system (with git being my current favorite). However, a quick fix is to use rsync. Following script tries to figure out which copy of the work space is newer and invokes rsync accordingly.

#!/bin/bash
if (( $# != 3 )); then
        echo "Usage: `basename $0` <local file> <remote host> <remote file>"
        exit 1;
fi
LOCAL_FILE="$1"
REMOTE_HOST="$2"
REMOTE_FILE="$3"

# make sure directories end with '/'
if [ -d "${LOCAL_FILE}" ]; then
        LOCAL_FILE=${LOCAL_FILE%/}/
        REMOTE_FILE=${REMOTE_FILE%/}/
        echo "sync directory: '`basename ${LOCAL_FILE}`' with ${REMOTE_HOST}"
else
        echo "sync file: '`basename ${LOCAL_FILE}`' with ${REMOTE_HOST}"
fi

# find out the last modification time in the entire directory
TM_LOCAL=`if [ -e "${LOCAL_FILE}" ]; then find $LOCAL_FILE -printf "%Ts %Pn"|sort|tail -n1; else echo 0; fi`
TM_REMOTE=`ssh ${REMOTE_HOST} "if [ -e "${REMOTE_FILE}" ]; then find $REMOTE_FILE -printf "%Ts %Pn"|sort|tail -n1; else echo 0; fi" < /dev/null`

echo Local Newest: $TM_LOCAL
echo Remote Newest: $TM_REMOTE
if [[ $TM_LOCAL < $TM_REMOTE ]]; then
        echo -n "remote => local"
        rsync -auz -e ssh --delete ${REMOTE_HOST}:""${REMOTE_FILE}"" "${LOCAL_FILE}"
        echo ",  Done!"
elif [[ $TM_LOCAL > $TM_REMOTE ]]; then
        echo -n "local => remote"
        rsync -auz -e ssh --delete "${LOCAL_FILE}" ${REMOTE_HOST}:""${REMOTE_FILE}""
        echo ",  Done!"
else
        echo "Nothing to do!"
fi

adding OCR to djvu file

For each page in the file “cake.djvu”, we can use the “tesseract” to process the page image:

djvused -e “select ${page};save-page-with “cake_page.djvu”” cake.djvu
convert cake_page.djvu cake.tif
tesseract cake.tif cake_box batch.nochop makebox
tesseract cake.tif cake_txt batch.nochop

This produces the information for the text structure (lines and words) and positioning (coordinate for each character). To convert this information to the hidden-text format for use with djvused, use

perl<<'EOL'>cake_text.txt
open TXT, "<:utf8", "cake_txt.txt";
open BOX, "<:utf8", "cake_box.txt";
$pxn = 1000000;
$pxx = 0;
$pyn = 1000000;
$pyx = 0;
$pagebuf = "";
while ($line = <TXT>) {
chop $line;
@words = split /s+/, $line;
next if $#words < 0;
$lxn = 1000000;
$lxx = 0;
$lyn = 1000000;
$lyx = 0;
$linebuf = "";
foreach $word (@words) {
$xmin = 1000000;
$xmax = 0;
$ymin = 1000000;
$ymax = 0;
$w = "";
for ($i = 0; $i < length($word); $i ++) {
$c = substr($word, $i, 1);
do {
$cline = <BOX>;
} while (substr($cline, 0, 1) ne $c);
($xn, $yn, $xx, $yx) = substr($cline, 2) =~ /S+/g;
$w = $w . '\' if $c eq '"';
$w = $w . '\' if $c eq '\';
$w = $w . substr($cline, 0, 1);
$xmin = $xn if ($xmin > $xn);
$xmax = $xx if ($xmax < $xx);
$ymin = $yn if ($ymin > $yn);
$ymax = $yx if ($ymax < $yx);
}
$wline = '(word ' . $xmin . ' ' . $ymin . ' ' . $xmax . ' ' . $ymax . ' "' . $w . '")';
$linebuf = $linebuf . "n  " . $wline;
$lxn = $xmin if ($lxn > $xmin);
$lxx = $xmax if ($lxx < $xmax);
$lyn = $ymin if ($lyn > $ymin);
$lyx = $ymax if ($lyx < $ymax);
}
$pagebuf = $pagebuf . "n (line $xmin $ymin $xmax $ymax" . $linebuf . ')';
$pxn = $lxn if ($pxn > $lxn);
$pxx = $lxx if ($pxx < $lxx);
$pyn = $lyn if ($pyn > $lyn);
$pyx = $lyx if ($pyx < $lyx);
}
close BOX;
close TXT;
binmode(STDOUT, ":utf8");
print "(page $pxn $pyn $pxx $pyx", $pagebuf, ')', "n";
EOL

which generates “cake_text.txt” in the accordant format. The hidden text can be saved back to the djvu file with

djvused -e “select ${page};set-txt “cake_text.txt”;save” cake.djvu

We just need to repeat this for all the desired pages.