home about contact feed
vagra vaBlo
C++(7) Solaris(4) tntnet(3) vi(3) Linux(2) DeleGate(2) Postgres(2) proxy(2) cxxtools(1) regex(1) readline(1) dtrace(1) gcc(1) MeeGo(1) ssh(1) firefox(1)

Building gcc4 with gcc3

Compiling gcc4 on an system with only old gcc versions available, can be tricky. Here is an HowTo for compiling gcc4.4 on Solaris 10, which only ships 3.4.3. It will be similar for other systems with an older gcc versions.

Monitoring Postgres autovacuum

Postgres does all updates on a database Copy-on-Write. That means, on every update on a row, additional space is needed on the filesystem. If noone is using an old entry anymore, the space allocated by these data is not freed automatically, nor will it be available for postgres to store new data. To get the space back, you have to vacuum, luckily there is an autovacuum in postgres. Autovacuum is an extra deamon that will run vacuum from time to time. It will not run a vacuum full, thus it will never free disk space, but postgres is able to reuse the already allocated space and thus the datafiles will not grow infinitely.

The cumbersome part is to check if autovacuum is working, since there is no way to ensure that it will do. First you can check, if autovacuum is configured to run. You might think setting "autovacuum = on" would be enough, it's not, since it will only work with some version depending statistics enabled.

Potstgres >=8.2

SHOW autovacuum;
SHOW track_counts;

Prior versions:

SHOW autovacuum;
SHOW stats_start_collector;
SHOW stats_row_level;

Next is to check, if the autovacuum daemon is running. There is one postgres process that will be named "postgres: autovacuum launcher process". If you are running solaris, ps will not show you that name, instead it will show the original process cmdline, since it is ignoring updates on argv[0] made by processes at runtime. You can help your self with a little loop.

 for PGPR in `/usr/bin/pgrep postgres`; do pargs -a $PGPR; done | grep 'argv\[0' | sed 's/ *$//'

An more easy and generic way is to look into the logfile, if you've not already deleted the log containing the startup messages, you'll find the following line there:

LOG:  autovacuum launcher started

You can also monitor the activity of autovacuum on the database. However, it will only show you activities if autovacuum had done something, thus on an fresh installed database it is very unlikely to see any activity.

postgres=# SELECT relname,last_autovacuum from pg_stat_user_tables;
         relname         | last_autovacuum 
 my_table                | 2011-05-02 05:05:11.78106+02
 my_other_table          | 2011-05-03 09:21:18.850887+02
 unlikely_for_update     |

If you select pg_stat_all_tables instead of pg_stat_user_tables you will also see vacuums for internal tables.

drace syscall times

Sometimes processes are slow without consuming much ressources. In such cases you can try to use truss (or strace on some systems) to find out what the process is waiting for, however, in multithreaded processes this might not help you that much. Also tracing the process from userland will slow down the process further what might not be an option in production einvironments.

Using dtrace will not only be less intrusive to your application flow, it can also summarize the times for you. The following script will not only summarize the times spending in system calls (/opt/DTT/procsystime would do), it will also print you a line for each system call when it retruns

#!/usr/sbin/dtrace -s

#pragma D option quiet

/ zonename == "myzone" && execname == "tntnet" /

/ zonename == "myzone" && self->t!=0 && execname == "tntnet" /
        printf("%d/%d spent %d nsecs in %s\n",
        pid, tid, timestamp - self->t, probefunc);

        @function[probefunc] = sum (timestamp - self->t);

For these who are familiar with awk, the syntax will not surprise you, in / / there are filters on which the code blocks apply, self is a special variable you can use to ensure coherency between threads.

14611/52 spent 3415416747 nsecs in pollsys
14611/52 spent 8583 nsecs in gtime
14611/52 spent 5333 nsecs in gtime
14611/52 spent 21666 nsecs in lwp_park
14611/52 spent 2798833 nsecs in nanosleep
14611/52 spent 20010167 nsecs in pollsys
14611/52 spent 18000 nsecs in read
14611/52 spent 9864917 nsecs in nanosleep
14611/52 spent 5417 nsecs in gtime
14611/53 spent 10008364573 nsecs in lwp_park
14611/53 spent 11500 nsecs in gtime
14611/53 spent 5500 nsecs in gtime

  lseek                                                         29583
  fcntl                                                         34666
  close                                                         45083
  accept                                                        57417
  getpid                                                       133751
  open                                                         134834
  stat                                                         248915
  gtime                                                        282585
  read                                                        1725499
  write                                                       2817832
  nanosleep                                                  80435583
  pollsys                                                  5699637580
  lwp_park                                                35622920131

You see, that the process is spending the most time on beeing idle and waiting for requests, but tntnet is actually not a really good example since it doesn't appreas to be slow.

Solaris ptools

Solaris has some features, many other systems are missing. One of my favorites is the large number of process tools. While i could write the whole day about ptools, i like to concentrate on my favorites here. Some of them may also be are available on other plattforms.


pldd acts like ldd but on processes, thus you see which librarys are really loaded by the program. Yes there might are differences to the binaries on the filesystems.

$ pldd 1259
1259:   /home/jw/bin/ncmpc


pargs shows arguments -a and environment variables -e of an running process.

$ pargs -ea 284
284:    /usr/lib/inet/ntpd -p /var/run/ntp.pid -g
argv[0]: /usr/lib/inet/ntpd
argv[1]: -p
argv[2]: /var/run/ntp.pid
argv[3]: -g

envp[0]: _=*213*/usr/lib/inet/ntpd
envp[1]: LANG=en_US.UTF-8
envp[2]: PATH=/usr/sbin:/usr/bin
envp[3]: PWD=/root
envp[4]: SHLVL=1
envp[5]: SMF_FMRI=svc:/network/ntp:default
envp[6]: SMF_METHOD=start
envp[7]: SMF_RESTARTER=svc:/system/svc/restarter:default
envp[8]: SMF_ZONENAME=global
envp[9]: TZ=Europe/Berlin
envp[10]: A__z="*SHLVL


pstack prints a strack trace from an running process.

$ pstack 11717
11717:  ksh
 ceb04875 waitid   (7, 0, 8046f70, f)
 ceaa00a5 waitpid  (ffffffff, 804703c, c, ce99076f) + 65
 ce9907c8 job_reap (0, ce9f7000, 80470a8, ce99287c) + c0
 ce992af7 job_wait (3c82, 806a8a8, 0, 8047130) + 4ab
 ce9c8ae3 sh_exec  (806755c, 4, 0, ce9a0f3d) + 43af
 ce9a10e6 exfile   (ce9f7bb0, ce8d0fa8) + 732
 ce9a0920 sh_main  (1, 8047964, 0, cea5ea42, 4, 0) + 830
 08050bd6 main     (1, 8047964, 804796c, 8050a9f) + 4a
 08050afd _start   (1, 8047a7c, 0, 8061210, 8061230, 8061248) + 7d


plimit works like ulimit on processes. Remarkable is, that it doesn't only print the limits of an running process, you can also modify the limits without restarting the process.

$ /usr/bin/plimit 11716
11716:  xterm
   resource              current         maximum
  time(seconds)         unlimited       unlimited
  file(blocks)          unlimited       unlimited
  data(kbytes)          unlimited       unlimited
  stack(kbytes)         8480            130336
  coredump(blocks)      unlimited       unlimited
  nofiles(descriptors)  256             65536
  vmemory(kbytes)       unlimited       unlimited

$ /usr/bin/plimit -n 1024 11716

$ /usr/bin/plimit 11716        
11716:  xterm
   resource              current         maximum
  time(seconds)         unlimited       unlimited
  file(blocks)          unlimited       unlimited
  data(kbytes)          unlimited       unlimited
  stack(kbytes)         8480            130336
  coredump(blocks)      unlimited       unlimited
  nofiles(descriptors)  1024            65536
  vmemory(kbytes)       unlimited       unlimited

cacheControl for tntnet

tntnet contains an static component, that allows you to serve static content directly from your filesystem using send-file. But the static component doesn't add Expire or cacheControl headers, thus clients or proxys will ask every time if the file as changed. For files like your CSS what will likely be loaded for every site but changes only seldom, this will significantly increase the number of unnecessary requests to tntnet.

Luckily tntnet allows you to pass a requests to more than one component, thus you can benefit from the easiness of the static componend and add additional haeders.

  #include <tnt/httpheader.h>
  reply.setHeader(tnt::httpheader::cacheControl, "public, max-age=86400");
  return DECLINED;

The return DECLINED will cause tntnet to keep whatever content the component added to reply and then pass the request to the next componend on the UrlMap.

MapUrl  ^/static/(.*)   expire@vtoc
MapUrl  ^/static/(.*)   static@tntnet static/$1

cxxtools regex

cxxtools contains a nice class that allows you to use regular expressions in C++. Since it is so handy and not very documented, i like to show an short example.

#include <string>
#include <cxxtools/regex.h>

bool TwoFour (const std::string& word)
        cxxtools::Regex checkTwoFour("^[aA-zZ]{2,4}$");
        return checkTwoFour.match(word);

The function will return true only if word is between 2 and 4 characters long and consist only of alphabetic characters. Of course you can use more compley regular expression.

non-root SMF service

Many services doesn't require to run as root. Traditionally most services do start as root and just drop their privileges. With Solaris (>=10) you don't need that anymore, you can start services directly as user from the SMF.

Some Services like Webserver need "root privileges" at start to bind port 80 for example. However, in Solaris there are no "root privileges". There are many privileges now, one privilege called "net_privaddr" is excactly the privilege a Webserver will need to bind port 80, so there is no need to give all privileges a root user normaly have to the webserver.

        exec='/usr/local/bin/tntnet -c /var/tntnet/tntnet.conf'
            <method_credential user='tntnet' group='tntnet' privileges='basic,net_privaddr'/>

This is an example from my tntnet SMF manifest, you can download the full manifest here.


Back in 2005 just everyone had a webblog, at least most of my friends. Today the people using Twitter or Facebook to let everyone know whatever they want them to know, and noone cares about the old weblogs. So it's the right time to start one.

Actually this is how the story could be told but it is not why i'm starting this blog. This blog is much more a software experiment rather than a blogging experiment. I wanted to start experimenting with C++ again since i stopped it arround 8 years ago. You might argue that starting a blog doesn't have mucht to do with C++ experiments, but this blog actually has, since the blog software is entirely written in C++ using tntnet.

So, it is a programming experiment, and i like to try the code in practise. I'll add some content about programming, tntnet, Unix and may be some other technical stuff here, since i need content and readers do get any practical resuls from this experiment.

Previous « 1 2