introduction

At this point in my learning, I have written a few thread examples but don't
understand what a thread pool is supposed to be. I don't know how the
superThread (I believe that would be main thread in C), decides to
instantiate or recycle a thread or if it terminates, plain and simple.
I need now then to study thread lifetime.

definitions

thread pool: in java, some classes extend the class Runnable, and have a
run method. These then become objects. We can then pass them to a ThreadPool
object and call ThreadPoolInstance.execute(new Runnable object).

that is a quote from the doc at sun, link is
http://java.sun.com/developer/Books/javaprogramming/threads/chap13.pdf

thread strategies

1. create a new thread for each incoming request, and let the thread die after it has serviced that request
2. create a work queue and let threads wait for something to become available off the work queue in a producer-consumer fashion

As an example of the consumer-producer pseudo-code:

        job *job_queue;
        mutex_t job_mutex;
        cond_t job_cond;
        void workthread (void *) {
                job *j;
                for (;;) {
                        lock (job_mutex);
                        while (!(j = job_queue))
                                wait (job_mutex, job_cond);
                        job_queue = j->next;
                        unlock (job_mutex);
                        do (j);
                }
        }

(source: http://www.scs.cs.nyu.edu/G22.3250/notes/l3.pdf )

Note that this gets complicated fairly quickly, necessitating use of fucked up
things like semaphores, stemming from the need to lock access to the queue.

problems throttling threads

I only have 32 connections in postgresql.conf and they are used up pretty fast, so my web server is returning 500's on pages
using postgresql DBI in the mod_perl pages. I have followed instructions on invoking Apache::DBI to "cache connection handles", but I am still getting 500 errors and my question is how can I increase the number of connections in postgresql.conf? I
tried increasing to 100 which is a number I saw on the net, but it refuses to start and the error logs are not diagnostic. MMy question is, do I need to increase other parameters in correspondance to the increase in max_connections?</p>

This has gotten frustrating as I had this problem with another mod_perl web application, too, and only with postgresql not with mysql dependent mod_perl code.

On this linux machine I already am doing the following in /etc/rc.d/rc.local
(mostly in order to set up oracle):

        echo "100 256 100 100" > /proc/sys/kernel/sem
        # shared memory
        # SHMMAX Defines the maximum allowable size of one shared memory segment.
        # SHMMNI Defines the maximum number of shared memory segments in the entire system.
        # SHMALL not-re-linux
        # SHMMIN Defines the minimum allowable size of a single shared memory segment.
        echo 2147483648 > /proc/sys/kernel/shmmax
        echo 100 > /proc/sys/kernel/shmmni
        echo 2097152 > /proc/sys/kernel/shmall

dbconnections and threads

        If you have a multi-threaded application, each thread can
        use its own cursor, so they can send SQL commands to the
        database without interfering with each other, and without
        having to open a separate connection for every thread.

(source: pgsql-novice list on Thu May 6 08:35:24 2004 by Oliver Fromme )

about forking database connection

The question concerns database pools and forking new processes. My problem is
having to reboot my web server every hour to kill unused (there are too many
connections coming in). Yet, here (immediately below) is a display of a perl
program running called "genxref" and the other line there was "genxref" which
is a perl prgram issuing postgres dbi commands NON-STOP. It has been running
for a very long time. However, it only starts the "postmaster" command.

Here is a top display of the problem:

                  PID USER     PRI  NI  SIZE  RSS SHARE STAT %CPU %MEM   TIME COMMAND
                  776 root       6 -10 30220  25M  5280 S <  35.6  2.5  33:47 X
                 9572 joe       25   0  6648 6648  2076 S    31.3  0.6   2:24 genxref
                 9573 postgres  15   0  4612 4612  3812 D    17.4  0.4   1:17 postmaster
                 8427 joe       15   0  9852 9848  6832 R    12.0  0.9   1:23 gnome-terminal
                10215 joe       15   0  1108 1108   844 R     0.9  0.1   0:02 top
                10201 joe       15   0 49840  48M 48068 S     0.3  4.8   0:00 httpd

I am at present ( on the to-do list is figuring out why postmaster is here not
a front-end to it)