[pgpool-hackers: 409] Re: Adding listen_backlog parameter

[Tatsuo Ishii]

> Hi pgpool hackers,
> 
> I would like to add new parameter "listen_backlog". This essentially
> same as Apache's ListenBacklog parameter:
> 
> The second parameter of listen(2) system call.
> 
> As you might already know, pgpool accepts concurrent connections up to
> num_init_children. Exceeding connections will be queued in kernel's
> backlog queue. If the queue is long enough to accept a new connection,
> the request from the client will be almost immediately accepted by
> pgpool once existing client disconnects to pgpool. However, if the
> queue is overflowed, the client might retry after 3 seconds or so, and
> user thinks that pgpool's response is too bad. Or disconnected by
> pgpool with "Connection to database "test" failed: could not connect
> to server: Connection timed out".
> 
> The solution is, making the backlog parameter large enough for such
> possible load. Currently the backlog is calculated by
> num_init_children * 2. But this may not be large enough for certain
> load. For the case, the new proposed parameter can set the listen()'s
> backlog parameter longer than num_init_children * 2.
> 
> Here is an experiment I did to illustrate the effect of the
> parameter(script attached).
> 
> I started pgpool with num_init_children = 2, and connected to it by
> using 128 forked pgbench.
> Each pgbench executes 100 read only queries.
> 
> 	 pgbench -C -c 1 -n -S test;date)&
> 
> The pgpool server is a Linux box running kernel 3.4.68.
> 
> Stock pgpool-II 3.3.1, which set the backlog parameter to 4, took 1
> minute and 7 seconds. And some requests failed with "Connection to
> database "test" failed" error.

Please note that you will find something like:

    2357 times the listen queue of a socket overflowed

by executing netstat -s in this case.

> On the other hand if I set the backlog parameter to 1024, it took only
> 15 seconds without any error.
> 
> Opinions?
> --
> Tatsuo Ishii
> SRA OSS, Inc. Japan
> English: http://www.sraoss.co.jp/index_en.php
> Japanese: http://www.sraoss.co.jp