[pgpool-hackers: 412] Re: Adding listen_backlog parameter

Tatsuo Ishii ishii at postgresql.org
Mon Nov 18 15:21:26 JST 2013


> Good idea.
> 
> Do not forget to increase sysctl params like
> 
> net.core.somaxconn
> net.core.netdev_max_backlog

Yes, those are what I forgot to mention.
> 
> And more faster get socket stats (netstat -an[s] | wc -l)
> cat /proc/net//sockstat

Thanks for the info.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp

> 2013/11/18 Tatsuo Ishii <ishii at postgresql.org>
> 
>> > Hi pgpool hackers,
>> >
>> > I would like to add new parameter "listen_backlog". This essentially
>> > same as Apache's ListenBacklog parameter:
>> >
>> > The second parameter of listen(2) system call.
>> >
>> > As you might already know, pgpool accepts concurrent connections up to
>> > num_init_children. Exceeding connections will be queued in kernel's
>> > backlog queue. If the queue is long enough to accept a new connection,
>> > the request from the client will be almost immediately accepted by
>> > pgpool once existing client disconnects to pgpool. However, if the
>> > queue is overflowed, the client might retry after 3 seconds or so, and
>> > user thinks that pgpool's response is too bad. Or disconnected by
>> > pgpool with "Connection to database "test" failed: could not connect
>> > to server: Connection timed out".
>> >
>> > The solution is, making the backlog parameter large enough for such
>> > possible load. Currently the backlog is calculated by
>> > num_init_children * 2. But this may not be large enough for certain
>> > load. For the case, the new proposed parameter can set the listen()'s
>> > backlog parameter longer than num_init_children * 2.
>> >
>> > Here is an experiment I did to illustrate the effect of the
>> > parameter(script attached).
>> >
>> > I started pgpool with num_init_children = 2, and connected to it by
>> > using 128 forked pgbench.
>> > Each pgbench executes 100 read only queries.
>> >
>> >        pgbench -C -c 1 -n -S test;date)&
>> >
>> > The pgpool server is a Linux box running kernel 3.4.68.
>> >
>> > Stock pgpool-II 3.3.1, which set the backlog parameter to 4, took 1
>> > minute and 7 seconds. And some requests failed with "Connection to
>> > database "test" failed" error.
>>
>> Please note that you will find something like:
>>
>>     2357 times the listen queue of a socket overflowed
>>
>> by executing netstat -s in this case.
>>
>> > On the other hand if I set the backlog parameter to 1024, it took only
>> > 15 seconds without any error.
>> >
>> > Opinions?
>> > --
>> > Tatsuo Ishii
>> > SRA OSS, Inc. Japan
>> > English: http://www.sraoss.co.jp/index_en.php
>> > Japanese: http://www.sraoss.co.jp
>> _______________________________________________
>> pgpool-hackers mailing list
>> pgpool-hackers at pgpool.net
>> http://www.pgpool.net/mailman/listinfo/pgpool-hackers
>>


More information about the pgpool-hackers mailing list