[pgpool-hackers: 410] Re: Adding listen_backlog parameter

[Sergey Logvinov]

Good idea.

Do not forget to increase sysctl params like

net.core.somaxconn
net.core.netdev_max_backlog


And more faster get socket stats (netstat -an[s] | wc -l)
cat /proc/net//sockstat




2013/11/18 Tatsuo Ishii <ishii at postgresql.org>

> > Hi pgpool hackers,
> >
> > I would like to add new parameter "listen_backlog". This essentially
> > same as Apache's ListenBacklog parameter:
> >
> > The second parameter of listen(2) system call.
> >
> > As you might already know, pgpool accepts concurrent connections up to
> > num_init_children. Exceeding connections will be queued in kernel's
> > backlog queue. If the queue is long enough to accept a new connection,
> > the request from the client will be almost immediately accepted by
> > pgpool once existing client disconnects to pgpool. However, if the
> > queue is overflowed, the client might retry after 3 seconds or so, and
> > user thinks that pgpool's response is too bad. Or disconnected by
> > pgpool with "Connection to database "test" failed: could not connect
> > to server: Connection timed out".
> >
> > The solution is, making the backlog parameter large enough for such
> > possible load. Currently the backlog is calculated by
> > num_init_children * 2. But this may not be large enough for certain
> > load. For the case, the new proposed parameter can set the listen()'s
> > backlog parameter longer than num_init_children * 2.
> >
> > Here is an experiment I did to illustrate the effect of the
> > parameter(script attached).
> >
> > I started pgpool with num_init_children = 2, and connected to it by
> > using 128 forked pgbench.
> > Each pgbench executes 100 read only queries.
> >
> >        pgbench -C -c 1 -n -S test;date)&
> >
> > The pgpool server is a Linux box running kernel 3.4.68.
> >
> > Stock pgpool-II 3.3.1, which set the backlog parameter to 4, took 1
> > minute and 7 seconds. And some requests failed with "Connection to
> > database "test" failed" error.
>
> Please note that you will find something like:
>
>     2357 times the listen queue of a socket overflowed
>
> by executing netstat -s in this case.
>
> > On the other hand if I set the backlog parameter to 1024, it took only
> > 15 seconds without any error.
> >
> > Opinions?
> > --
> > Tatsuo Ishii
> > SRA OSS, Inc. Japan
> > English: http://www.sraoss.co.jp/index_en.php
> > Japanese: http://www.sraoss.co.jp
> _______________________________________________
> pgpool-hackers mailing list
> pgpool-hackers at pgpool.net
> http://www.pgpool.net/mailman/listinfo/pgpool-hackers
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.sraoss.jp/pipermail/pgpool-hackers/attachments/20131118/786d1094/attachment.html>