Just upgraded to 3.5.2 this week- am seeing a hundred-fold reduction in server load and everything is running faster than before. Thanks guys!<br><br>On Saturday, October 10, 2015, Joe Schaefer <<a href="mailto:joesuf4@gmail.com">joesuf4@gmail.com</a>> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">It's new to me too. It doesn't seem to happen for our four-node application, just with the two node one. I don't know if I can come up with a test case since it only seems to happen when our application exceeds about 60 users.<div><br></div><div>The pgpool configuration on both sets of nodes is essentially the same, so I'm at a loss what to do for the two-node application other than to disable load-balancing.</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Oct 10, 2015 at 7:05 PM, Tatsuo Ishii <span dir="ltr"><<a href="javascript:_e(%7B%7D,'cvml','ishii@postgresql.org');" target="_blank">ishii@postgresql.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span>> Hi,<br>
><br>
> Love the product so far but have run into a few bugs that will soon be<br>
> showstoppers for us. Here's what we've found:<br>
><br>
> Our application provides persistent connections to a web socket app,<br>
> potentially supporting hundreds of concurrent users. At one site we have<br>
> num_init_children set to 2048 because we are replicating hundreds of<br>
> databases across four different master-slave postgres nodes. The problem<br>
> here isn't really a big deal, but it's something that should be worked out<br>
> because no similar server-side software still possesses thundering-herd<br>
> type issues like we do with pgpool. Our typical load averages for the<br>
> pgpool master is around 200-400. CPU idle is fine, so we're not concerned<br>
> other than our load average statistics for that host are out of whack.<br>
<br>
</span>The thundering-herd problem has been discussed recently (see<br>
[pgpool-general: 3934] accept() scalability issues)<br>
<a href="http://www.pgpool.net/pipermail/pgpool-general/2015-August/003992.html" rel="noreferrer" target="_blank">http://www.pgpool.net/pipermail/pgpool-general/2015-August/003992.html</a><br>
<br>
Next major version of pgpool-II will have a parameter to overcome the<br>
problem.<br>
<br>
<a href="http://www.pgpool.net/pipermail/pgpool-committers/2015-October/002711.html" rel="noreferrer" target="_blank">http://www.pgpool.net/pipermail/pgpool-committers/2015-October/002711.html</a><br>
<span><br>
> Secondly, we have two-postgres-node setup with tens of databases and<br>
> num_init_children set to 512. It is basically the same application but for<br>
> a different site. What happens here is far more concerning: when we see<br>
> about 60+ concurrrent users (and hence have about 200 pgpool backend<br>
> connections) pgpool stops correctly shipping write queries to the master.<br>
> Instead it ships a certain percentage to the slave, which breaks our<br>
> application completely.<br>
<br>
</span>This is new to me. Is there any test case to reproduce the problem?<br>
<span><br>
> Of the two, my real concern lies with the second situation as users are<br>
> impacted by pgpool's behavior during heavy load periods. This doesn't seem<br>
> to happen under light load with less than 20 users and hence less than 100<br>
> backend connections.<br>
><br>
> Anyone seen anything similar and know how to workaround it? For kicks here<br>
> are my whitelist rules for sql functions:<br>
><br>
> white_function_list = 'random,count,extract,date_part'<br>
> # Comma separated list of function names<br>
> # that don't write to database<br>
> # Regexp are accepted<br>
> black_function_list = 'currval,lastval,nextval,setval,.*'<br>
> # Comma separated list of function names<br>
> # that write to database<br>
> # Regexp are accepted<br>
<br>
</span>Best regards,<br>
--<br>
Tatsuo Ishii<br>
SRA OSS, Inc. Japan<br>
English: <a href="http://www.sraoss.co.jp/index_en.php" rel="noreferrer" target="_blank">http://www.sraoss.co.jp/index_en.php</a><br>
Japanese:<a href="http://www.sraoss.co.jp" rel="noreferrer" target="_blank">http://www.sraoss.co.jp</a><br>
</blockquote></div><br></div>
</blockquote>