<div dir="ltr"><div><div><div>Hi Tatsuo,<br><br></div>Am not able to close connection jmeter until the process completed. Load and memory goes high with 2 hours and request slows down :(<br><br>top - 09:22:34 up 3 days, 23:43, 2 users, load average: 38.14, 35.55, 27.58<br>Tasks: 2142 total, 12 running, 2130 sleeping, 0 stopped, 0 zombie<br>%Cpu(s): 89.7 us, 4.3 sy, 0.0 ni, 2.2 id, 0.0 wa, 0.0 hi, 3.8 si, 0.0 st<br>KiB Mem : 28811504 total, 13259276 free, 14191604 used, 1360624 buff/cache<br>KiB Swap: 0 total, 0 free, 0 used. 13808564 avail Mem <br><br> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND <br> 8500 root 20 0 375112 147420 1552 S 6.1 0.5 3:24.52 pgpool <br> 8543 root 20 0 375384 147584 1552 S 6.1 0.5 3:27.50 pgpool <br> 6992 root 20 0 380484 152872 1552 S 5.8 0.5 3:37.96 pgpool <br> 7001 root 20 0 374860 147028 1552 S 5.8 0.5 3:21.55 pgpool <br> 8520 root 20 0 374340 147424 1552 S 5.8 0.5 3:22.88 pgpool <br> 6705 root 20 0 375856 147892 1552 S 5.5 0.5 3:26.17 pgpool <br> 6746 root 20 0 377352 149632 1552 S 5.5 0.5 3:32.25 pgpool <br> 6747 root 20 0 377864 150276 1552 R 5.5 0.5 3:32.64 pgpool <br> 6966 root 20 0 375784 147948 1552 S 5.5 0.5 3:26.64 pgpool <br> 7489 root 20 0 378320 150740 1552 R 5.5 0.5 3:34.04 pgpool <br> 8170 root 20 0 371288 143188 1552 S 5.5 0.5 3:13.26 pgpool <br> 8209 root 20 0 376976 149300 1552 S 5.5 0.5 3:30.02 pgpool <br> 8306 root 20 0 371124 143308 1552 R 5.2 0.5 3:16.86 pgpool <br> 8409 root 20 0 373364 145544 1552 S 5.2 0.5 3:17.43 pgpool <br> 8490 root 20 0 372132 144200 1552 R 5.2 0.5 3:18.36 pgpool <br> 8576 root 20 0 374488 146660 1552 S 5.2 0.5 3:21.82 pgpool <br> 7669 root 20 0 373664 145804 1552 S 4.9 0.5 3:22.21 pgpool <br> 8165 root 20 0 377304 149512 1552 S 4.9 0.5 3:31.29 pgpool <br> 8252 root 20 0 377180 149448 1552 S 4.9 0.5 3:32.48 pgpool <br> 8301 root 20 0 374192 146504 1552 S 4.9 0.5 3:21.87 pgpool <br> 8305 root 20 0 378708 151176 1552 S 4.9 0.5 3:36.54 pgpool <br> 8494 root 20 0 373588 145716 1552 S 4.9 0.5 3:16.83 pgpool <br> 8510 root 20 0 374984 147192 1552 S 4.9 0.5 3:19.79 pgpool <br> 8512 root 20 0 375156 147312 1552 S 4.9 0.5 3:26.39 pgpool <br> 8535 root 20 0 378300 150712 1552 S 4.9 0.5 3:29.06 pgpool <br> 8542 root 20 0 377912 150176 1552 S 4.9 0.5 3:27.46 pgpool <br> 7347 root 20 0 378188 151536 1552 S 4.6 0.5 3:33.59 pgpool <br> 7794 root 20 0 373264 145372 1552 R 4.6 0.5 3:17.37 pgpool <br> 8130 root 20 0 371908 144100 1552 R 4.6 0.5 3:14.56 pgpool <br> 8215 root 20 0 375508 147592 1552 S 4.6 0.5 3:22.25 pgpool <br> 8243 root 20 0 376120 148292 1552 S 4.6 0.5 3:30.88 pgpool <br><br><br></div>Here are the conf :<br><br>#------------------------------------------------------------------------------<br># POOLS<br>#------------------------------------------------------------------------------<br><br># - Concurrent session and pool size -<br><br>num_init_children = 2000<br> # Number of concurrent sessions allowed<br> # (change requires restart)<br>max_pool = 4<br> # Number of connection pool caches per connection<br> # (change requires restart)<br><br># - Life time -<br><br>#child_life_time = 300<br>child_life_time = 300<br> # Pool exits after being idle for this many seconds<br>child_max_connections = 100<br> # Pool exits after receiving that many connections<br> # 0 means no exit<br>connection_life_time =<br> # Connection to backend closes after being idle for this many seconds<br> # 0 means no close<br>client_idle_limit = 0<br> # Client is disconnected after being idle for that many seconds<br> # (even inside an explicit transactions!)<br> # 0 means no disconnection<br></div><div><br></div><div>We have 2 servers one is master and other one is slave with replication. ( both servers has 16core and 28G Memory with 2k connection open in each server)<br></div><div><br></div><div><br></div>Let me know if you need any other details. <br><div><div><br></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Sep 12, 2017 at 12:03 PM, Tatsuo Ishii <span dir="ltr"><<a href="mailto:ishii@sraoss.co.jp" target="_blank">ishii@sraoss.co.jp</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Occasionally close connections from jemeter to Pgpool-II. I am not<br>
sure if you can do it with jmeter though.<br>
<br>
And set this:<br>
<br>
child_max_connections = 100<br>
<br>
will restart Pgpool-II child process every 100 times connections to<br>
Pgpool-II and it will free memory used by Pgpool-II child process.<br>
<div class="HOEnZb"><div class="h5"><br>
Best regards,<br>
--<br>
Tatsuo Ishii<br>
SRA OSS, Inc. Japan<br>
English: <a href="http://www.sraoss.co.jp/index_en.php" rel="noreferrer" target="_blank">http://www.sraoss.co.jp/index_<wbr>en.php</a><br>
Japanese:<a href="http://www.sraoss.co.jp" rel="noreferrer" target="_blank">http://www.sraoss.co.<wbr>jp</a><br>
<br>
> Team,<br>
><br>
> We have upgraded the latest version. but still the memory leak issue not<br>
> resolved. We are doing the performance testing using jmeter with 4k<br>
> concurrent users for more than a day where the memory is getting filled and<br>
> request has been slow down with errors.<br>
><br>
> Any possible solutions?<br>
><br>
><br>
> On Thu, Sep 7, 2017 at 9:24 AM, Tatsuo Ishii <<a href="mailto:ishii@sraoss.co.jp">ishii@sraoss.co.jp</a>> wrote:<br>
><br>
>> >>From the above setup we are trying hit the concurrent users of 4k... But<br>
>> at<br>
>> > some point of time PGpool load got increased and memory is fully occupied<br>
>> > even if i increase the Memory full memory got occupied. This resulting<br>
>> > requests slow down and request per second throughput and failures.<br>
>><br>
>> Sounds like an old problem in Pgpool-II 3.6.2. You should upgrade to<br>
>> the latest version: 3.6.6.<br>
>> ><br>
>> > But without PGpool directly connecting to Master works fine without<br>
>> errors<br>
>> > and much better throughput.<br>
>> ><br>
>> > We would like to scale 20k concurrent users with PGPool setup. We would<br>
>> > like to know whether PGpool is enterprise standard to work on.. Else we<br>
>> > have to look for other solutions.<br>
>><br>
>> But that requires PostgreSQL to accept 20k concurrent connections or<br>
>> 40k connections (num_init_children * max_pool), which I have never<br>
>> seen in the field. Probably you should test out with PostgreSQL<br>
>> whether it works with 20k concurrent connections first.<br>
>><br>
>> Best regards,<br>
>> --<br>
>> Tatsuo Ishii<br>
>> SRA OSS, Inc. Japan<br>
>> English: <a href="http://www.sraoss.co.jp/index_en.php" rel="noreferrer" target="_blank">http://www.sraoss.co.jp/index_<wbr>en.php</a><br>
>> Japanese:<a href="http://www.sraoss.co.jp" rel="noreferrer" target="_blank">http://www.sraoss.co.<wbr>jp</a><br>
>><br>
><br>
><br>
><br>
> --<br>
><br>
> Regards,<br>
> Rishi<br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><br>Regards,<br>Rishi<br></div>
</div>