<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
span.E-MailFormatvorlage17
{mso-style-type:personal-compose;
font-family:"Verdana","sans-serif";
color:windowtext;
font-weight:normal;
font-style:normal;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 2.0cm 70.85pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:492331958;
mso-list-type:hybrid;
mso-list-template-ids:-1771680642 67567633 67567641 67567643 67567631 67567641 67567643 67567631 67567641 67567643;}
@list l0:level1
{mso-level-text:"%1\)";
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="DE" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Verdana","sans-serif"">Dear list,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Verdana","sans-serif""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9.0pt;font-family:"Verdana","sans-serif"">we just encountered a production issue with pg-pool-3.0.4.: pg-pool killed all idle childprocesses which were waiting for a connection request and failed to create
new ones.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9.0pt;font-family:"Verdana","sans-serif""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9.0pt;font-family:"Verdana","sans-serif"">I am unsure why the childs exited:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9.0pt;font-family:"Verdana","sans-serif""><o:p> </o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-18.0pt;mso-list:l0 level1 lfo1"><![if !supportLists]><span lang="EN-US" style="font-size:9.0pt;font-family:"Verdana","sans-serif""><span style="mso-list:Ignore">1)<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span lang="EN-US" style="font-size:9.0pt;font-family:"Verdana","sans-serif"">As we have child_life_time = 300 I guess the childs exited normally after a 5 minute period.
<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-18.0pt;mso-list:l0 level1 lfo1"><![if !supportLists]><span lang="EN-US" style="font-size:9.0pt;font-family:"Verdana","sans-serif""><span style="mso-list:Ignore">2)<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span lang="EN-US" style="font-size:9.0pt;font-family:"Verdana","sans-serif"">We did a Config reload on pg_pool, and that could mean also that waiting child processes are killed<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9.0pt;font-family:"Verdana","sans-serif""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9.0pt;font-family:"Verdana","sans-serif"">but pg_pool did not create new ones. So we were left with ~110 active pg-pool processes (which still seemed to work) but we could make no new connections to pgppol
(can’t tell if the connections have been queued or not).<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9.0pt;font-family:"Verdana","sans-serif"">This is what the syslog says:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9.0pt;font-family:"Verdana","sans-serif""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9.0pt;font-family:"Verdana","sans-serif"">May 7 15:43:13 pgpool-PrepaidDB-HA1 pgpool: 2012-05-07 15:43:13 LOG: pid 938: reload config files.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9.0pt;font-family:"Verdana","sans-serif"">May 7 15:43:13 pgpool-PrepaidDB-HA1 pgpool: 2012-05-07 15:43:13 LOG: pid 2707: reload config files.<b><o:p></o:p></b></span></p>
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:9.0pt;font-family:"Verdana","sans-serif"">May 7 16:09:29 pgpool-PrepaidDB-HA1 pgpool: 2012-05-07 16:09:17 ERROR: pid 938: fork() failed. reason: Resource temporarily unavailable<o:p></o:p></span></b></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9.0pt;font-family:"Verdana","sans-serif"">May 7 16:09:34 pgpool-PrepaidDB-HA1 pgpool: 2012-05-07 16:09:34 LOG: pid 29194: connect_inet_domain_socket_by_port: exit request has been sent<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9.0pt;font-family:"Verdana","sans-serif"">May 7 16:09:34 pgpool-PrepaidDB-HA1 pgpool: 2012-05-07 16:09:34 ERROR: pid 29194: connection to <backendserver ha1 here> (5432) failed<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9.0pt;font-family:"Verdana","sans-serif"">May 7 16:09:34 pgpool-PrepaidDB-HA1 pgpool: 2012-05-07 16:09:34 ERROR: pid 29194: new_connection: create_cp() failed<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9.0pt;font-family:"Verdana","sans-serif"">May 7 16:09:34 pgpool-PrepaidDB-HA1 pgpool: 2012-05-07 16:09:34 LOG: pid 29194: notice_backend_error: 0 fail over request from pid 29194<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9.0pt;font-family:"Verdana","sans-serif"">May 7 16:30:22 pgpool-PrepaidDB-HA1 heartbeat: [2425]: info: pgpool-prepaiddb-ha1 wants to go standby [all]<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9.0pt;font-family:"Verdana","sans-serif""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9.0pt;font-family:"Verdana","sans-serif"">We also noticed some high load on the server which may be related to the issue.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9.0pt;font-family:"Verdana","sans-serif""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9.0pt;font-family:"Verdana","sans-serif"">Any idea what the the cause of this issue is? Do you recommend setting child_life_time = 0? I think there are some memory leak issues in the 3.0.4 versions.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9.0pt;font-family:"Verdana","sans-serif"">3.0.4 is the latest official version in the Ubuntu repository and we would rather not go to self-compiled versions for reasons of maintenance and compliance.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9.0pt;font-family:"Verdana","sans-serif""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9.0pt;font-family:"Verdana","sans-serif"">Best regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9.0pt;font-family:"Verdana","sans-serif"">Ludwig<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:9.0pt;font-family:"Verdana","sans-serif""><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:8.0pt;font-family:"Verdana","sans-serif";mso-fareast-language:DE">petaFuel GmbH</span></b><span style="font-size:8.0pt;font-family:"Verdana","sans-serif";mso-fareast-language:DE"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-size:8.0pt;font-family:"Verdana","sans-serif";mso-fareast-language:DE">Ludwig Adam<br>
Geschäftsführer<br>
<br>
Münchner Strasse 4<br>
85354 Freising<br>
<br>
Tel.: +49 (0) 8161 4060 - 202 Mobil: +49 (0) 179-22 911 02<br>
Fax: +49 (0) 8161 4060 - 401 eMail: </span><span style="font-family:"Verdana","sans-serif";mso-fareast-language:DE"><a href="mailto:ludwig.adam@petafuel.de"><span style="font-size:8.0pt;color:blue">ludwig.adam@petafuel.de</span></a></span><span style="font-size:8.0pt;font-family:"Verdana","sans-serif";mso-fareast-language:DE"><br>
<br>
Geschäftsführer: Ludwig Adam, Dr. Peter Schönweitz<br>
Amtsgericht München, HRB 133773<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Verdana","sans-serif";mso-fareast-language:DE"><a href="www.petaFuel.de"><span style="color:blue">www.petaFuel.de</span></a><o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>