<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">More info:<br>
<br>
<div dir="ltr">
<div class="markdown-here-wrapper"
id="markdown-here-wrapper-398933" style="">
<p style="margin:1.2em 0px!important">Here is a syslog
snippet:</p>
<pre style="font-size:0.85em;font-family:Consolas,Inconsolata,Courier,monospace;font-size:1em;line-height:1.2em;overflow:auto;margin:1.2em 0px"><code style="font-size:0.85em;font-family:Consolas,Inconsolata,Courier,monospace;margin:0px 0.15em;padding:0px 0.3em;white-space:nowrap;border:1px solid rgb(234,234,234);background-color:rgb(248,248,248);border-top-left-radius:3px;border-top-right-radius:3px;border-bottom-right-radius:3px;border-bottom-left-radius:3px;display:inline;white-space:pre;border-top-left-radius:3px;border-top-right-radius:3px;border-bottom-right-radius:3px;border-bottom-left-radius:3px;border:1px solid rgb(204,204,204);padding:0.5em 0.7em;display:block;padding:0.5em;color:rgb(51,51,51);background-color:rgb(248,248,255);background-repeat:initial initial">Jul 26 12:06:33 pgpool01 pgpool: 2013-07-26 12:06:33 LOG: pid 12847: wd_create_send_socket: connect() reports failure (Connection refused). You can safely ignore this while starting up.
Jul 26 12:06:49 pgpool01 pgpool: 2013-07-26 12:06:49 LOG: pid 13243: wd_chk_sticy: all commands have sticky bit
Jul 26 12:06:49 pgpool01 pgpool: 2013-07-26 12:06:49 LOG: pid 13243: watchdog might call network commands which using sticky bit.
Jul 26 12:06:49 pgpool01 pgpool: 2013-07-26 12:06:49 LOG: pid 13243: wd_create_send_socket: connect() reports failure (Connection refused). You can safely ignore this while starting up.
Jul 26 12:06:52 pgpool01 pgpool: 2013-07-26 12:06:52 LOG: pid 13243: wd_escalation: escalated to master pgpool
Jul 26 12:06:54 pgpool01 pgpool: 2013-07-26 12:06:54 LOG: pid 13243: wd_create_send_socket: connect() reports failure (Connection refused). You can safely ignore this while starting up.
Jul 26 12:06:54 pgpool01 pgpool: 2013-07-26 12:06:54 LOG: pid 13243: wd_escalation: escaleted to delegate_IP holder
Jul 26 12:06:54 pgpool01 pgpool: 2013-07-26 12:06:54 LOG: pid 13243: wd_init: start watchdog
Jul 26 12:06:54 pgpool01 pgpool: 2013-07-26 12:06:54 LOG: pid 13243: pgpool-II successfully started. version 3.2.4 (namameboshi)
Jul 26 12:06:54 pgpool01 pgpool: 2013-07-26 12:06:54 LOG: pid 13243: find_primary_node: primary node id is 0
Jul 26 12:10:13 pgpool01 pgpool: 2013-07-26 12:10:13 LOG: pid 13243: received fast shutdown request
Jul 26 12:10:13 pgpool01 pgpool: 2013-07-26 12:10:13 LOG: pid 13243: watchdog_pid: 13257
<b>Jul 26 12:10:53 pgpool01 pgpool: 2013-07-26 12:10:53 ERROR: pid 13257: wd_IP_down: ifconfig down failed</b>
Jul 26 12:10:53 pgpool01 pgpool: 2013-07-26 12:10:53 LOG: pid 13257: wd_create_send_socket: connect() reports failure (Connection refused). You can safely ignore this while starting up.
Jul 26 12:11:19 pgpool01 pgpool: 2013-07-26 12:11:19 LOG: pid 13613: wd_chk_sticy: all commands have sticky bit</code></pre>
<p style="margin:1.2em 0px!important">I also attached 2
straces - one was the pgpool process as I issued a stop, and
the other was the watchdog process.</p>
</div>
</div>
Unfortunately, it doesn't look like apt.postgresql.org provides
dbg files, so I'll have to rebuild pgpool2 so I can get the
symbols for the backtraces.<br>
<br>
<br>
On 07/26/13 07:48, Jeff Frost wrote:<br>
</div>
<blockquote
cite="mid:WM!ebda45320d0432d0b032840fa7aff46b385fb236f4b33cb93217c9402814088e68d74e06b2c619e63906825209f9a6e6!@asav-3.01.com"
type="cite">
<pre wrap="">Yes, you can see the pgpool processes stuck in my ps output below.
They happily exit once I kill -9 the sudo process.
I'll see if I can get some stack traces but if you can't reproduce on Ubuntu or CentOS, I suspect it's something with Debian Squeeze's sudo or ifconfig commands.
On Jul 26, 2013, at 3:24 AM, Yugo Nagata <a class="moz-txt-link-rfc2396E" href="mailto:nagata@sraoss.co.jp"><nagata@sraoss.co.jp></a> wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Hi,
Does pgpool hang as well as ifconfig when it is stopped?
I cannot reproduce this on CentOS and Ubuntu. Both pgpool and
ifconfig stops normally.
Could you please provide me the stack trace of hanging pgpool and
log msessages?
On Thu, 25 Jul 2013 09:56:36 -0700
Jeff Frost <a class="moz-txt-link-rfc2396E" href="mailto:jeff@pgexperts.com"><jeff@pgexperts.com></a> wrote:
</pre>
<blockquote type="cite">
<pre wrap="">This seems to be the same on 3.2.3, 3.2.4 and 3.2.5.
The watchdog section of pgpool.conf looks like this:
use_watchdog = on
delegate_IP = '10.100.2.72'
wd_hostname = '10.100.2.70'
wd_port = 9000
ifconfig_path = '/usr/bin'
arping_path = '/usr/bin'
if_up_cmd = 'sudo ifconfig eth0:1 $_IP_$ netmask 255.255.255.0 up'
if_down_cmd = 'sudo ifconfig eth0:1 $_IP_$ netmask 255.255.255.0 down'
arping_cmd = 'sudo arping -U $_IP_$ -w 1'
wd_interval = 3
other_pgpool_hostname0 = '10.100.2.71'
other_pgpool_port0 = 9999
other_wd_port0 = 9000
virtual IP starts up great and properly moves to the secondary pgpool server
if you stop pgpool. However, the ifconfig becomes defunct and never exits
requiring a kill -9:
jeff@pgpool01:/tmp/pgpool$ ps -ef|grep pgpool
postgres 19974 1 0 09:51 pts/0 00:00:00 /tmp/pgpool/bin/pgpool -n
postgres 19975 1 0 09:51 pts/0 00:00:00 logger -t pgpool -p local0.info
postgres 19978 19974 0 09:51 pts/0 00:00:00 pgpool: watchdog
postgres 19979 19974 0 09:51 pts/0 00:00:00 pgpool: lifecheck
jeff 20735 1615 0 09:54 pts/0 00:00:00 grep pgpool
jeff@pgpool01:/tmp/pgpool$ ps -ef|grep ifconfig
root 20439 19979 0 09:52 pts/0 00:00:00 sudo ifconfig eth0:1
10.100.2.72 netmask 255.255.255.0 down
root 20440 20439 0 09:52 pts/0 00:00:00 [ifconfig] <defunct>
jeff 20737 1615 0 09:54 pts/0 00:00:00 grep ifconfig
System is Debian Squeeze. Any idea how to fix this? kill -9 of the sudo
allows pgpool to exit.
--
Jeff Frost <a class="moz-txt-link-rfc2396E" href="mailto:jeff@pgexperts.com"><jeff@pgexperts.com></a>
CTO, PostgreSQL Experts, Inc.
Phone: 1-888-PG-EXPRT x506
FAX: 415-762-5122
<a class="moz-txt-link-freetext" href="http://www.pgexperts.com/">http://www.pgexperts.com/</a>
_______________________________________________
pgpool-general mailing list
<a class="moz-txt-link-abbreviated" href="mailto:pgpool-general@pgpool.net">pgpool-general@pgpool.net</a>
<a class="moz-txt-link-freetext" href="http://www.pgpool.net/mailman/listinfo/pgpool-general">http://www.pgpool.net/mailman/listinfo/pgpool-general</a>
</pre>
</blockquote>
<pre wrap="">
--
Yugo Nagata <a class="moz-txt-link-rfc2396E" href="mailto:nagata@sraoss.co.jp"><nagata@sraoss.co.jp></a>
</pre>
</blockquote>
<pre wrap="">
---
Jeff Frost <a class="moz-txt-link-rfc2396E" href="mailto:jeff@pgexperts.com"><jeff@pgexperts.com></a>
CTO, PostgreSQL Experts, Inc.
Phone: 1-888-PG-EXPRT x506
FAX: 415-762-5122
<a class="moz-txt-link-freetext" href="http://www.pgexperts.com/">http://www.pgexperts.com/</a>
_______________________________________________
pgpool-general mailing list
<a class="moz-txt-link-abbreviated" href="mailto:pgpool-general@pgpool.net">pgpool-general@pgpool.net</a>
<a class="moz-txt-link-freetext" href="http://www.pgpool.net/mailman/listinfo/pgpool-general">http://www.pgpool.net/mailman/listinfo/pgpool-general</a>
</pre>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="78">--
Jeff Frost <a class="moz-txt-link-rfc2396E" href="mailto:jeff@pgexperts.com"><jeff@pgexperts.com></a>
CTO, PostgreSQL Experts, Inc.
Phone: 1-888-PG-EXPRT x506
FAX: 415-762-5122
<a class="moz-txt-link-freetext" href="http://www.pgexperts.com/">http://www.pgexperts.com/</a>
</pre>
</body>
</html>