Hi Tatsuo, <div><br></div><div>We are also facing the same issue. But network team is claiming no network issue and said pool have to handle the situation. </div><div>We are also in dialama. </div><div>Please suggest what is process to follow to overcome this issues? .</div><div>What are config parameters to configure to withstand this kind of issues? .</div><div><br></div><div>Your suggestions highly appreciated. </div><div>Thanks a lot for your support.</div><div><br></div><div>Regards, </div><div>Siva.<br><br>On Friday, September 13, 2019, Tatsuo Ishii <<a href="mailto:ishii@sraoss.co.jp">ishii@sraoss.co.jp</a>> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">In the manual:<br>
<br>
5.14.3. Upstream server connection<br>
<br>
------------------------------<wbr>------------------------------<wbr>------<br>
trusted_servers (string)<br>
<br>
Specifies the list of trusted servers to check the up stream<br>
connections. Each server in the list is required to respond to<br>
ping. Specify a comma separated list of servers such as<br>
"hostA,hostB,hostC". If none of the server are reachable, watchdog<br>
will regard it as failure of the Pgpool-II. Therefore, it is<br>
recommended to specify multiple servers.<br>
------------------------------<wbr>------------------------------<wbr>------<br>
<br>
It's not clear actually what will happen after "watchdog will regard<br>
it as failure of the Pgpool-II." What actually happens is, Pgpool-II<br>
sucides (see the log below for example).<br>
<br>
I think we should clearly stat that Pgpool-II will go down if pgpool<br>
cannot reach to non of trusted servers, something like:<br>
<br>
------------------------------<wbr>------------------------------<wbr>------<br>
trusted_servers (string)<br>
<br>
Specifies the list of trusted servers to check the up stream<br>
connections. Each server in the list is required to respond to<br>
ping. Specify a comma separated list of servers such as<br>
"hostA,hostB,hostC". If none of the server are reachable, watchdog<br>
will regard it as failure of the Pgpool-II and the Pgpool-II will<br>
shut down. Therefore, it is recommended to specify multiple<br>
servers.<br>
------------------------------<wbr>------------------------------<wbr>------<br>
<br>
<br>
2019-09-13 10:35:53: pid 30659: WARNING: watchdog failed to ping host"192.192.192.192"<br>
2019-09-13 10:35:53: pid 30659: DETAIL: ping process exits with code: 1<br>
2019-09-13 10:35:53: pid 30659: WARNING: watchdog lifecheck, failed to connect to any trusted servers<br>
2019-09-13 10:35:53: pid 30659: LOG: informing the node status change to watchdog<br>
2019-09-13 10:35:53: pid 30659: DETAIL: node id :0 status = "NODE DEAD" message:"trusted server is unreachable"<br>
2019-09-13 10:35:53: pid 30656: LOG: new IPC connection received<br>
2019-09-13 10:35:53: pid 30656: LOG: received node status change ipc message<br>
2019-09-13 10:35:53: pid 30656: DETAIL: trusted server is unreachable<br>
2019-09-13 10:35:53: pid 30656: WARNING: watchdog lifecheck reported, we are disconnected from the network<br>
2019-09-13 10:35:53: pid 30656: DETAIL: changing the state to LOST<br>
2019-09-13 10:35:53: pid 30656: LOG: watchdog node state changed from [MASTER] to [LOST]<br>
2019-09-13 10:35:53: pid 30656: FATAL: system has lost the network<br>
2019-09-13 10:35:53: pid 30656: LOG: Watchdog is shutting down<br>
2019-09-13 10:35:53: pid 30813: LOG: watchdog: de-escalation started<br>
2019-09-13 10:35:53: pid 30646: LOG: watchdog child process with pid: 30656 exits with status 768<br>
2019-09-13 10:35:53: pid 30646: FATAL: watchdog child process exit with fatal error. exiting pgpool-II<br>
2019-09-13 10:35:53: pid 30814: LOG: setting the local watchdog node name to "localhost:50000 Linux tishii-CFSV7-1"<br>
2019-09-13 10:35:53: pid 30814: LOG: watchdog cluster is configured with 2 remote nodes<br>
2019-09-13 10:35:53: pid 30814: LOG: watchdog remote node:0 on localhost:50006<br>
2019-09-13 10:35:53: pid 30814: LOG: watchdog remote node:1 on localhost:50010<br>
2019-09-13 10:35:53: pid 30814: LOG: interface monitoring is disabled in watchdog<br>
2019-09-13 10:35:53: pid 30814: LOG: watchdog node state changed from [DEAD] to [LOADING]<br>
2019-09-13 10:35:53: pid 30814: LOG: new outbound connection to localhost:50006 <br>
2019-09-13 10:35:53: pid 30814: LOG: new outbound connection to localhost:50010 <br>
2019-09-13 10:35:53: pid 30814: LOG: watchdog node state changed from [LOADING] to [INITIALIZING]<br>
2019-09-13 10:35:53: pid 30814: LOG: new watchdog node connection is received from "<a href="http://127.0.0.1:60611" target="_blank">127.0.0.1:60611</a>"<br>
2019-09-13 10:35:53: pid 30814: LOG: new node joined the cluster hostname:"localhost" port:50006 pgpool_port:50004<br>
2019-09-13 10:35:53: pid 30814: LOG: new watchdog node connection is received from "<a href="http://127.0.0.1:61123" target="_blank">127.0.0.1:61123</a>"<br>
2019-09-13 10:35:53: pid 30814: LOG: new node joined the cluster hostname:"localhost" port:50010 pgpool_port:50008<br>
2019-09-13 10:35:53: pid 30814: LOG: Watchdog is shutting down<br>
______________________________<wbr>_________________<br>
pgpool-hackers mailing list<br>
<a href="mailto:pgpool-hackers@pgpool.net">pgpool-hackers@pgpool.net</a><br>
<a href="http://www.pgpool.net/mailman/listinfo/pgpool-hackers" target="_blank">http://www.pgpool.net/mailman/<wbr>listinfo/pgpool-hackers</a><br>
</blockquote></div>