Sep 1 11:26:22 centos8i3 pgpool[13452]: [1-1] pid 13452: LOG: Backend status file /var/log/pgpool-II/pgpool_status discarded Sep 1 11:26:22 centos8i3 pgpool[13452]: [2-1] pid 13452: LOG: memory cache initialized Sep 1 11:26:22 centos8i3 pgpool[13452]: [2-2] pid 13452: DETAIL: memcache blocks :64 Sep 1 11:26:22 centos8i3 pgpool[13452]: [3-1] pid 13452: LOG: pool_discard_oid_maps: discarded memqcache oid maps Sep 1 11:26:22 centos8i3 pgpool[13452]: [4-1] pid 13452: LOG: waiting for watchdog to initialize Sep 1 11:26:22 centos8i3 pgpool[13454]: [4-1] pid 13454: LOG: setting the local watchdog node name to "centos8i3-int:5432 Linux centos8i3.localdomain" Sep 1 11:26:22 centos8i3 pgpool[13454]: [5-1] pid 13454: LOG: watchdog cluster is configured with 2 remote nodes Sep 1 11:26:22 centos8i3 pgpool[13454]: [6-1] pid 13454: LOG: watchdog remote node:0 on centos8i1-int:9000 Sep 1 11:26:22 centos8i3 pgpool[13454]: [7-1] pid 13454: LOG: watchdog remote node:1 on centos8i2-int:9000 Sep 1 11:26:22 centos8i3 pgpool[13454]: [8-1] pid 13454: LOG: interface monitoring is disabled in watchdog Sep 1 11:26:22 centos8i3 pgpool[13454]: [9-1] pid 13454: LOG: watchdog node state changed from [DEAD] to [LOADING] Sep 1 11:26:22 centos8i3 pgpool[13454]: [10-1] pid 13454: LOG: new outbound connection to centos8i1-int:9000 Sep 1 11:26:22 centos8i3 pgpool[13454]: [11-1] pid 13454: LOG: new outbound connection to centos8i2-int:9000 Sep 1 11:26:27 centos8i3 pgpool[13454]: [12-1] pid 13454: LOG: watchdog node state changed from [LOADING] to [JOINING] Sep 1 11:26:27 centos8i3 pgpool[13454]: [13-1] pid 13454: LOG: watchdog node state changed from [JOINING] to [INITIALIZING] Sep 1 11:26:28 centos8i3 pgpool[13454]: [14-1] pid 13454: LOG: watchdog node state changed from [INITIALIZING] to [STANDING FOR MASTER] Sep 1 11:26:28 centos8i3 pgpool[13454]: [15-1] pid 13454: LOG: our stand for coordinator request is rejected by node "centos8i1-int:5432 Linux centos8i1.localdomain" Sep 1 11:26:28 centos8i3 pgpool[13454]: [16-1] pid 13454: LOG: watchdog node state changed from [STANDING FOR MASTER] to [PARTICIPATING IN ELECTION] Sep 1 11:26:33 centos8i3 pgpool[13454]: [17-1] pid 13454: LOG: watchdog node state changed from [PARTICIPATING IN ELECTION] to [JOINING] Sep 1 11:26:33 centos8i3 pgpool[13454]: [18-1] pid 13454: LOG: new watchdog node connection is received from "10.0.0.1:57491" Sep 1 11:26:33 centos8i3 pgpool[13454]: [19-1] pid 13454: LOG: new node joined the cluster hostname:"centos8i1-int" port:9000 pgpool_port:5432 Sep 1 11:26:33 centos8i3 pgpool[13454]: [19-2] pid 13454: DETAIL: Pgpool-II version:"4.1.2" watchdog messaging version: 1.1 Sep 1 11:26:33 centos8i3 pgpool[13454]: [20-1] pid 13454: LOG: watchdog node state changed from [JOINING] to [INITIALIZING] Sep 1 11:26:33 centos8i3 pgpool[13454]: [21-1] pid 13454: LOG: new watchdog node connection is received from "10.0.0.2:49325" Sep 1 11:26:33 centos8i3 pgpool[13454]: [22-1] pid 13454: LOG: new node joined the cluster hostname:"centos8i2-int" port:9000 pgpool_port:5432 Sep 1 11:26:33 centos8i3 pgpool[13454]: [22-2] pid 13454: DETAIL: Pgpool-II version:"4.1.2" watchdog messaging version: 1.1 Sep 1 11:26:34 centos8i3 pgpool[13454]: [23-1] pid 13454: LOG: watchdog node state changed from [INITIALIZING] to [STANDING FOR MASTER] Sep 1 11:26:34 centos8i3 pgpool[13454]: [24-1] pid 13454: LOG: our stand for coordinator request is rejected by node "centos8i1-int:5432 Linux centos8i1.localdomain" Sep 1 11:26:34 centos8i3 pgpool[13454]: [25-1] pid 13454: LOG: watchdog node state changed from [STANDING FOR MASTER] to [PARTICIPATING IN ELECTION] Sep 1 11:26:39 centos8i3 pgpool[13454]: [26-1] pid 13454: LOG: watchdog node state changed from [PARTICIPATING IN ELECTION] to [JOINING] Sep 1 11:26:39 centos8i3 pgpool[13454]: [27-1] pid 13454: LOG: watchdog node state changed from [JOINING] to [INITIALIZING] Sep 1 11:26:40 centos8i3 pgpool[13454]: [28-1] pid 13454: LOG: watchdog node state changed from [INITIALIZING] to [STANDING FOR MASTER] Sep 1 11:26:40 centos8i3 pgpool[13454]: [29-1] pid 13454: LOG: our stand for coordinator request is rejected by node "centos8i2-int:5432 Linux centos8i2.localdomain" Sep 1 11:26:40 centos8i3 pgpool[13454]: [30-1] pid 13454: LOG: watchdog node state changed from [STANDING FOR MASTER] to [PARTICIPATING IN ELECTION] Sep 1 11:26:45 centos8i3 pgpool[13454]: [31-1] pid 13454: LOG: watchdog node state changed from [PARTICIPATING IN ELECTION] to [JOINING] Sep 1 11:26:45 centos8i3 pgpool[13454]: [32-1] pid 13454: LOG: watchdog node state changed from [JOINING] to [INITIALIZING] Sep 1 11:26:46 centos8i3 pgpool[13454]: [33-1] pid 13454: LOG: watchdog node state changed from [INITIALIZING] to [STANDING FOR MASTER] Sep 1 11:26:46 centos8i3 pgpool[13454]: [34-1] pid 13454: LOG: our stand for coordinator request is rejected by node "centos8i1-int:5432 Linux centos8i1.localdomain" Sep 1 11:26:46 centos8i3 pgpool[13454]: [35-1] pid 13454: LOG: watchdog node state changed from [STANDING FOR MASTER] to [PARTICIPATING IN ELECTION] Sep 1 11:26:51 centos8i3 pgpool[13454]: [36-1] pid 13454: LOG: watchdog node state changed from [PARTICIPATING IN ELECTION] to [JOINING] Sep 1 11:26:51 centos8i3 pgpool[13454]: [37-1] pid 13454: LOG: watchdog node state changed from [JOINING] to [INITIALIZING] Sep 1 11:26:52 centos8i3 pgpool[13454]: [38-1] pid 13454: LOG: watchdog node state changed from [INITIALIZING] to [STANDING FOR MASTER] Sep 1 11:26:52 centos8i3 pgpool[13454]: [39-1] pid 13454: LOG: our stand for coordinator request is rejected by node "centos8i1-int:5432 Linux centos8i1.localdomain" Sep 1 11:26:52 centos8i3 pgpool[13454]: [40-1] pid 13454: LOG: watchdog node state changed from [STANDING FOR MASTER] to [PARTICIPATING IN ELECTION] Sep 1 11:26:56 centos8i3 pgpool[13454]: [41-1] pid 13454: LOG: watchdog node state changed from [PARTICIPATING IN ELECTION] to [INITIALIZING] Sep 1 11:26:56 centos8i3 pgpool[13454]: [42-1] pid 13454: LOG: setting the remote node "centos8i1-int:5432 Linux centos8i1.localdomain" as watchdog cluster master Sep 1 11:26:57 centos8i3 pgpool[13454]: [43-1] pid 13454: LOG: watchdog node state changed from [INITIALIZING] to [STANDBY] Sep 1 11:26:57 centos8i3 pgpool[13454]: [44-1] pid 13454: LOG: successfully joined the watchdog cluster as standby node Sep 1 11:26:57 centos8i3 pgpool[13454]: [44-2] pid 13454: DETAIL: our join coordinator request is accepted by cluster leader node "centos8i1-int:5432 Linux centos8i1.localdomain" Sep 1 11:26:57 centos8i3 pgpool[13452]: [5-1] pid 13452: LOG: watchdog process is initialized Sep 1 11:26:57 centos8i3 pgpool[13452]: [5-2] pid 13452: DETAIL: watchdog messaging data version: 1.1 Sep 1 11:26:57 centos8i3 pgpool[13454]: [45-1] pid 13454: LOG: new IPC connection received Sep 1 11:26:57 centos8i3 pgpool[13452]: [6-1] pid 13452: LOG: we have joined the watchdog cluster as STANDBY node Sep 1 11:26:57 centos8i3 pgpool[13452]: [6-2] pid 13452: DETAIL: syncing the backend states from the MASTER watchdog node Sep 1 11:26:57 centos8i3 pgpool[13454]: [46-1] pid 13454: LOG: new IPC connection received Sep 1 11:26:57 centos8i3 pgpool[13454]: [47-1] pid 13454: LOG: new IPC connection received Sep 1 11:26:57 centos8i3 pgpool[13454]: [48-1] pid 13454: LOG: received the get data request from local pgpool-II on IPC interface Sep 1 11:26:57 centos8i3 pgpool[13454]: [49-1] pid 13454: LOG: get data request from local pgpool-II node received on IPC interface is forwarded to master watchdog node "centos8i1-int:5432 Linux centos8i1.localdomain" Sep 1 11:26:57 centos8i3 pgpool[13454]: [49-2] pid 13454: DETAIL: waiting for the reply... Sep 1 11:26:57 centos8i3 pgpool[13480]: [6-1] pid 13480: LOG: 3 watchdog nodes are configured for lifecheck Sep 1 11:26:57 centos8i3 pgpool[13480]: [7-1] pid 13480: LOG: watchdog nodes ID:0 Name:"centos8i3-int:5432 Linux centos8i3.localdomain" Sep 1 11:26:57 centos8i3 pgpool[13480]: [7-2] pid 13480: DETAIL: Host:"centos8i3-int" WD Port:9000 pgpool-II port:5432 Sep 1 11:26:57 centos8i3 pgpool[13480]: [8-1] pid 13480: LOG: watchdog nodes ID:1 Name:"centos8i1-int:5432 Linux centos8i1.localdomain" Sep 1 11:26:57 centos8i3 pgpool[13480]: [8-2] pid 13480: DETAIL: Host:"centos8i1-int" WD Port:9000 pgpool-II port:5432 Sep 1 11:26:57 centos8i3 pgpool[13452]: [7-1] pid 13452: LOG: master watchdog node "centos8i1-int:5432 Linux centos8i1.localdomain" returned status for 3 backend nodes Sep 1 11:26:57 centos8i3 pgpool[13480]: [9-1] pid 13480: LOG: watchdog nodes ID:2 Name:"centos8i2-int:5432 Linux centos8i2.localdomain" Sep 1 11:26:57 centos8i3 pgpool[13480]: [9-2] pid 13480: DETAIL: Host:"centos8i2-int" WD Port:9000 pgpool-II port:5432 Sep 1 11:26:57 centos8i3 pgpool[13452]: [8-1] pid 13452: LOG: Setting up socket for 0.0.0.0:5432 Sep 1 11:26:57 centos8i3 pgpool[13452]: [9-1] pid 13452: LOG: Setting up socket for :::5432 Sep 1 11:26:57 centos8i3 pgpool[13517]: [10-1] pid 13517: LOG: PCP process: 13517 started Sep 1 11:26:57 centos8i3 pgpool[13452]: [10-1] pid 13452: LOG: pgpool-II successfully started. version 4.1.2 (karasukiboshi) Sep 1 11:26:57 centos8i3 pgpool[13452]: [11-1] pid 13452: LOG: node status[0]: 0 Sep 1 11:26:57 centos8i3 pgpool[13452]: [12-1] pid 13452: LOG: node status[1]: 0 Sep 1 11:26:57 centos8i3 pgpool[13452]: [13-1] pid 13452: LOG: node status[2]: 0 Sep 1 11:26:58 centos8i3 pgpool[13484]: [10-1] pid 13484: LOG: set SO_REUSEPORT option to the socket Sep 1 11:26:58 centos8i3 pgpool[13484]: [11-1] pid 13484: LOG: creating watchdog heartbeat receive socket. Sep 1 11:26:58 centos8i3 pgpool[13484]: [11-2] pid 13484: DETAIL: set SO_REUSEPORT Sep 1 11:26:58 centos8i3 pgpool[13486]: [10-1] pid 13486: LOG: set SO_REUSEPORT option to the socket Sep 1 11:26:58 centos8i3 pgpool[13486]: [11-1] pid 13486: LOG: creating socket for sending heartbeat Sep 1 11:26:58 centos8i3 pgpool[13486]: [11-2] pid 13486: DETAIL: set SO_REUSEPORT Sep 1 11:26:58 centos8i3 pgpool[13482]: [10-1] pid 13482: LOG: set SO_REUSEPORT option to the socket Sep 1 11:26:58 centos8i3 pgpool[13482]: [11-1] pid 13482: LOG: creating socket for sending heartbeat Sep 1 11:26:58 centos8i3 pgpool[13482]: [11-2] pid 13482: DETAIL: set SO_REUSEPORT Sep 1 11:26:58 centos8i3 pgpool[13481]: [10-1] pid 13481: LOG: set SO_REUSEPORT option to the socket Sep 1 11:26:58 centos8i3 pgpool[13481]: [11-1] pid 13481: LOG: creating watchdog heartbeat receive socket. Sep 1 11:26:58 centos8i3 pgpool[13481]: [11-2] pid 13481: DETAIL: set SO_REUSEPORT ***** POWER OFF NODE centos8i2-int ***** Sep 1 11:27:52 centos8i3 pgpool[13520]: [10-1] pid 13520: LOG: failed to connect to PostgreSQL server on "centos8i2-int:5433", timed out Sep 1 11:27:52 centos8i3 pgpool[13520]: [11-1] pid 13520: ERROR: failed to make persistent db connection Sep 1 11:27:52 centos8i3 pgpool[13520]: [11-2] pid 13520: DETAIL: connection to host:"centos8i2-int:5433" failed Sep 1 11:27:52 centos8i3 pgpool[13520]: [12-1] pid 13520: LOG: health check retrying on DB node: 1 (round:1) Sep 1 11:27:57 centos8i3 pgpool[13518]: [10-1] pid 13518: LOG: trying connecting to PostgreSQL server on "centos8i2-int:5433" by INET socket Sep 1 11:27:57 centos8i3 pgpool[13518]: [10-2] pid 13518: DETAIL: timed out. retrying... Sep 1 11:28:03 centos8i3 pgpool[13520]: [13-1] pid 13520: LOG: failed to connect to PostgreSQL server on "centos8i2-int:5433", timed out Sep 1 11:28:03 centos8i3 pgpool[13520]: [14-1] pid 13520: ERROR: failed to make persistent db connection Sep 1 11:28:03 centos8i3 pgpool[13520]: [14-2] pid 13520: DETAIL: connection to host:"centos8i2-int:5433" failed Sep 1 11:28:03 centos8i3 pgpool[13520]: [15-1] pid 13520: LOG: health check retrying on DB node: 1 (round:2) Sep 1 11:28:07 centos8i3 pgpool[13518]: [11-1] pid 13518: LOG: trying connecting to PostgreSQL server on "centos8i2-int:5433" by INET socket Sep 1 11:28:07 centos8i3 pgpool[13518]: [11-2] pid 13518: DETAIL: timed out. retrying... Sep 1 11:28:14 centos8i3 pgpool[13520]: [16-1] pid 13520: LOG: failed to connect to PostgreSQL server on "centos8i2-int:5433", timed out Sep 1 11:28:14 centos8i3 pgpool[13520]: [17-1] pid 13520: ERROR: failed to make persistent db connection Sep 1 11:28:14 centos8i3 pgpool[13520]: [17-2] pid 13520: DETAIL: connection to host:"centos8i2-int:5433" failed Sep 1 11:28:14 centos8i3 pgpool[13520]: [18-1] pid 13520: LOG: health check retrying on DB node: 1 (round:3) Sep 1 11:28:17 centos8i3 pgpool[13518]: [12-1] pid 13518: LOG: trying connecting to PostgreSQL server on "centos8i2-int:5433" by INET socket Sep 1 11:28:17 centos8i3 pgpool[13518]: [12-2] pid 13518: DETAIL: timed out. retrying... Sep 1 11:28:25 centos8i3 pgpool[13520]: [19-1] pid 13520: LOG: failed to connect to PostgreSQL server on "centos8i2-int:5433", timed out Sep 1 11:28:25 centos8i3 pgpool[13520]: [20-1] pid 13520: ERROR: failed to make persistent db connection Sep 1 11:28:25 centos8i3 pgpool[13520]: [20-2] pid 13520: DETAIL: connection to host:"centos8i2-int:5433" failed Sep 1 11:28:25 centos8i3 pgpool[13520]: [21-1] pid 13520: LOG: health check failed on node 1 (timeout:0) Sep 1 11:28:25 centos8i3 pgpool[13520]: [22-1] pid 13520: LOG: received degenerate backend request for node_id: 1 from pid [13520] Sep 1 11:28:25 centos8i3 pgpool[13454]: [50-1] pid 13454: LOG: new IPC connection received Sep 1 11:28:25 centos8i3 pgpool[13454]: [51-1] pid 13454: LOG: failover request from local pgpool-II node received on IPC interface is forwarded to master watchdog node "centos8i1-int:5432 Linux centos8i1.localdomain" Sep 1 11:28:25 centos8i3 pgpool[13454]: [51-2] pid 13454: DETAIL: waiting for the reply... Sep 1 11:28:25 centos8i3 pgpool[13454]: [52-1] pid 13454: LOG: remote node "centos8i1-int:5432 Linux centos8i1.localdomain" is asking to inform about quarantined backend nodes Sep 1 11:28:25 centos8i3 pgpool[13452]: [14-1] pid 13452: LOG: Pgpool-II parent process received watchdog quorum change signal from watchdog Sep 1 11:28:25 centos8i3 pgpool[13520]: [23-1] pid 13520: LOG: degenerate backend request for node_id: 1 from pid [13520], will be handled by watchdog, which is building consensus for request Sep 1 11:28:25 centos8i3 pgpool[13454]: [53-1] pid 13454: LOG: new IPC connection received Sep 1 11:28:25 centos8i3 pgpool[13452]: [15-1] pid 13452: LOG: watchdog cluster now holds the quorum Sep 1 11:28:25 centos8i3 pgpool[13452]: [15-2] pid 13452: DETAIL: updating the state of quarantine backend nodes Sep 1 11:28:25 centos8i3 pgpool[13454]: [54-1] pid 13454: LOG: new IPC connection received Sep 1 11:28:25 centos8i3 pgpool[13452]: [16-1] pid 13452: LOG: Pgpool-II parent process received inform quarantine nodes signal from watchdog Sep 1 11:28:27 centos8i3 pgpool[13518]: [13-1] pid 13518: LOG: trying connecting to PostgreSQL server on "centos8i2-int:5433" by INET socket Sep 1 11:28:27 centos8i3 pgpool[13518]: [13-2] pid 13518: DETAIL: timed out. retrying... Sep 1 11:28:30 centos8i3 pgpool[13452]: [17-1] pid 13452: LOG: Pgpool-II parent process received sync backend signal from watchdog Sep 1 11:28:30 centos8i3 pgpool[13454]: [55-1] pid 13454: LOG: new IPC connection received Sep 1 11:28:30 centos8i3 pgpool[13452]: [18-1] pid 13452: LOG: master watchdog has performed failover Sep 1 11:28:30 centos8i3 pgpool[13452]: [18-2] pid 13452: DETAIL: syncing the backend states from the MASTER watchdog node Sep 1 11:28:30 centos8i3 pgpool[13454]: [56-1] pid 13454: LOG: new IPC connection received Sep 1 11:28:30 centos8i3 pgpool[13454]: [57-1] pid 13454: LOG: received the get data request from local pgpool-II on IPC interface Sep 1 11:28:30 centos8i3 pgpool[13454]: [58-1] pid 13454: LOG: get data request from local pgpool-II node received on IPC interface is forwarded to master watchdog node "centos8i1-int:5432 Linux centos8i1.localdomain" Sep 1 11:28:30 centos8i3 pgpool[13454]: [58-2] pid 13454: DETAIL: waiting for the reply... Sep 1 11:28:30 centos8i3 pgpool[13452]: [19-1] pid 13452: LOG: master watchdog node "centos8i1-int:5432 Linux centos8i1.localdomain" returned status for 3 backend nodes Sep 1 11:28:30 centos8i3 pgpool[13452]: [20-1] pid 13452: LOG: backend:1 is set to down status Sep 1 11:28:30 centos8i3 pgpool[13452]: [20-2] pid 13452: DETAIL: backend:1 is DOWN on cluster master "centos8i1-int:5432 Linux centos8i1.localdomain" Sep 1 11:28:30 centos8i3 pgpool[13452]: [21-1] pid 13452: LOG: 1 backend node(s) were detached because of backend status sync from "centos8i1-int:5432 Linux centos8i1.localdomain" Sep 1 11:28:30 centos8i3 pgpool[13452]: [21-2] pid 13452: DETAIL: restarting the children processes Sep 1 11:28:30 centos8i3 pgpool[13452]: [22-1] pid 13452: LOG: Node 0 is not down (status: 1) Sep 1 11:28:30 centos8i3 pgpool[13518]: [14-1] pid 13518: LOG: trying to connect to PostgreSQL server on "centos8i2-int:5433" using INET socket Sep 1 11:28:30 centos8i3 pgpool[13518]: [14-2] pid 13518: DETAIL: select() interrupted. retrying... Sep 1 11:28:34 centos8i3 pgpool[13520]: [24-1] pid 13520: LOG: failed to connect to PostgreSQL server on "centos8i2-int:5433", getsockopt() detected error "No route to host" Sep 1 11:28:34 centos8i3 pgpool[13520]: [25-1] pid 13520: ERROR: failed to make persistent db connection Sep 1 11:28:34 centos8i3 pgpool[13520]: [25-2] pid 13520: DETAIL: connection to host:"centos8i2-int:5433" failed Sep 1 11:28:34 centos8i3 pgpool[13520]: [26-1] pid 13520: LOG: health check retrying on DB node: 1 (round:1) Sep 1 11:28:37 centos8i3 pgpool[13520]: [27-1] pid 13520: LOG: failed to connect to PostgreSQL server on "centos8i2-int:5433", getsockopt() detected error "No route to host" Sep 1 11:28:37 centos8i3 pgpool[13520]: [28-1] pid 13520: ERROR: failed to make persistent db connection Sep 1 11:28:37 centos8i3 pgpool[13520]: [28-2] pid 13520: DETAIL: connection to host:"centos8i2-int:5433" failed Sep 1 11:28:37 centos8i3 pgpool[13520]: [29-1] pid 13520: LOG: health check retrying on DB node: 1 (round:2) Sep 1 11:28:40 centos8i3 pgpool[13518]: [15-1] pid 13518: LOG: trying connecting to PostgreSQL server on "centos8i2-int:5433" by INET socket Sep 1 11:28:40 centos8i3 pgpool[13518]: [15-2] pid 13518: DETAIL: timed out. retrying... Sep 1 11:28:40 centos8i3 pgpool[13520]: [30-1] pid 13520: LOG: failed to connect to PostgreSQL server on "centos8i2-int:5433", getsockopt() detected error "No route to host" Sep 1 11:28:40 centos8i3 pgpool[13520]: [31-1] pid 13520: ERROR: failed to make persistent db connection Sep 1 11:28:40 centos8i3 pgpool[13520]: [31-2] pid 13520: DETAIL: connection to host:"centos8i2-int:5433" failed Sep 1 11:28:40 centos8i3 pgpool[13520]: [32-1] pid 13520: LOG: health check retrying on DB node: 1 (round:3) Sep 1 11:28:43 centos8i3 pgpool[13520]: [33-1] pid 13520: LOG: failed to connect to PostgreSQL server on "centos8i2-int:5433", getsockopt() detected error "No route to host" Sep 1 11:28:43 centos8i3 pgpool[13520]: [34-1] pid 13520: ERROR: failed to make persistent db connection Sep 1 11:28:43 centos8i3 pgpool[13520]: [34-2] pid 13520: DETAIL: connection to host:"centos8i2-int:5433" failed Sep 1 11:28:50 centos8i3 pgpool[13520]: [35-1] pid 13520: LOG: failed to connect to PostgreSQL server on "centos8i2-int:5433", getsockopt() detected error "No route to host" Sep 1 11:28:50 centos8i3 pgpool[13520]: [36-1] pid 13520: ERROR: failed to make persistent db connection Sep 1 11:28:50 centos8i3 pgpool[13520]: [36-2] pid 13520: DETAIL: connection to host:"centos8i2-int:5433" failed Sep 1 11:28:50 centos8i3 pgpool[13520]: [37-1] pid 13520: LOG: health check retrying on DB node: 1 (round:1) Sep 1 11:28:50 centos8i3 pgpool[13518]: [16-1] pid 13518: LOG: trying connecting to PostgreSQL server on "centos8i2-int:5433" by INET socket Sep 1 11:28:50 centos8i3 pgpool[13518]: [16-2] pid 13518: DETAIL: timed out. retrying... Sep 1 11:28:53 centos8i3 pgpool[13520]: [38-1] pid 13520: LOG: failed to connect to PostgreSQL server on "centos8i2-int:5433", getsockopt() detected error "No route to host" Sep 1 11:28:53 centos8i3 pgpool[13518]: [17-1] pid 13518: LOG: failed to connect to PostgreSQL server on "centos8i2-int:5433", getsockopt() detected error "No route to host" Sep 1 11:28:53 centos8i3 pgpool[13520]: [39-1] pid 13520: ERROR: failed to make persistent db connection Sep 1 11:28:53 centos8i3 pgpool[13518]: [18-1] pid 13518: ERROR: failed to make persistent db connection Sep 1 11:28:53 centos8i3 pgpool[13520]: [39-2] pid 13520: DETAIL: connection to host:"centos8i2-int:5433" failed Sep 1 11:28:53 centos8i3 pgpool[13518]: [18-2] pid 13518: DETAIL: connection to host:"centos8i2-int:5433" failed Sep 1 11:28:53 centos8i3 pgpool[13520]: [40-1] pid 13520: LOG: health check retrying on DB node: 1 (round:2) Sep 1 11:28:56 centos8i3 pgpool[13520]: [41-1] pid 13520: LOG: failed to connect to PostgreSQL server on "centos8i2-int:5433", getsockopt() detected error "No route to host" Sep 1 11:28:56 centos8i3 pgpool[13520]: [42-1] pid 13520: ERROR: failed to make persistent db connection Sep 1 11:28:56 centos8i3 pgpool[13520]: [42-2] pid 13520: DETAIL: connection to host:"centos8i2-int:5433" failed Sep 1 11:28:56 centos8i3 pgpool[13520]: [43-1] pid 13520: LOG: health check retrying on DB node: 1 (round:3) Sep 1 11:28:59 centos8i3 pgpool[13520]: [44-1] pid 13520: LOG: failed to connect to PostgreSQL server on "centos8i2-int:5433", getsockopt() detected error "No route to host" Sep 1 11:28:59 centos8i3 pgpool[13520]: [45-1] pid 13520: ERROR: failed to make persistent db connection Sep 1 11:28:59 centos8i3 pgpool[13520]: [45-2] pid 13520: DETAIL: connection to host:"centos8i2-int:5433" failed Sep 1 11:29:03 centos8i3 pgpool[13518]: [19-1] pid 13518: ERROR: Failed to check replication time lag Sep 1 11:29:03 centos8i3 pgpool[13518]: [19-2] pid 13518: DETAIL: No persistent db connection for the node 1 Sep 1 11:29:03 centos8i3 pgpool[13518]: [19-3] pid 13518: HINT: check sr_check_user and sr_check_password Sep 1 11:29:03 centos8i3 pgpool[13518]: [19-4] pid 13518: CONTEXT: while checking replication time lag Sep 1 11:29:03 centos8i3 pgpool[13518]: [20-1] pid 13518: LOG: worker process received restart request Sep 1 11:29:03 centos8i3 pgpool[13452]: [23-1] pid 13452: LOG: worker child process with pid: 13518 exits with status 256 Sep 1 11:29:03 centos8i3 pgpool[13452]: [24-1] pid 13452: LOG: fork a new worker child process with pid: 13767 Sep 1 11:30:44 centos8i3 pgpool[13517]: [11-1] pid 13517: LOG: forked new pcp worker, pid=13889 socket=9 Sep 1 11:30:44 centos8i3 pgpool[13454]: [59-1] pid 13454: LOG: new IPC connection received Sep 1 11:30:44 centos8i3 pgpool[13517]: [12-1] pid 13517: LOG: PCP process with pid: 13889 exit with SUCCESS. Sep 1 11:30:44 centos8i3 pgpool[13517]: [13-1] pid 13517: LOG: PCP process with pid: 13889 exits with status 0