[pgpool-general: 5864] Troubleshooting assistance on node failure and connections blocking

Mike Koepke mike.koepke at gmail.com
Wed Jan 17 02:04:25 JST 2018


All,


We are trying to trace down a problem that looks to be with our pgpool
configuration or operation



We have 2 nodes running.   The stack on each node is:



Our app

HikariCP

Jdbc

Pg-pool 3.6.7

Postgres 9.6.6



Pgpool is set for Master-Slave with streaming replication; no load
balancing.



We are testing our disaster recovery and failover capabilities.    If we
gracefully shutdown node 1 (2nd node), the 1st node proceeds as is nothing
happened.  The app continues to run without missing a beat. As you would
expect.



Our problem is when we encounter a “hard” error.  If node 1 becomes
disconnected (network is removed), node 0 becomes impacted.   The app will
freeze up as it can no longer get database connections.   We see the
app/spring talk to Hikari, Hikari talks to jdbc, jdbc cannot get connection
, eventaully Hikari times out (with 30 sec connection wait) and reples to
app and we get exceptions.  This repeats as the app continues to try talk
to the database.    Pgpool is aware that the node1 is gone as it is in
recovery mode and node 0 pgpool retries to establish connectivity to pgpool
on node 1 per pgpool.conf intervals.



So the thing that really has us stumped is if node 0 is only talking
through it’s stack to node 0 postgres, why is this failure on node1 having
any impact on node 0 and freezing the db connections?    Obviously when a
graceful shutdown occurs pgpool graceful handles this and things work as
you expect.  With a hard failure, it does not.    I have attached our
pgpool.conf file.   Can someone provide some guidance into the internals of
pgpool and why this node1 hard failure causes node 0 impacts?



Thanks



Mike
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.sraoss.jp/pipermail/pgpool-general/attachments/20180116/d8484ff2/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pgpool.conf
Type: application/octet-stream
Size: 33859 bytes
Desc: not available
URL: <http://www.sraoss.jp/pipermail/pgpool-general/attachments/20180116/d8484ff2/attachment-0001.obj>


More information about the pgpool-general mailing list