[pgpool-general: 4664] Re: Problem shutting down interfaces

[Lazar Krumov]

Hi Ricardo, 

I'm experiencing a similar problem with PgPool failover. 
I am running 2 PgPool servers version 3.5.2 with watchdog - Active and Standby PgPool instances, running a virtual IP address (pgpool.cong: delegate_IP parameter).
When a failover event appears, the StandBy server activates the virtual IP address. The clients which are in the same subnet see the new active PgPool with very short delay. The clients which are in other IP network segments see the new active PgPool server with more than 30 min. delay! 

Investigating of the problem found that the ARP table of the router-switch (L3 Cisco switch), responsible for routing the IP-network where the virtual PgPool IP address is, is not updated adequately. The old ARP entry, pointing the the old active PgPool server, is kept for more than 30 min. If I replace the ARP entry in the router-switch, the connections become possible immediately. 

Currently I'm investigating the problem. It's rather a network problem, than PGPOOL problem.
Probably important details are:
- my virtual IP address and the primary interface of both PgPool servers are in the same IP-network segment
- my servers are GNU/Debian Linux and I use bond fromm 4 physical UTP cards. The virtual IP is a second IP-address of the bond interface.
- the two PgPool servers are connected on different LAN swtches. These two switches are uplinked through the router-switch, the same router-switch which cause the problem with ARP table update.

So you could check if your problem is not a netowrk problem, as mine seems to be.

Best Regards,
Lazar Krumov



> ------------------------------
> 
> Message: 2
> Date: Thu, 5 May 2016 16:55:02 -0300
> From: Ricardo Larra?aga <ricardo.larranaga at gmail.com>
> To: pgpool-general at pgpool.net
> Subject: [pgpool-general: 4663] Problem shutting down interfaces
> Message-ID:
> 	<CAJ6jHN8FzrzO1mpe0xc7EFh4XVsSmXA5_R=O+bj8nyyYEmEjkw at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
> 
> Hello guys:
> I am running pgpool 3.5.2 in a 2 node cluster with postgresql-9.5 in
> master-slave and streaming replication mode.
> 
> I have been testing failover and failback for a while. While doing failover
> by shutting down the processes, everything looks fine.
> 
> 
> The one test that is failing is when i shutdown the interface of one node.
> Right now i am shutting down the interface of the slave node (both pgool
> slave and postgresql slave).
> 
> The problem i am running into is that after doing this, all connections to
> my database (through pgpool) hang. i am testing it with psql, and psql just
> hangs and does not give me an output. When i bring the interface of the
> slave back up, and try to connect to psql again, it looks like pgpool never
> marked the postgresql node as disconnected.
> 
> I tried both with and without health check, and also tried with different
> values of health_check_timeout. my connect timeout value is default (10
> seconds).
> 
> Any one has encountered this issue? I just dont see pgpool attempting to do
> any failover. Pgpool is still running though, i can see the logs still
> comming. I just never see an error.
> 
> I am NOT using interface monitoring, and i would prefer not to use it.
> 
> Any pointers on how could i troubleshoot this?
> Thanks.
> Regards
> End of pgpool-general Digest, Vol 55, Issue 3
> *********************************************
>