[pgpool-general: 6247] Re: FATAL: Add to watchdog cluster request is rejected by node "redshirt2018090300:9000"

[Sebastian Stellingwerff]

Should perhaps have gone to the list: 

>It's the -other- node that is rejecting the node due to the configuration. When I start the rejected one first, the other one is rejected by that one. This leads me to believe there is a problem with differences in the configuration, if there was a configuration problem in itself, I would expect it's own process to complain. 
>As this is an isolated test setup, there is no firewall on the machines. 
>The past in the previous mail was the -diff- of the two configurations, as other_pgpool_port and other_wd_port are the same on both hosts, they are not shown. 

I started looking at the source code, and this seems to the relevant check: 

if ((wdNode->wd_port == tempNode->wd_port && wdNode->pgpool_port == tempNode->pgpool_port) && 
((strcmp(wdNode->hostname, conn->addr) == 0) || (strcmp(wdNode->hostname, tempNode->hostname) == 0))) 

This led me to notice 'wdNode->pgpool_port == tempNode->pgpool_port'. 

other_pgpool_port0=5433 on both sides, so both the configurations where right. but the port field was set to 5432 on both sides. 

So this bit matches 'other_pgpool_port0' on one side with 'port' on the other side, which makes sense. 

I would like to kindly request that the error states which fields are unmatched. This one 'if' has a lot of (at least 13) cases how it could fail, counting only when a field is unmatched, not which. Some matches like 'strcmp(wdNode->hostname, conn->addr)' might come from unexpected angles. 
A simple 'verify the other watchdog node configurations' doesn't really cut it I think. 

Thanks! 

Sebastian 

> From: "Pierre Timmermans" <ptim007 at yahoo.com>
> To: "pgpool-general" <pgpool-general at pgpool.net>, "Sebastian Stellingwerff"
> <sebas at bakkerspees.nl>
> Sent: Monday, October 8, 2018 8:30:04 PM
> Subject: Re: [pgpool-general: 6245] FATAL: Add to watchdog cluster request is
> rejected by node "redshirt2018090300:9000"

> Not sure if it is important but when I look at my configuration I have some
> extra variables: other_pgpool_port and other_wd_port. Since it complains about
> the config, maybe it is relevant ?

> Also make sure that firewall ports 9694 and 9000 are open on both servers

> Here is my config (3 servers)

> heartbeat_destination0 = 'pgpool01'
> heartbeat_destination_port0 = 9694

> heartbeat_destination1 = 'pgpool02'
> heartbeat_destination_port1 = 9694

> heartbeat_destination2 = 'pgpool03'
> heartbeat_destination_port2 = 9694

> other_pgpool_hostname0 = 'pgpool02'
> other_pgpool_port0 = 9999
> other_wd_port0 = 9000

> other_pgpool_hostname1 = 'pgpool03'
> other_pgpool_port1 = 9999
> other_wd_port1 = 9000

> Rgds,

> Pierre

> On Monday, October 8, 2018, 4:59:15 PM GMT+2, Sebastian Stellingwerff
> <sebas at bakkerspees.nl> wrote:

> I've setup a cluster with two nodes, but I run into this problem, one of them
> dies with the error:

> FATAL: Add to watchdog cluster request is rejected by node
> "redshirt2018090300:9000"
> HINT: check the watchdog configurations.

> Now I've checked the configurations, and run a diff on the two, but the only
> thing that differs is the hostnames.

> 125c125
> < wd_hostname = 'redshirt2018090300'
> ---
> > wd_hostname = 'redshirt2018091100'
> 157c157
> < heartbeat_destination0 = 'redshirt2018091100'
> ---
> > heartbeat_destination0 = 'redshirt2018090300'
> 160c160
> < other_pgpool_hostname0 = 'redshirt2018091100'
> ---
> > other_pgpool_hostname0 = 'redshirt2018090300'

> Is there another way of seeing what else this could be? I already set loglevel
> to 3 and on verbose.

> Thanks!

> sebastian
> _______________________________________________
> pgpool-general mailing list
> [ mailto:pgpool-general at pgpool.net | pgpool-general at pgpool.net ]
> [ http://www.pgpool.net/mailman/listinfo/pgpool-general |
> http://www.pgpool.net/mailman/listinfo/pgpool-general ]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.pgpool.net/pipermail/pgpool-general/attachments/20181015/175ec2d6/attachment.htm>