<div dir="ltr">Hi Everyone,<div><br></div><div>I seem to have situations where BOTH pgpool servers will assume control of the same VIP, which causes obvious networking issues. The main scenario is when both servers are rebooted, as there is no watchdog running so I presume they just take control of the VIP. The issue is that they never seem to renegotiate, so the VIP will permanently stay assigned to both hosts.</div><div><br></div><div>If I start one, wait for a minute and then start the other, things work fine. The VIP is assigned to the first host that boots, and will automatically fail-over to the second host if pgpool is stopped.<br><div><br></div><div>I have a pretty basic setup with 2 x pgpool and 2 x PostgreSQL servers, all running on Ubuntu. The versions are:</div><div><br></div></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div><div><font face="monospace, monospace">pgpool-II v3.2.4</font></div></div><div><div><font face="monospace, monospace">PostgreSQL v9.3.6 (Streaming Replication)</font></div></div><div><div><font face="monospace, monospace">Ubuntu v14.04</font></div></div></blockquote><div><div><br></div><div><br></div><div>I've attached the pgpool.conf files from both server1, and server2, but here is a diff to make life easier:</div><div><br></div></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div><div><div><font face="monospace, monospace">greg:~$ diff server1.pgpool.conf server2.pgpool.conf</font></div></div></div><div><div><font face="monospace, monospace">443c443</font></div></div><div><div><font face="monospace, monospace">< wd_hostname = 'PGPOOL_SERVER_01'</font></div></div><div><div><font face="monospace, monospace">---</font></div></div><div><div><font face="monospace, monospace">> wd_hostname = 'PGPOOL_SERVER_02'</font></div></div><div><div><font face="monospace, monospace">669c669</font></div></div><div><div><font face="monospace, monospace">< other_pgpool_hostname0 = 'PGPOOL_SERVER_02'</font></div></div><div><div><font face="monospace, monospace">---</font></div></div><div><div><font face="monospace, monospace">> other_pgpool_hostname0 = 'PGPOOL_SERVER_01'</font></div></div><div><div><font face="monospace, monospace">673c673</font></div></div><div><div><font face="monospace, monospace">< heartbeat_destination0 = 'PGPOOL_SERVER_02'</font></div></div><div><div><font face="monospace, monospace">---</font></div></div><div><div><font face="monospace, monospace">> heartbeat_destination0 = 'PGPOOL_SERVER_01'</font></div></div></blockquote><div><div><br></div><div><br></div><div>One thing to note is that I'm using SUID versions of ifconfig and arping as described here:</div><div><br></div></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><div><font face="monospace, monospace"><a href="http://www.pgpool.net/pgpool-web/contrib_docs/watchdog_master_slave/en.html#config_setuid">http://www.pgpool.net/pgpool-web/contrib_docs/watchdog_master_slave/en.html#config_setuid</a></font></div></div></blockquote><div><div><br></div><div><br></div><div>Basically I have copies of those binaries that are group executable by postgres, and SUID root:</div><div><br></div></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><div><div><font face="monospace, monospace">root:/var/lib/postgres/sbin# ls -al</font></div></div></div><div><div><div><font face="monospace, monospace">total 104</font></div></div></div><div><div><div><font face="monospace, monospace">drwx------ 2 postgres postgres 4096 Jun 1 10:41 .</font></div></div></div><div><div><div><font face="monospace, monospace">drwx------ 5 postgres postgres 4096 Jun 1 10:39 ..</font></div></div></div><div><div><div><font face="monospace, monospace">-rwsr-x--- 1 root postgres 26592 Jun 1 10:41 arping</font></div></div></div><div><div><div><font face="monospace, monospace">-rwsr-x--- 1 root postgres 68040 Jun 1 10:40 ifconfig</font></div></div></div></blockquote><div><div><br></div><div>... but with the switchover/switchback working, I don't think that's the cause. To me it's a race condition, as there is no way to lock an IP before it's used... and whilst both of the servers probably check to see if it's being used before assuming control of it, if they both check at the same time then there is a window when they can both assume control of it.</div><div><br></div><div>My guess as to a fix would be that once a server finds a watchdog partner, they should renegotiate on who should control any addresses... thus it may get double-allocated for a short while, but that will get automatically fixed once they establish a watchdog connection to each other.</div><div><br></div><div>Either that, or I'm missing something obvious in the config files, so feel free to point out my idiocy.</div><div><br></div><div>Many Thanks.</div><div><b><span style="font-family:Helvetica,sans-serif;color:rgb(68,68,68)">Greg Clough</span></b></div><div><div class="gmail_signature"><div dir="ltr"><div dir="ltr"><p style="margin-right:0cm;margin-bottom:1.5pt;margin-left:0cm"><b><span style="font-family:Helvetica,sans-serif;color:rgb(68,68,68)">Contract Database Administrator | CWT Digital</span></b></p><p style="margin-right:0cm;margin-bottom:1.5pt;margin-left:0cm"><b style="line-height:28.5pt"><span style="font-family:Helvetica,sans-serif;color:rgb(68,68,68)">t.</span></b><span style="line-height:28.5pt;font-family:Helvetica,sans-serif;color:rgb(68,68,68)"> 0845 456 0070 <b>w.</b> </span><span style="line-height:28.5pt;font-family:Helvetica,sans-serif"><a href="http://www.cwtdigital.com/" style="color:rgb(17,85,204)" target="_blank"><span style="color:rgb(255,153,0)">cwtdigital.com</span></a></span></p><p style="margin-right:0cm;margin-bottom:1.5pt;margin-left:0cm"><span style="font-size:12pt;font-family:'Times New Roman',serif"></span><span style="font-size:9pt;font-family:Helvetica,sans-serif"><u></u><u></u></span></p><p style="margin-right:0cm;margin-bottom:1.5pt;margin-left:0cm"><img src="http://www.cwtdigital.com/hosted/sig.png"></p><p style="margin-right:0cm;margin-bottom:1.5pt;margin-left:0cm"><br><span style="font-family:Helvetica,sans-serif;color:rgb(68,68,68)">Units A/B, Level 8 North, New England House, <br>New England Street, Brighton, East Sussex, BN1 4GH</span></p><p><a href="https://twitter.com/cwtdigital" style="color:rgb(17,85,204)" target="_blank"><span style="font-family:Arial,sans-serif;text-decoration:none;background-image:initial;background-repeat:initial"><img border="0" width="20" height="20" src="cid:image002.png@01CFC827.D51F6D00" alt="https://ci6.googleusercontent.com/proxy/spbuXLCE_kiYn1CBmHRh6DoZX1WwETayC0rma88wNedREVrYPrjvkpQLcBPeuKfSkfkOpDSkuDvye9g2Ps4-BW4s=s0-d-e1-ft#http://www.cwtdigital.com/hosted/twitter.png"></span></a><span style="font-family:Arial,sans-serif;background-image:initial;background-repeat:initial"> </span><a href="https://www.facebook.com/CWTdigital" style="color:rgb(17,85,204)" target="_blank"><span style="font-family:Arial,sans-serif;text-decoration:none;background-image:initial;background-repeat:initial"><img border="0" width="20" height="20" src="cid:image003.png@01CFC827.D51F6D00" alt="https://ci5.googleusercontent.com/proxy/q4GOyacafkMnvs7TLpy3rOcwVK_uxMoECtqVEVpwmxiLShfkGNOZGisnwu4oHGtcEK_C-TfrLguFshp1VKC-5zZw6g=s0-d-e1-ft#http://www.cwtdigital.com/hosted/facebook.png"></span></a><span style="font-family:Arial,sans-serif;background-image:initial;background-repeat:initial"> </span><a href="https://www.linkedin.com/company/cwt-digital" style="color:rgb(17,85,204)" target="_blank"><span style="font-family:Arial,sans-serif;text-decoration:none;background-image:initial;background-repeat:initial"><img border="0" width="21" height="20" src="cid:image004.png@01CFC827.D51F6D00" alt="https://ci4.googleusercontent.com/proxy/BVh2CnIvbQ_Rp5LFVzt4p72zUvhG3u7ksBMtlWRjbWjXL-DXIFEXXZJCYWMlSd3MshfG4UN0NAuTj1eElX8r5iu2KQ=s0-d-e1-ft#http://www.cwtdigital.com/hosted/linkedin.png"></span></a><span style="font-family:Arial,sans-serif;background-image:initial;background-repeat:initial"> </span><a href="https://plus.google.com/s/CWT%20Digital" style="color:rgb(17,85,204)" target="_blank"><span style="font-family:Arial,sans-serif;text-decoration:none;background-image:initial;background-repeat:initial"><img border="0" width="20" height="20" src="cid:image005.png@01CFC827.D51F6D00" alt="https://ci4.googleusercontent.com/proxy/QJJKpRXCvrkfCat5ycimzE7Lve3Pp9wFZWL5eBF5ELDkpl_gA8UVOc-R2p29DTS5DJmfD-FHGg2cjSf4lEuz2Ts=s0-d-e1-ft#http://www.cwtdigital.com/hosted/google.png"></span></a><span style="font-family:Arial,sans-serif;background-image:initial;background-repeat:initial"> </span><a href="http://instagram.com/cwtdigital" style="color:rgb(17,85,204)" target="_blank"><span style="font-family:Arial,sans-serif;text-decoration:none;background-image:initial;background-repeat:initial"><img border="0" width="20" height="20" src="cid:image006.png@01CFC827.D51F6D00" alt="https://ci4.googleusercontent.com/proxy/X2ddbFYXE1hmnS4LGzoCr-Pl8fpK6yoO9R-jjF3__nbZkbuCmQR2LD3-Ts3deetyHm0L8E-hjyXeRS0YTkcqtNNGkxM=s0-d-e1-ft#http://www.cwtdigital.com/hosted/instagram.png"></span></a><br></p></div></div></div></div>
</div></div>