<div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, May 8, 2013 at 2:32 PM, Tatsuo Ishii <span dir="ltr"><<a href="mailto:ishii@sraoss.co.jp" target="_blank">ishii@sraoss.co.jp</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Ahsan,<br>
<div class="im"><br>
> Hi Tatsuo,<br>
><br>
> Good to see that you found the problem. We have a customer who has been waiting for a fix for this issue for sometime.<br>
><br>
> Which branches are you planning to commit this to and when?<br>
<br>
</div>The fix has been already committed to master, 3.2-stable and 3.1-stable.<br></blockquote><div><br></div><div style>Thanks Tatsuo. </div><div style><br></div><div style>Were you able to reproduce the issue? As i understand it wasn't straight forward to reproduce.</div>
<div style><br></div><div style> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="HOEnZb"><div class="h5">--<br>
Tatsuo Ishii<br>
SRA OSS, Inc. Japan<br>
English: <a href="http://www.sraoss.co.jp/index_en.php" target="_blank">http://www.sraoss.co.jp/index_en.php</a><br>
Japanese: <a href="http://www.sraoss.co.jp" target="_blank">http://www.sraoss.co.jp</a><br>
<br>
> -- Ahsan<br>
><br>
> -----Original Message-----<br>
> From: Tatsuo Ishii <<a href="mailto:ishii@postgresql.org">ishii@postgresql.org</a>><br>
> Sender: <a href="mailto:pgpool-hackers-bounces@pgpool.net">pgpool-hackers-bounces@pgpool.net</a><br>
> Date: Wed, 08 May 2013 18:02:03<br>
> To: <<a href="mailto:anaeem.it@gmail.com">anaeem.it@gmail.com</a>><br>
> Cc: <<a href="mailto:pgpool-hackers@pgpool.net">pgpool-hackers@pgpool.net</a>><br>
> Subject: [pgpool-hackers: 231] Re: Unexpected behavior of pgpool-II<br>
><br>
> Sorry for delay. I think I found a problem regarding %H for<br>
> follow_master_command. Pgpool-II assigns it "new master node", which<br>
> is defined as "the first live node". On the other hand %H value<br>
> expected in follow_master_command is "new primary node", which is not<br>
> neccesary same as "new master node".<br>
><br>
> Attached patch should fix the problem(against 3.2-stable).<br>
> --<br>
> Tatsuo Ishii<br>
> SRA OSS, Inc. Japan<br>
> English: <a href="http://www.sraoss.co.jp/index_en.php" target="_blank">http://www.sraoss.co.jp/index_en.php</a><br>
> Japanese: <a href="http://www.sraoss.co.jp" target="_blank">http://www.sraoss.co.jp</a><br>
><br>
>> Hi,<br>
>><br>
>> There is unexpected behavior in pgpool2 that seems to be a bug. Please find<br>
>> the details as following i.e.<br>
>><br>
>> Phenomenon<br>
>><br>
>>> pgpool2 configuration option "follow_master_command" special character %H<br>
>>> (Hostname of the new master node) is not working as expected<br>
>>> In master slave mode after failover, 2nd node become new master but got<br>
>>> unexpected hostname (via %H). Please find attached pgpool.conf for details.<br>
>><br>
>><br>
>> Versions<br>
>><br>
>>> OS version: RHEL - 64<br>
>>> pgpool version: pgpool-II version 3.2.1 (namameboshi)<br>
>><br>
>><br>
>> *Details/Observations* :-<br>
>><br>
>>> At Start<br>
>><br>
>>> node_id | hostname | port | status | lb_weight | role<br>
>>> ---------+----------+------+--------+-----------+---------<br>
>>> 0 | dmp02 | 5444 | 2 | 0.333333 | primary<br>
>>> 1 | dmp03 | 5444 | 2 | 0.333333 | standby<br>
>>> 2 | dmp04 | 5444 | 2 | 0.333333 | standby<br>
>>><br>
>><br>
>><br>
>>> Failover arguments when the user stopped DMP02.<br>
>><br>
>>> 2013îN 2åé 8ì˙ ã‡ójì˙ 12:44:12 JST<br>
>><br>
>> FAILED_NODE_ID =0<br>
>>> FAILED_NODE_NAME=dmp02<br>
>>> FAILED_NODE_PORT=5444<br>
>>> FAILED_NODE_DATA=/opt/PostgresPlus/9.2AS/data<br>
>>> NEW_MASTER_ID =1<br>
>>> OLD_MASTER_ID =0<br>
>>> NEW_MASTER_NAME =dmp03<br>
>>> OLD_PRIMARY_ID =0<br>
>>> NEW_MASTER_DATA =/opt/PostgresPlus/9.2AS/data<br>
>>> 0<br>
>>><br>
>><br>
>><br>
>>> Then, these arguments told user that the master node has been changed<br>
>><br>
>>> 2013îN 2åé 8ì˙ ã‡ójì˙ 12:44:13 JST<br>
>>> FAILED_NODE_ID =0<br>
>>> FAILED_NODE_NAME=dmp02<br>
>>> FAILED_NODE_PORT=5444<br>
>>> FAILED_NODE_DATA=/opt/PostgresPlus/9.2AS/data<br>
>>> NEW_MASTER_ID =1<br>
>>> OLD_MASTER_ID =0<br>
>>> NEW_MASTER_NAME =dmp03<br>
>>> OLD_PRIMARY_ID =0<br>
>>> NEW_MASTER_DATA =/opt/PostgresPlus/9.2AS/data<br>
>>> 2013îN 2åé 8ì˙ ã‡ójì˙ 12:44:17 JST<br>
>>> FAILED_NODE_ID =2<br>
>>> FAILED_NODE_NAME=dmp04<br>
>>> FAILED_NODE_PORT=5444<br>
>>> FAILED_NODE_DATA=/opt/PostgresPlus/9.2AS/data<br>
>>> NEW_MASTER_ID =1<br>
>>> OLD_MASTER_ID =0<br>
>>> NEW_MASTER_NAME =dmp02 *<--- *** The master name is unexpected ****<br>
>>> OLD_PRIMARY_ID =0<br>
>>> NEW_MASTER_DATA =/opt/PostgresPlus/9.2AS/data<br>
>>><br>
>><br>
>><br>
>>> node_id | hostname | port | status | lb_weight | role<br>
>>> ---------+----------+------+--------+-----------+---------<br>
>>> 0 | dmp02 | 5444 | 2 | 0.333333 | standby<br>
>>> 1 | dmp03 | 5444 | 2 | 0.333333 | primary<br>
>>> 2 | dmp04 | 5444 | 2 | 0.333333 | standby<br>
>>><br>
>><br>
>><br>
>>> pid | usesysid | usename | application_name | client_addr |<br>
>>> client_hostname | client_port | backend_start | state<br>
>>> | sent_location | write_location | flush_location | replay_location |<br>
>>> sync_priority | sync_state<br>
>>><br>
>>> -------+----------+--------------+------------------+---------------+-----------------+-------------+----------------------------------+-----------+---------------+----------------+----------------+-----------------+---------------+------------<br>
>>> 20641 | 10 | enterprisedb | dmp02 | 172.24.217.23 |<br>
>>> | 16948 | 08-FEB-13 17:39:44.078173 +09:00 | streaming |<br>
>>> 14/C50000E0 | 14/C50000E0 | 14/C50000E0 | 14/C50000E0 |<br>
>>> 0 | async<br>
>>> (1 çs)<br>
>>><br>
>><br>
>> There is one pg_stat_replication record because dmp04 is connecting to dmp02<br>
>><br>
>> This phenomenon is reproduced by user. What we need to do is just make the<br>
>> scripts like follow_master.sh and launch pgpool process with -F [path to<br>
>> pcp.conf].<br>
>><br>
>> 1. Before failover<br>
>><br>
>>> -bash-3.2$ psql -h 10.1.1.187 -p 9999 -c "show pool_nodes"<br>
>>> node_id | hostname | port | status | lb_weight | role<br>
>>> ---------+----------+------+--------+-----------+---------<br>
>>> 0 | slave1 | 5444 | 2 | 0.333333 | primary<br>
>>> 1 | slave2 | 5444 | 2 | 0.333333 | standby<br>
>>> 2 | slave3 | 5444 | 2 | 0.333333 | standby<br>
>>> (3 行)<br>
>><br>
>><br>
>> 2. We stopped slave 1 node with "pg_ctl -D /data -m immediate stop".<br>
>><br>
>> 3. After failover<br>
>><br>
>>> -bash-3.2$ psql -h 10.1.1.187 -p 9999 -c "show pool_nodes"<br>
>>> node_id | hostname | port | status | lb_weight | role<br>
>>> ---------+----------+------+--------+-----------+---------<br>
>>> 0 | slave1 | 5444 | 2 | 0.333333 | standby<br>
>>> 1 | slave2 | 5444 | 2 | 0.333333 | primary<br>
>>> 2 | slave3 | 5444 | 2 | 0.333333 | standby<br>
>>> (3 行)<br>
>><br>
>><br>
>> 4. Then user got output like below after failover that is not expected that<br>
>> is present at the end of follow_master.sh.log i.e.<br>
>><br>
>>> NEW_MASTER_NAME =slave1<br>
>><br>
>><br>
>> [root@master1 ~]# cat /tmp/failover.sh.log<br>
>><br>
>>> 2013年 3月 13日 水曜日 11:36:28 JST<br>
>>> FAILED_NODE_ID =0<br>
>>> FAILED_NODE_NAME=slave1<br>
>>> FAILED_NODE_PORT=5444<br>
>>> FAILED_NODE_DATA=/opt/PostgresPlus/9.2AS/data<br>
>>> NEW_MASTER_ID =1<br>
>>> OLD_MASTER_ID =0<br>
>>> NEW_MASTER_NAME =slave2<br>
>>> OLD_PRIMARY_ID =0<br>
>>> NEW_MASTER_DATA =/opt/PostgresPlus/9.2AS/data<br>
>>> 0<br>
>>><br>
>> [root@master1 ~]# cat /tmp/follow_master.sh.log<br>
>><br>
>>> 2013年 3月 13日 水曜日 11:36:30 JST<br>
>>> FAILED_NODE_ID =0<br>
>>> FAILED_NODE_NAME=slave1<br>
>>> FAILED_NODE_PORT=5444<br>
>>> FAILED_NODE_DATA=/opt/PostgresPlus/9.2AS/data<br>
>>> NEW_MASTER_ID =1<br>
>>> OLD_MASTER_ID =0<br>
>>> NEW_MASTER_NAME =slave2<br>
>>> OLD_PRIMARY_ID =0<br>
>>> NEW_MASTER_DATA =/opt/PostgresPlus/9.2AS/data<br>
>>> 2013年 3月 13日 水曜日 11:36:32 JST<br>
>>> FAILED_NODE_ID =2<br>
>>> FAILED_NODE_NAME=slave3<br>
>>> FAILED_NODE_PORT=5444<br>
>>> FAILED_NODE_DATA=/opt/PostgresPlus/9.2AS/data<br>
>>> NEW_MASTER_ID =1<br>
>>> OLD_MASTER_ID =0<br>
>>> NEW_MASTER_NAME =*slave1<br>
>>> *OLD_PRIMARY_ID =0<br>
>>> NEW_MASTER_DATA =/opt/PostgresPlus/9.2AS/data<br>
>><br>
>><br>
>> Please guide. Thanks.<br>
>><br>
>> Best Regards,<br>
>> Asif Naeem<br>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br>Ahsan Hadi<br>Snr Director Product Development<br>EnterpriseDB Corporation<br>The Enterprise Postgres Company<br><br>Phone: +92-51-8358874 <br>Mobile: +92-333-5162114<br>
<br>Website: <a href="http://www.enterprisedb.com">www.enterprisedb.com</a><br>EnterpriseDB Blog: <a href="http://blogs.enterprisedb.com/">http://blogs.enterprisedb.com/</a><br>Follow us on Twitter: <a href="http://www.twitter.com/enterprisedb">http://www.twitter.com/enterprisedb</a><br>
<br>This e-mail message (and any attachment) is intended for the use of the individual or entity to whom it is addressed. This message contains information from EnterpriseDB Corporation that may be privileged, confidential, or exempt from disclosure under applicable law. If you are not the intended recipient or authorized to receive this for the intended recipient, any use, dissemination, distribution, retention, archiving, or copying of this communication is strictly prohibited. If you have received this e-mail in error, please notify the sender immediately by reply e-mail and delete this message.
</div></div>