[pgpool-hackers: 662] Re: Review: pgpool-II 3.5 road map

[Tatsuo Ishii]

> Thanks Tatsuo for your feedback. Please see my response in-line..
> 
> On Wed, Nov 12, 2014 at 10:22 AM, Tatsuo Ishii <ishii at postgresql.org> wrote:
> 
>> Ahasan,
>>
>> I have reviewed you proposal of 9.5 road map (#8, #9 #10 are my
>>
> 
> Do you mean 3.5 road map?

Oops. You are right.

>> additions).  It seems #4, #8, #10 are feasible. I think others need to
>> be studied or elaborated more.
>>
> 
> Are you proposing 4,8,10 for pgpool 3.5? I don't think this would be enough
> from a feature perspective for a major pgpool release, we need to include a
> feature or two form the performance or HA type of features.
> 
> You are right we need RND on some of the important items however i feel
> that we should start that RND work in the current team if no one else picks
> it up.

I think we don't need to decide all the features for 3.5 right
now. Item #4, #8, #9 are just looking feasible and useful at this
moment. (Yugo will work on #10, which is easy and it's not a feature
of pgpool-II itself anyway). We can add feature later.

>> 1)  performance issue with prepared protocol.
>>     - testing pgpool for query latency and find out where we can make
>> improvement.
>>
>>     This is already on the TODO list for years now but nobody comes up
>>     with an idea to implement it. I think we will wait for someone
>>     thinks of nice idea.
>>
> 
> Yeah this is very important, we should really get someone to look at this
> for 3.5.  The testing of query latency will also give ideas for other
> performance improvements.
> 
> 
>>
>> 2) Next steps for HA...Is there a way to provide automatic and
>>    seamless failover for active sessions.
>>
>>    I have no idea how to implement it. Suppose failover happens while
>>    client is receiving SELECT result data from pgpool-II. We need to
>>    find a way to remember what data has been already sent and not to
>>    switch to different backend and to continue the session.
>>
>>
> I believe Usama is planning to do some RND on watchdog soon. He will study
> the techniques and architecture used by cloud and other systems for HA and
> propose some enhancements that we can implement for watchdog in pgpool II.
> 
> 
>> 3) multi master communication. two independent pairs of pgpool needed
>>    to communicate.
>>
>>    Can you please elaborate what needed for pgpool-II? Sounds vague to me.
>>
> 
> Yeah i agree the requirements is a bit vague. Basically we had a customer
> who was running two pairs of pgpool in two different countries. He wanted
> to have some sort of multi master communication between the pairs so if a
> node goes down in one pair, a message is sent to other pair. The clients
> connecting to one pair
> 
> Usama, Can you give some further clarifications on this since you also
> looked into the customer query...
> 
> Here is the snippet form the user's email :
> 
> As you know, we have a pair of PG-Pool boxes in both of our 2 sites.
> 
> Each pair uses watchdog to communicate and to manage the VIP. Each
> pair is configured
> to
> use all 4 of the back-end DBs ­ 2 in each site. I have set LB weighting so
> that
> the queries are sent to the local DBs in the vast majority of cases.
> 
> What I think we need to do is make the 2 pairs of PG-Pool communicate so
> that
> if a node is detached at one side, this info is passed through to the other
> PG-Pool pair. I have tried to setup watchdog between all 4 of the PG-Pool
> boxes
> and although I can see that the additional WD packets are being sent
> they don¹t seem to be processed at the far end.

Well, pgpool-II is not designed for WAN. Watchdog uses UDP, not TCP
for example. Also mutimaster requires transaction conflict resolution,
which I think pgpool-II should not be responsible for, rather logical
replication or bi-directional replication of PostgreSQL should be in
charge of.

>> 4) parser integration with latest pg version
>>
>>    Yes, we should integrate PostgreSQL 9.5's parser for pgpool-II 3.5.
>>
>>
>> 5) Ability to re-issue queries when it failover to the stand-by node.
>>    Currently the user has re-issue queries
>>
>>    Looks identical to #2.
>>
> 
> Hmm...I believe 2 is about doing a seamless failover for active sessions...

But if client has already received part of data from standby, then
pgpool-II should send rest of the data, not whole data
again. Otherwise the client will be confused by doubled data. Also if
#2 is implemented, I don't see any benefit in implementing #5.

> This one came from Bruce...
> 
> Bruce, do you think 5 and 2 are identical?
> 
> 
>> 6) Would like to see pgpool in separate sub modules that can be
>>    installed separately for each feature, along with a common-libs
>>    sub-package or so.
>>
>>    I have no idea what this actually means. We will wait until someone
>>    comes up with more concrete idea.
>>
>> 7) Adding transaction based pooling to pgpool
>>
>>    Probably we should try pgbouncer + pgpool-II combo first to see if
>>    it's actually useful.
>>
>> 8) Enhance pcp commands
>>
>>    http://pgpool.net/mediawiki/index.php/TODO#Enhance_pcp_commands
>>
>> 9) create separate process for health checking
>>
>>
>> http://pgpool.net/mediawiki/index.php/TODO#Create_separate_process_for_health_checking
>>
>> 10) Create yum repository
>>
>> Best regards,
>> --
>> Tarts Ishii
>> SRA OSS, Inc. Japan
>> English: http://www.sraoss.co.jp/index_en.php
>> Japanese:http://www.sraoss.co.jp
>> _______________________________________________
>> pgpool-hackers mailing list
>> pgpool-hackers at pgpool.net
>> http://www.pgpool.net/mailman/listinfo/pgpool-hackers
>>
> 
> 
> 
> -- 
> Ahsan Hadi
> Snr Director Product Development
> EnterpriseDB Corporation
> The Enterprise Postgres Company
> 
> Phone: +92-51-8358874
> Mobile: +92-333-5162114
> 
> Website: www.enterprisedb.com
> EnterpriseDB Blog: http://blogs.enterprisedb.com/
> Follow us on Twitter: http://www.twitter.com/enterprisedb
> 
> This e-mail message (and any attachment) is intended for the use of the
> individual or entity to whom it is addressed. This message contains
> information from EnterpriseDB Corporation that may be privileged,
> confidential, or exempt from disclosure under applicable law. If you are
> not the intended recipient or authorized to receive this for the intended
> recipient, any use, dissemination, distribution, retention, archiving, or
> copying of this communication is strictly prohibited. If you have received
> this e-mail in error, please notify the sender immediately by reply e-mail
> and delete this message.