[pgpool-hackers: 2212] Re: [pgpool-committers: 3907] pgpool: Mega patch to fix "kind mismatch" (or derived) errors in stream

[Tatsuo Ishii]

I created this patch in the hope to fix variety of kind mismatch
and/or hang problems in streaming replication mode. According to
Sergey Kim (thanks for the effort to test it out!), the fix gave us a
positive result. So I decide to merge the patch into 3.6 and 3.5
stable trees as well (3.4 or prior releases does not have that kind of
problem).

One thing I forgot to mention in the git log was, "parse before bind"
problem has been fixed in the patch as well. 

What is "parse before bind" problem anyway? Details were reported in:

http://www.pgpool.net/mantisbt/view.php?id=271

Pgpool-II load balances read queries even In an explicit
transaction. However once a write query is sent, subsequent read
queries should be sent to primary server, otherwise those read queries
could see outdated results due to replication lag. Moreover, if a read
query was parsed before the write query, then a bind request is sent,
Pgpool-II should re-parse the query on the primary node before sending
the bind request. This has been done by a function called
parse_before_bind(). Since 3.5, I believe this hasn't been working
well. I have addressed the problem in the patch too.

Best regards,
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese:http://www.sraoss.co.jp

> mega patch to fix "kind mismatch" (or derived) errors in streaming replication mode.
> 
> The errors are caused by wrong prediction in which (or both) database
> node will send response to Pgpool-II. Previous implementation using
> "sync map" are weak and sometimes fail in the prediction.
> 
> This patch introduces new implementation using "pending message
> queue", which records all sent message to backends. The element of the
> queue stores info regarding messages types
> (parse/bind/execute/describe/close/sync), to which database node the
> message was sent and so on. It's a simple FIFO queue. When a message
> arrives from backend, by looking at the head of the "pending message
> queue", it is possible to reliably predict what kind of message and
> from which database node it will arrive. After receiving the message,
> the element is removed from the queue.
> 
> I would like to thank to Sergey Kim, who has been helping me in
> testing series of patches.
> 
> See bug 271:
> http://www.pgpool.net/mantisbt/view.php?id=271
> 
> and discussion in pgpool-hackers mailing list [pgpool-hackers: 2043]
> and [pgpool-hackers: 2140] for more details.
> 
> Branch
> ------
> master
> 
> Details
> -------
> http://git.postgresql.org/gitweb?p=pgpool2.git;a=commitdiff;h=adcb63696b55463d6f6aa493b1b510e135ea67d7
> 
> Modified Files
> --------------
> src/auth/pool_auth.c                       |  13 +-
> src/context/pool_query_context.c           |  80 ++++-
> src/context/pool_session_context.c         | 545 +++++++++++++++++++---------
> src/include/context/pool_query_context.h   |   3 +-
> src/include/context/pool_session_context.h | 107 +++---
> src/include/pool.h                         |   2 +
> src/include/protocol/pool_proto_modules.h  |   9 +-
> src/protocol/CommandComplete.c             | 130 ++++---
> src/protocol/pool_process_query.c          | 556 +++++++++++++++--------------
> src/protocol/pool_proto_modules.c          | 424 +++++++++++++---------
> src/query_cache/pool_memqcache.c           |  12 +-
> 11 files changed, 1124 insertions(+), 757 deletions(-)
>