<div dir="ltr">Hi Yugo<div><br></div><div>I think the subject mentions the wrong pgpool-II version number<br><div><div class="gmail_extra"><br></div><div class="gmail_extra">Thanks</div><div class="gmail_extra">Best regards</div><div class="gmail_extra">Muhammad Usama</div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Jan 29, 2016 at 2:56 PM, Yugo Nagata <span dir="ltr"><<a href="mailto:nagata@sraoss.co.jp" target="_blank">nagata@sraoss.co.jp</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Pgpool Global Development Group is pleased to announce the availability of<br>
pgpool-II 3.5.0 and pgpoolAdmin 3.5.0.<br>
<br>
You can download the source codes and RPMs from:<br>
<a href="http://pgpool.net/mediawiki/index.php/Downloads" rel="noreferrer" target="_blank">http://pgpool.net/mediawiki/index.php/Downloads</a><br>
<br>
V3.5 new features are:<br>
<br>
- Improved performance in extended query protocol<br>
- Overcoming the thundering herd problem<br>
- Watchdog feature enhancements to be more robust and adaptable<br>
- PCP command enhancements<br>
- Import PostgreSQL 9.5 parser<br>
- etc.<br>
<br>
Please see the below for details:<br>
<a href="http://pgpool.net/mediawiki/index.php?title=pgpool-II_3.5_features&redirect=no" rel="noreferrer" target="_blank">http://pgpool.net/mediawiki/index.php?title=pgpool-II_3.5_features&redirect=no</a><br>
<br>
Release notes are here:<br>
<br>
=========================================================================<br>
3.5.0 (ekieboshi) 2016/01/29<br>
<br>
* Version 3.5.0<br>
<br>
This is the first version of pgpool-II 3.5 series.<br>
That is, a "major version up" from 3.4 series.<br>
<br>
__________________________________________________________________<br>
<br>
* Incompatible changes<br>
<br>
- Remove parallel query mode codes (Yugo Nagata)<br>
<br>
There are very few users and the maintenance efforts are not worth it.<br>
Codes for system db are also removed, since this was used in only parallel<br>
query mode and old query cache on disk.<br>
<br>
- Rename parameter name from ifconfig_path to if_cmd_path (Yugo Nagata)<br>
<br>
ifconfig_path isn't appropriate for the parameter name because<br>
ifconfig command is obsolete and now ip command is used as default.<br>
<br>
- Change the syntax of pcp commands arguments (Muhammad Usama)<br>
<br>
- Change the output and the argument semantics of pcp_watchdog_info command<br>
(Muhammad Usama)<br>
<br>
The command is enhanced to show watchdog information about all pgpool-II<br>
nodes in the cluster. According to this, the argument to specify node index<br>
is changed so that 0 means local pgpool-II node instead of the first remote<br>
node.<br>
<br>
__________________________________________________________________<br>
<br>
* New features<br>
<br>
- Add new parameter "serialize_accept" (Tatsuo Ishii)<br>
<br>
This parameter defines whether to serialize accept() call for incoming<br>
client connections. Default is off, which means no serializing (same<br>
behavior as pgpool-II 3.4 or before). If this is off, the kernel<br>
wakes up all of pgpool-II child process to execute accept() and one of<br>
them actually accepts the incoming connection. Problem here is,<br>
because so my child process wake up at a same time, heavy context<br>
switching occurred and the performance is affected. This phenomena is<br>
a classic problem called "the thundering herd problem". By enabling<br>
serialize_accept, only one of pgpool-II child process is woken up and<br>
executes accept() and the problem can be avoided.<br>
<br>
- Import parser of PostgreSQL 9.5 (Yugo Nagata)<br>
<br>
pgpool-II can understand new syntax introduced in PostgreSQL 9.5 now.<br>
Especially GROUPING SET, CUBE, ROLLUP and TABLESAMPLE can be be load<br>
balanced and can be used in query cache (except for TABLESAMPLE).<br>
<br>
Also INSERT...ON CONFLICT and UPDATE tab SET (col1, col2, ...) =<br>
(SELECT ...) ... can now be properly handled in query rewriting in<br>
native replication mode.<br>
<br>
- Allow to specify database for health check and streaming replication<br>
delay check. (Tatsuo Ishii)<br>
<br>
For this purpose new directive "health_check_database" and<br>
"sr_check_database" are added.<br>
<br>
__________________________________________________________________<br>
<br>
* Enhancements<br>
<br>
- Performance improvement in extened query protocol (Tatsuo Ishii)<br>
<br>
Unnecessary "flush" messages which used to be sent in each step of<br>
extended protocol messages (parse, bind, describe and execute) and<br>
brought significant communication overhead are removed. For now it<br>
only affects to streaming replication mode, that is, the performance<br>
of other modes remains same.<br>
<br>
- watchdog feature enhancements (Muhammad Usama, Yugo Nagata)<br>
<br>
The goal of this enhancement is to address the shortcomings and problems<br>
in the pgpool watchdog and make the watchdog system more robust and<br>
adaptable. Patch created by Usama, and reviewed, tested, and debugged<br>
by Yugo.<br>
<br>
-- The watchdog should consider the quorum and only elect the master/leader<br>
node if the quorum exist in the watchdog cluster.<br>
-- All the participating pgpool-II nodes in the watchdog cluster should have<br>
similar pgpool-II configurations.<br>
-- Watchdog nodes should have configurable watchdog node priority, to give<br>
users more control on which node should become a leader node.<br>
-- More options for the node health-checking, especially watchdog should allow<br>
external/3rd party node health checking system to integrate with it.<br>
-- The watchdog should keep looking for problems like split-brain syndrome and<br>
should automatically recover from it.<br>
-- Allow user to provide scripts to be executed at time of escalation and<br>
de-escalation to master/leader nodes.<br>
<br>
Add these new parameters:<br>
<br>
-- wd_ipc_socket_dir:<br>
This parameter is used to specify the directory where the UNIX domain<br>
socket accepting pgpool-II watchdog IPC connections will be created.<br>
<br>
-- wd_priority:<br>
This new parameter can be used to elevate the current watchdog node<br>
priority in leader elections. The node with the higher wd_priority<br>
value will get selected as master/coordinator watchdog node when<br>
cluster will be electing its master node at cluster startup or in the<br>
event of old master watchdog node failure.<br>
<br>
-- wd_de_escalation_command:<br>
This parameter holds the command that watchdog will execute on the<br>
master pgpool-II watchdog node when that node resigns from the master<br>
node responsibilities.<br>
<br>
-- wd_monitoring_interfaces_list:<br>
This parameter holds a comma separated list of network device names to<br>
be monitored by the watchdog process for the network link state.<br>
<br>
- Overhauling pcp commands (Muhammad Usama)<br>
<br>
-- Improved handling of command argument:<br>
Long command line options can be used now.<br>
-- PCP password is safe:<br>
Don't pass password via command line, which causes security risks.<br>
-- Mutiple concurrent execution of pcp commands:<br>
Allow to execute a pcp command while running pcp_recovery_node<br>
which takes sometime to finish.<br>
<br>
- SELECT count statics in "show pool_nodes" command results (Tatsuo Ishii)<br>
<br>
show pool_nodes results which briefly describes the status of each backend<br>
now shows how many SELECTs are issued to them. So you can quickly<br>
recognize the effect of load balancing for example;<br>
<br>
test=# show pool_nodes;<br>
node_id | hostname | port | status | lb_weight | role | select_cnt<br>
---------+----------+-------+--------+-----------+---------+------------<br>
0 | /tmp | 11002 | 2 | 0.500000 | primary | 338230<br>
1 | /tmp | 11003 | 2 | 0.500000 | standby | 163939<br>
(2 rows)<br>
<br>
- Parser enhancements for query rewriting (Yugo Nagata)<br>
<br>
INSERT/UPDATE/DELETE with WITH clause, writable CTE, RETURNING clause<br>
can now be properly handled in query rewriting in native replication mode.<br>
<br>
- Allow to open pool_passwd file in read only mode by pgpool-II main<br>
process (Tatsuo Ishii)<br>
<br>
- Allow to run pgpool in debug mode in regression tests (Tatsuo Ishii)<br>
<br>
- doc:Add pgpool_adm documents (Tatsuo Ishii)<br>
<br>
- doc:Update Chinese documents (Bambo Huang)<br>
<br>
__________________________________________________________________<br>
<br>
* Bug fixes (since 3.4.3)<br>
<br>
- doc: Fix misinformation regarding load balancing in docs (Tatsuo Ishii)<br>
<br>
In streaming replication mode, DECLARE, FETCH, CLOSE and SHOW are sent<br>
to primary node only. Pointed out in [pgpool-general-jp: 1378].<br>
<br>
- Issue fsync() when writing pgpool_status. (Tatsuo Ishii)<br>
<br>
This ensures that pgpool_status is saved to permanent storage and<br>
allow to survive after system crash.<br>
<br>
- Fix reset query stuck problem (Muhammad Usama)<br>
<br>
It is reported that reset query (DISCARD ALL etc.) occasionally does<br>
not finish and pgpool child remain waiting for reply from the backend<br>
thus client cannot connect to pgpool (for example bug report #107).<br>
The cause of problem is not identified yet but if client suddenly<br>
closes connection to pgpool in the middle of query processing, backend<br>
may not accept the reset queries because they are not ready for query.<br>
<br>
The fix is to keep track of the status of frontend socket state and not<br>
caching the PostgreSQL connection if the connection to frontend was not<br>
properly terminated.<br>
<br>
- test: Fix false alerm of regression tset 062 (Yugo Nagata)<br>
<br>
- test: Fix broken regression test for native replication (Yugo Nagata)<br>
<br>
Test for insert-lock didn't work.<br>
<br>
- test: Fix regress test to sleep after pgpool_reload (Yugo Nagata)<br>
<br>
Some regression tests modify configuration file and reloads. However,<br>
sometimes these were not applied to pgpool just after reloading.<br>
<br>
- Fix inconsistency of sequence values in replication mode (Yugo Nagata)<br>
<br>
When a schema name was provided, the table name was quoted wrongly as<br>
like "public.mytbl" instead of "public"."mytbl". So, pool_regclass<br>
and to_regclass couldn't find right talbe oid and insert lock was<br>
never executed. This caused inconsistency between DB nodes.<br>
<br>
- test: Fix complication error and segfault of timestamp rewrite test (Yugo Nagata)<br>
<br>
- doc: Fix untranslated sentence in Japanese document (Yugo Nagata)<br>
<br>
- Fix ancient bug of pool_push() and friends. (Tatsuo Ishii)<br>
<br>
It allocates buffer using realloc and uses the pointer<br>
returned. However it does the pointer calculation *before* realloc<br>
gets called. So the calculation uses the old pointer value, which<br>
causes various problems including segfault later.<br>
<br>
Also there were other problems with it. The buffer pointer and buffer<br>
size variable is not initialized. The buffer is not freed by<br>
pool_close. Typo in debugging message (3.4 or later only). They are<br>
fixed as well.<br>
<br>
- doc: Fix wrong description in documents about log_standby_delay (Yugo Nagata)<br>
<br>
- Fix segfalut that occurs when function is used in FROM clause (Yugo Nagata)<br>
<br>
PRPARE statements in streaming-reaplication mode and INSERT/UPDATE<br>
with SELECT in native-replication mode were affected. For example:<br>
<br>
- prepare p as select * from generate_series(1,1);<br>
- insert into tbl select now(), * from generate_series(1,1);<br>
<br>
- doc: Add caution about JDBC driver version regarding app_name_redirect_preference_list<br>
(Tatsuo Ishii)<br>
<br>
- Fix hang problmen reported in bug #145. (Tatsuo Ishii)<br>
<br>
The problem occurs when all the condition below are met:<br>
<br>
1) pgpool-II 3.4 or later<br>
2) streaming replication mode<br>
3) primary node is also load balance node<br>
4) extended protocol is used<br>
5) client_idle_limit reached<br>
<br>
- Fix a in-memory query cache bug (Tatsuo Ishii)<br>
<br>
If extended query protocol is used and a bind/execute message arrives<br>
which uses a statement created by prior parse message, the temp_cache<br>
is not initialized by a parse messages. Thus messages are added to pre<br>
existing temp cache buffer which causes the trouble, that is, when the<br>
cached result returns, Data Row message and Command Complete message<br>
appeared twice.<br>
<br>
Per by bug #152.<br>
<br>
- test: Fix regression test 065. (Tatsuo Ishii)<br>
<br>
The path to JDBC driver was explicitly defined. This is wrong. This<br>
should be inherited from JDBC_DRIVER environment variable.<br>
<br>
- test: Fix possible hang-up of regression test 054.postgres_fdw (Yugo Nagata)<br>
<br>
- test: Add option to regress.sh and pgpool_setup for unix domain socket<br>
directory (Yugo Nagata)<br>
<br>
- doc: Add missing descriptions about default values to documents<br>
(Yugo Nagata)<br>
<br>
- test: Fix regression test 055 for rhel7 rpm (Yugo Nagata)<br>
<br>
- Fix reset query stuck problem (Muhammad Usama)<br>
<br>
The issue is already fixed in older branches and this fix adopts the<br>
same solution used by 3.3 series, i.e. closing the backend connection<br>
when client idle limit is reached.<br>
<br>
- Fix bug with "SET TRANSACTION READ ONLY" (Tatsuo Ishii)<br>
<br>
Pgpool-II remembers that non read only queries (including SET) were<br>
executed in an explicit transaction and adds a "writing transaction"<br>
mark to the transaction. The mark affects the query routing behavior<br>
of pgpool-II while running in streaming replication mode. Pgpool-II<br>
starts sending queries to the primary after the mark is set. Because<br>
the effect of writing queries may appear on standbys after some delay<br>
in streaming replication mode, it is safer to route read queries to<br>
the primary after the mark is set.<br>
<br>
However there's oversight here. "SET TRANSACTION READ ONLY" does no<br>
data modification and should be treated as an exception.<br>
<br>
Per bug #157.<br>
<br>
- test: Fix to use timeout command to handle time out of regress test 062<br>
(Yugo Nagata)<br>
<br>
- Fix to show wrong error (Tatsuo Ishii)<br>
<br>
connect_with_timeout() does not show proper error info when<br>
getsockopt(SO_ERROR) reports an error. Pointed out in bug #159.<br>
<br>
- test: Add regress.sh missing \n in help messages (Yugo Nagata)<br>
<br>
- Fix the logic issue in get_backends_status() function (Muhammad Usama)<br>
<br>
get_backends_status () function counts the number of current valid and down<br>
backend nodes. The function assumed that the node was also invalid when its<br>
connection status was down. However, that is not always right.<br>
<br>
- Fix white/black_memqcache_table_list not to require quotaion (Yugo Nagata)<br>
<br>
The tables and schemas name specified in *_memqcache_table_list<br>
were needed to be quoted by double quotation.<br>
<br>
Patch contributed by Dang Minh Huong.<br>
Per [pgpool-hackers: 1323]<br>
<br>
- Fix FATAL error with reloading (Tatsuo Ishii)<br>
<br>
While reloading pgpool.conf, the number of DB nodes is tentatively set<br>
to 0, then counted up until reaching to the actual number of backends<br>
by the pgpool main process. Unfortunately the variable was on the<br>
shared memory and it confused pgpool child process when they were using<br>
the variable and this caused FATAL error.<br>
<br>
Per bug #156 report by harukat.<br>
<br>
- test: Add some tests for white/black_memqcache_table_list to regression<br>
test 006.memqcache (Yugo Nagata)<br>
<br>
- Fix reset query stuck problem (Muhammad Usama, Tatsuo Ishii)<br>
<br>
When pool_read fails to read from frontend or pool_flush fails to<br>
write to the frontend, report FRONTEND_ERROR, rather than ERROR to<br>
disconnect and terminate pgpool child process, to prevent the query<br>
stuck problem.<br>
<br>
- test: Fix some regression tests that failed in debug mode (Yugo Nagata)<br>
<br>
- Fix performance degradation while using IPv6 (Muhammad Usama)<br>
<br>
Per bug #165.<br>
<span><font color="#888888"><br>
--<br>
Yugo Nagata <<a href="mailto:nagata@sraoss.co.jp" target="_blank">nagata@sraoss.co.jp</a>><br>
_______________________________________________<br>
pgpool-general mailing list<br>
<a href="mailto:pgpool-general@pgpool.net" target="_blank">pgpool-general@pgpool.net</a><br>
<a href="http://www.pgpool.net/mailman/listinfo/pgpool-general" rel="noreferrer" target="_blank">http://www.pgpool.net/mailman/listinfo/pgpool-general</a><br>
</font></span></blockquote></div><br></div></div></div></div>