[pgpool-hackers: 328] pgpool-II 3.3.0 and pgpoolAdmin 3.3.0 officially released
Yugo Nagata
nagata at sraoss.co.jp
Tue Jul 30 18:39:56 JST 2013
Pgpool Global Development Group is pleased to announce the availability of
pgpool-II 3.3.0 and pgpoolAdmin 3.3.0.
You can download them from here:
(pgpool-II) http://www.pgpool.net/download.php?f=pgpool-II-3.3.0.tar.gz
(pgpoolAdmin) http://www.pgpool.net/download.php?f=pgpoolAdmin-3.3.0.tar.gz
V3.3's new features are:
Enhancements for watchdog
New monitoring method of watchdog lifecheck using heartbeat signal
Interlocking of failover/failback script
Secure watchdog communication
PCP command for retrieving the watchdog status
etc.
Othres
Import PostgreSQL 9.2 raw parser
New pgpool_setup tool
Support for using CREATE EXTENSION to install pgpool specific extensions
regression test suit
etc.
Manual:
http://www.pgpool.net/docs/pgpool-II-3.3.0/pgpool-en.html
Release Note:
http://www.pgpool.net/docs/pgpool-II-3.3.0/NEWS.txt
===============================================================================
pgpool-II 3.3.0
===============================================================================
This is the first version of pgpool-II 3.3 series.
That is, a "major version up" from 3.2 series.
__________________________________________________________________
* Incompatible changes
- All the follwing are about watchdog.
See "New features" section below for details of these changes.
- Default monitoring method was changed from query mode to heartbeat mode.
- Failover/failback commands are executed in only one pgpool-II.
- In default, all the query caches on shared memory are cleared when
standby pgpool-II escalates to active.
- Database name, user name, and password used for monitring other
pgpool-II by query are specified by dedicated parameters.
Previously, template1, recovery_user, and recovery_password are used.
__________________________________________________________________
* New features
** Watchdog
- Add a new monitring method using heartbeat signal of UDP packet in
lifecheck. (Yugo Nagata)
You can select a method from either of two methods, "heartbeat" mode or
"query" mode.
The heartbeat mode is the new method. In this mode, watchdog monitors
other pgpool-II process by using heartbeat signal. Watchdog receives
heartbeat signals sent by other pgpool-II periodically. If there are no
signal for a certain period, watchdog regards it as failure of the
pgpool-II. For redundancy you can use multiple networks for heartbeat
exchange between pgpool-IIs. This mode is default and recommended one.
The query mode is the conventional method. In this mode, watchdog monitors
pgpool-II's service rather than process. Watchdog sends queries to other
pgpool-II and checks the response. Note that this method requires connections
from other pgpool-IIs, so it would fail motoring if num_init_children isn't
large enough. This mode is deprecated and left for backward compatibility.
Add these new parameters:
- wd_lifecheck_method
- wd_heartbeat_port
- wd_heartbeat_keepalive
- wd_heartbeat_deadtime
- heartbeat_destinationX
- heartbeat_destination_portX
- heartbeat_deviceX
- Add interlocking mechanism of exclusive failover/failback command
execution. (Yugo Nagata)
When using multiple pgpool-IIs with watchdog enabled, failover commands
(failover_command, failback_command, and follow_master_command) get
executed only at one pgpool-II.
Previously, these command got executed at all pgpool-IIs.
- Add authentication mechanism for watchdog packet communication.
(Yugo Nagata)
Watchdog packets (include heartbeat signal) from pgpool-II with wrong
authentication key are rejected. All pgpool-IIs must have the same key,
which is specified wd_authkey parameter in pgpool.conf. pgpool-II with
wrong authkey can't even start watchdog, because the startup packet is
rejected by other pgpool-IIs.
- Add clear_memqcache_on_escalation parameter. (Yugo Nagata)
If this is on, all the query caches on shared memory are cleared when
standby pgpool-II escalates to active.
This is aimed to prevent the new active pgpool-II from using inconsistent
query caches with the previous active.
- Add wd_escalation_command parameter. (Yugo Nagata)
This specifies command which is executed at escalation on the new active
pgpool-II server. The timing is just before virtual IP is brought up.
- Add parameters wd_lifecheck_dbname, wd_lifecheck_user, and
wd_lifecheck_password. (Yugo Nagata)
These parameters specify the database name, the user name, and password
used in query mode lifecheck of watchdog . Previously, these are hard
coded to use template1, recovery_user, and recovery_password.
- When delegate_IP parameter is emply, viertual IP is neither brought up
nor switched. (Yugo Nagata)
This allows multi-master like configuration without virtual IP.
- Add pcp_watchdog_info command (Yugo Nagata)
This is PCP command for retrieving the watchdog status.
** Others
- Import PostgreSQL 9.2 raw parser. (Nozomi Anzai, Tatsuo Ishii)
- Add a tool called pgpool_setup to set up pgpool-II and PostgreSQL
temporary installation in current directory for *testing* purpose.
(Tatsuo Ishii)
usage: pgpool_setup [-m r|s][-n num_clusters][--no-stop]
-m s: create an installation as streaming replication mode.
(the default)
-m r: create an installation as native replication mode.
-n num_clusters: create num_clusters PostgreSQL database cluster nodes
-p base_port: specify base port. The first PostgreSQL node's port is
base_port. Second PostgreSQL node's port is base_port+1
... nth PostgreSQL node's port is base_port+n-1, pgpool
port is base_port+n, pcp port is base_port+n+1.
--no-stop: do not stop pgpool and PostgreSQL after the work
- Support installation method using CREATE EXTENSION for pgpool-recovery
and pgpool-regclass. (Tatsuo Ishii)
Older installtion method is still preserved.
Note: extension names are "pgpool_recovery" and "pgpool_regclass", not
"pgpool-recovery" and "pgpool-regclass" because latters are not
convenient in using CREATE EXTENSION command (requires double quotes).
- Add a function "pgpool_pgctl()" which enebles to execute
pg_ctl stop/restart/reload (except for start) by SQL. (Nozomi Anzai)
$ psql sales -c "select pgpool_pgctl('reload', 'fast')";
pgpool_pgctl
--------------
t
(1 row)
This function always ignores the actual result and returns 't', so the
user can't know if pg_ctl succeeded or failed. To use this we have to set
the custom variable for security which limits the users to execute pg_ctl
who has the permission of data directory.
- Add shell scripts to install pgpool-II and pgpoolAdmin by RPM.
(Nozomi Anzai, Yugo Nagata)
To make the installer package execute getsources.sh, and the directory
named "work" will be created. And you rpmbuild each spec files in work/,
put RPMs into work/installer and make tar ball of work/installer.
The installer does not only install RPMs but also edit postgresql.conf,
pgpool.conf, pg_hba.conf, recovery.conf and scripts for failover and
online recovery.
This assumes two-nodes configuration and the install script have to be
executed in both nodes.
- Add new parameter "search_primary_node_timeout".
(Muhammad Usama, Tatsuo Ishii)
The parameter specifies the maximum amount of time in seconds to
search for a primary node when a failover scenario occurs. Patch
contributed by Muhammad Usama. Japanese doc and slight editing of
English doc by Tatsuo Ishii.
- Chinese tutorials for memqcache and watchdog. (Bambo Huang)
- Add regression test suit. (Tatsuo Ishii)
__________________________________________________________________
* Bug fixes
- Consider timeout waiting for compeletion of failback request in on line
recovery. (Tatsuo Ishii)
This will prevent the situation that recovery operation continues forever
and we cannot even shutdown pgpool-II main process. This could happen
especially while executing follow master command.
- Fix a bug that %H of follow_master_command is not assigned correctly the
new primary node in stream replication mode.
(Tatsuo Ishii)
- Fix not to execute escalation when the pgpool-II which is already active
receives down notification from other pgpool-II. (Yugo Nagata)
- Fix wd_create_send_socket() not to execute select() before connect().
(Yugo Nagata)
How select() works on an unconnected socket is undefined, and differs
between platform. On Linux, this returns 2 and it is eventually harmless.
However, on Soraris, this returns 0 and it is indistinguishable from time
timeout, so watchdog wouldn't work correctly.
- Fix error when pgpool_regclass is not installed. (Tatsuo Ishii)
The query used in pool_has_pgpool_regclass() fails if pgpool_regclass
does not exist. The bug was introduced in 3.2.4. See [pgpool-general:
1722] for more details.
[pgpool-general: 1722] [PgPool-II 3.2.4] pgpool_regclass now mandatory?
http://www.sraoss.jp/pipermail/pgpool-general/2013-May/001749.html
- Fix do_query() not to hang when PostgreSQL returns an error.
(Tatsuo Ishii)
The typical symptom is "I see SELECT is keep on running according to
pg_stat_activity". To fix this pgpool-II just exits the process and
kill the existig connection. This is not gentle but at this point I
believe this is the best solution.
- Fix possible deadlock during failover with watchdog enabled.
(Yugo Nagata)
This is reported in Bug track #54 by arshu arora
http://www.pgpool.net/mantisbt/view.php?id=54
- Fix unnecessary degeneration caused by error on commit. (Tatsuo Ishii)
In master slave mode, if master gets an error at commit, while other
slaves are normal at commit, we don't need to degenrate any backend
because it is likely that the "kind mismatch error" was caused by a
deferred trigger.
- Fix bug with do_query which causes hung in extended protocol.
(Tatsuo Ishii)
This problem could occur when insert lock is enabled and
pgpool_catalog.insert_lock exists, See [pgpool-general: 1684] for more
details.
[pgpool-general: 1684] insert_lock hangs
http://www.sraoss.jp/pipermail/pgpool-general/2013-May/001711.html
- Fix possible failure of query cache invalidation for DML in transaction.
(Tatsuo Ishii)
CREATE TABLE t1(i INTEGER);
CREATE TABLE t2(i INTEGER);
SELECT * FROM t1;
BEGIN;
DELETE FROM t2 WHERE i = 0;
INSERT INTO t1(i) VALUES(1);
COMMIT;
SELECT * FROM t1;
At commit pgpool tries to delete cache for t2 but failes because
there's no oid table entry for t2. Problem is, it fails to check oid
table for t1. So cahce for t1 remains and the last SELECT incorrectly
returns cached data. Fix is, continuing to check oid table entries.
This is reported in Bug track #58 by wms
http://www.pgpool.net/mantisbt/view.php?id=58
- Fix to register pgpool_regclass in pg_catalog schema (Tatsuo Ishii)
This is necessary to deal with clients which restricts schema search path
to pg_catalog only. Postgres_fdw is such a client.
- Fix a hang of "pgpool -m f stop" (Tatsuo Ishii)
This is caused by unmanaged pgpool children remaining. This could happen
when multiple PostgreSQLs are going down or even when pgpool is started
before PostgreSQL startup.
- Fix a potential crash in pg_md5 command (Muhammad Usama)
- Fix a segmentation fault that occurs when on memory query cache enabled
and the query is issued in extended query mode and the result is too large
(Tatsuo Ishii)
This is reported in Bug track #63 by harukat.
Analysis and a test case are also provided.
#63 Child process was terminated by segmentation fault with memcached
http://www.pgpool.net/mantisbt/view.php?id=63
- Fix a segmentation fault of a child process that occurs when a startup
packet has no PostgreSQL user information (Yugo Nagata)
You can reproduce it by
$ psql -p 9999 -U ''
If enable_pool_hba is on, a child process terminates by segmentation
fault. Otherwise if enable_pool_hba is off, the error message is
ERROR: pool_discard_cp: cannot get connection pool for user (null)
database (null)
In both cases, psql terminates with no message on frontend.
In the fixed version, if PostgreSQL user is not specified in startup packet,
the message as following is output to both log and frontend. This is
the same behavior as PostgreSQL.
FATAL: no PostgreSQL user name specified in startup packet
- Fix memory allocation logic in extended query processing with on-memory
query cache enabled (Tatsuo Ishii)
When very long query string (> 1024 bytes) supplied in extended query
with bind parameters, it fails to allocate enough memory.
- Fix to verify the backend node number in pcp_recovery_node (Yugo Nagata)
When an invalid number is used, null value is passed as an arguments
of recovery script, and this causes a malfunction. In especially,
rsync may delete unrelated files in basebackup scripts.
- Fix a segmentation fault on main process that could occures after backend
error detection (Tatsuo Ishii)
This is reported in Bug track #62 by tuomas.
#62 Slave network outage causes a segmentation fault on main process
http://www.pgpool.net/mantisbt/view.php?id=62
- Fix bug with health check when used with child_life_time (Tatsuo Ishii)
Failover could happen even if the backend was running fine.
This problem is reported in [pgpool-general: 1892] by larisa sabban.
[pgpool-general: 1892] Pgpool is unable to connect backend PostgreSQL
http://www.sraoss.jp/pipermail/pgpool-general/2013-July/001920.html
- Fix bug in parsing prepared statements with transaction handling in
replication mode (Tatsuo Ishii)
Parse() automatically starts a transaction for non SELECT query to keep
consistency among nodes in replication mode. But this wasn't closed. If
wrong query comes in, the transaction goes into an abort state but pgpool
does not close the transaction. Thus next query causes error because the
transaction is still in abort status.
This problem was reported in [pgpool-general: 1877] by Sean Hogan.
[pgpool-general: 1877] current transaction is aborted, commands ignored
http://www.sraoss.jp/pipermail/pgpool-general/2013-July/001905.html
__________________________________________________________________
* Enhancements
- Add mention about "-D" option to the man page.
(Tatsuo Ishii)
- Fix to restart watchdog processes automatically when these exit abnormally.
(Yugo Nagata)
- Add more error checks and reportings to functions executing ping command
with watchdog enabled. (Tatsuo Ishii)
- Replace some unsafe functions, sprintf and strncpy, with more safe ones,
snprintf and strlcpy respectively. (Yugo Nagata)
- Replace "sticky bit" to "setuid bit" in log message, comments and
funcation names. (Yugo Nagata)
These words were used mistakenly and caused confusion.
- Fix description on SSL in pool_hba.conf.sample. (Tatsuo Ishii)
- Allow to load balancing in an explicit transaction in replication mode.
(Tatsuo Ishii)
The condition to allow the load balancing is as follows:
1) replicate_select is off
2) no writing functions are used
3) transaction isolation level is not serializable
4) no DML/DDL are issued in the transaction
- Chinese manual is updated to the latest especially about watchdog. (Bambo Huang)
- Add mention about "-D" option to the man page. (Tatsuo Ishii)
- Move ssl_ca_cert and ssl_ca_cert_dir descriptions to the SSL section
in the document (Yugo Nagata)
- Add ssl_ca_cert and ssl_ca_cert_dir descriptions to the japanese document
(Yugo Nagata)
===============================================================================
pgpoolAdmin 3.3
===============================================================================
pgpoolAdmin 3.3 adopts pgpool-II 3.3.
** New features
- Add new parameters for pgpool-II 3.3 (Nozomi Anzai)
wd_authkey, wd_escalation_command, clear_memqcache_on_escalation,
wd_lifecheck_method, wd_heartbeat_port, wd_heartbeat_keepalive,
wd_heartbeat_deadtime wd_lifecheck_dbname, wd_lifecheck_user,
wd_lifecheck_password, heartbeat_destination, heartbeat_destination_port,
heartbeat_device
- Add a watchdog statuse view in status page (Nozomi Anzai, Yugo Nagata)
This show watchdog statuses of pgpool-IIs and the virtual IP etc. when
watchdog is enabled.
- Add functionality to stop/reload/restart PostgreSQL from GUI
(Nozomi Anzai)
pgpool_pgctl function need to be installed in the backends. This function
is a new feature of pgpool-II 3.3 provided by pgpool_recovery extension.
In addition, a superuser whose name is same as the pgpoolAdmin's login user
also must exists in the backend.
- Add functionality to add/remove a backend from GUI (Nozomi Anzai)
- Show the hostname on which pgpoolAdmin is working in status page
(Andrew Bruce, Nozomi Anzai)
- Add fuctionality to show limited lines in log view (Yugo Nagata)
** Bug fixes
- Fix to not write deprecated parmeters to when updating pgpool.conf
(Yugo Nagata)
--
Yugo Nagata <nagata at sraoss.co.jp>
More information about the pgpool-hackers
mailing list