[pgpool-hackers: 450] penultimate patch for exception and memory manager ( EXCEPTION_MGR branch)

[Muhammad Usama]

Please find the penultimate patch for integration of memory and exception
manager APIs into pgpool.
The patch is generated on the EXCEPTION_MGR development branch and
integrates the said APIs in query processing portion of pgpool-II code.

Patch summary for review
======================
The patch is very large mainly because of replacing of pool_error(),
pool_debug() and pool_log() function calls to ereport(ERROR,..)
ereport(DEBUG,..) and ereport(LOG,..) and it would be little time consuming
to review the patch. Although all the above also needs the thorough review,
as replacing pool_error() with ereport(ERROR) or ereport(FATAL) is just not
a cosmetic but changes the flow of code, but there are few things, that
requires the more attention than others.

Lifecycle of objects.
================
Since with integration of memory manager, it is very important to manage
the life cycle of each object and making sure the object is created in the
proper MemoryContext. Here is the summary of some important objects and
their residences.

Some Important Memory Contexts
============================

ProcessLoopContext:
This context resets at every main loop iteration of a process

TopMemoryContext:
This context is same as PostgreSQL's TopMemoryContext and lives for a life
time of process.

ErrorContext:
Same as PostgreSQL's ErrorContext and used for error processing.

QueryContext:
Used by pgpool child process and it lives for single query cycle.


Lifecycle of Important Objects
========================

SessionContext:
The session context consists of a backend and frontend connection and it
lives for a single iteration of main child process loop.
So the session context object is created in ProcessLoopContext.

ProcessContext:
ConnectionPool:
BackendConnection:
The above objects are required for the life of process hence are created in
TopMemoryContext which lives throughout the process life time.

Child Frontend connection:
Frontend connection object is placed in the ProcessLoopContext since new
frontend connection is created at every main loop iteration of pgpool child
process.

PCP Frontend connection:
This object can live till the pcp process so the object for pcp frontend is
created in TopMemoryContext.

new configuration parameters
========================

The patch also adds three new configuration parameters to control the error
reporting style and details.
Parameters names and their behaviours are borrowed from PostgreSQL.

int log_error_verbosity;    /* controls how much detail about error should
be emitted */

int client_min_messages;    /*controls which message should be sent to
client */

int log_min_messages;       /*controls which message should be emitted to
server log */


Some other notable changes introduced by the patch.
==========================================

The patch changes the error reporting behaviour and now pgpool-II tries to
send all the error and log messages to the connected frontend clients
(controllable by "client_min_messages" configuration parameter ).

The patch added an extra call back "on_system_exit" to the elog API, which
is not present in the PostgreSQL.
The call back is called at system exit by elog API just before calling the
"on_shmem_exit" call backs.
"on_system_exit" is a single call back and can be used to clean up things
which are dependent on shared memory.

"repalloc()" behaviour in PostgreSQL palloc API is little different from
C-API counterpart "realloc()".
("realloc()" behaves as "malloc()" if ptr argument is NULL. While
"repalloc()" does not works on NULL pointers)
So I have changed the behaviour of "repalloc()" to make it more consistent
with C-API.

The patch makes the libpcp exclusive for the tool binaries only (compile
with POOL_PRIVATE flag ). pgpool binary no longer has the dependancy on the
lib, and uses the pcp_stream.c function directly.
For this new arrangement the patch also rearranges the source code
organisation and moves the pcp_stream.c and pcp_error.c files to
src/utils/pcp directory from src/lib/pcp directory.

What is still remaining.
==================
1- Need to decide the MemoryContext for memcache objects.
2- Watchdog and few rewrite function are still without the managers.
3- Replace all remaining occurrences of  pool_error, pool_debug, and
pool_log with appropriate ereport() calls
4 -Code cleanup

Reviews comments and suggestions are most welcome

Thanks
Usama
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.sraoss.jp/pipermail/pgpool-hackers/attachments/20140210/5d6b292b/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: exmge_panultimate.patch.zip
Type: application/zip
Size: 216076 bytes
Desc: not available
URL: <http://www.sraoss.jp/pipermail/pgpool-hackers/attachments/20140210/5d6b292b/attachment-0001.zip>