[pgpool-hackers: 452] Re: penultimate patch for exception and memory manager ( EXCEPTION_MGR branch)

[Tatsuo Ishii]

Usama,

I have checked EXCEPTION_MGR branch and it seems it is far behind main
branch. The last patch brought to the EXCEPTION_MGR was dated
2013/12/19. Can you please rebase the branch?

Best regards,
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp

From: Muhammad Usama <m.usama at gmail.com>
Subject: penultimate patch for exception and memory manager ( EXCEPTION_MGR branch)
Date: Mon, 10 Feb 2014 19:53:41 +0500
Message-ID: <CAEJvTzUZHKKHjz-ZvYaAsviwVVBKSoX-ByNq9rhANO1zPJYR=w at mail.gmail.com>

> Please find the penultimate patch for integration of memory and exception
> manager APIs into pgpool.
> The patch is generated on the EXCEPTION_MGR development branch and
> integrates the said APIs in query processing portion of pgpool-II code.
> 
> Patch summary for review
> ======================
> The patch is very large mainly because of replacing of pool_error(),
> pool_debug() and pool_log() function calls to ereport(ERROR,..)
> ereport(DEBUG,..) and ereport(LOG,..) and it would be little time consuming
> to review the patch. Although all the above also needs the thorough review,
> as replacing pool_error() with ereport(ERROR) or ereport(FATAL) is just not
> a cosmetic but changes the flow of code, but there are few things, that
> requires the more attention than others.
> 
> Lifecycle of objects.
> ================
> Since with integration of memory manager, it is very important to manage
> the life cycle of each object and making sure the object is created in the
> proper MemoryContext. Here is the summary of some important objects and
> their residences.
> 
> Some Important Memory Contexts
> ============================
> 
> ProcessLoopContext:
> This context resets at every main loop iteration of a process
> 
> TopMemoryContext:
> This context is same as PostgreSQL's TopMemoryContext and lives for a life
> time of process.
> 
> ErrorContext:
> Same as PostgreSQL's ErrorContext and used for error processing.
> 
> QueryContext:
> Used by pgpool child process and it lives for single query cycle.
> 
> 
> Lifecycle of Important Objects
> ========================
> 
> SessionContext:
> The session context consists of a backend and frontend connection and it
> lives for a single iteration of main child process loop.
> So the session context object is created in ProcessLoopContext.
> 
> ProcessContext:
> ConnectionPool:
> BackendConnection:
> The above objects are required for the life of process hence are created in
> TopMemoryContext which lives throughout the process life time.
> 
> Child Frontend connection:
> Frontend connection object is placed in the ProcessLoopContext since new
> frontend connection is created at every main loop iteration of pgpool child
> process.
> 
> PCP Frontend connection:
> This object can live till the pcp process so the object for pcp frontend is
> created in TopMemoryContext.
> 
> new configuration parameters
> ========================
> 
> The patch also adds three new configuration parameters to control the error
> reporting style and details.
> Parameters names and their behaviours are borrowed from PostgreSQL.
> 
> int log_error_verbosity;    /* controls how much detail about error should
> be emitted */
> 
> int client_min_messages;    /*controls which message should be sent to
> client */
> 
> int log_min_messages;       /*controls which message should be emitted to
> server log */
> 
> 
> Some other notable changes introduced by the patch.
> ==========================================
> 
> The patch changes the error reporting behaviour and now pgpool-II tries to
> send all the error and log messages to the connected frontend clients
> (controllable by "client_min_messages" configuration parameter ).
> 
> The patch added an extra call back "on_system_exit" to the elog API, which
> is not present in the PostgreSQL.
> The call back is called at system exit by elog API just before calling the
> "on_shmem_exit" call backs.
> "on_system_exit" is a single call back and can be used to clean up things
> which are dependent on shared memory.
> 
> "repalloc()" behaviour in PostgreSQL palloc API is little different from
> C-API counterpart "realloc()".
> ("realloc()" behaves as "malloc()" if ptr argument is NULL. While
> "repalloc()" does not works on NULL pointers)
> So I have changed the behaviour of "repalloc()" to make it more consistent
> with C-API.
> 
> The patch makes the libpcp exclusive for the tool binaries only (compile
> with POOL_PRIVATE flag ). pgpool binary no longer has the dependancy on the
> lib, and uses the pcp_stream.c function directly.
> For this new arrangement the patch also rearranges the source code
> organisation and moves the pcp_stream.c and pcp_error.c files to
> src/utils/pcp directory from src/lib/pcp directory.
> 
> What is still remaining.
> ==================
> 1- Need to decide the MemoryContext for memcache objects.
> 2- Watchdog and few rewrite function are still without the managers.
> 3- Replace all remaining occurrences of  pool_error, pool_debug, and
> pool_log with appropriate ereport() calls
> 4 -Code cleanup
> 
> Reviews comments and suggestions are most welcome
> 
> Thanks
> Usama