[pgpool-hackers: 457] Re: penultimate patch for exception and memory manager ( EXCEPTION_MGR branch)

Tatsuo Ishii ishii at postgresql.org
Thu Feb 13 07:49:54 JST 2014


Usama,

Thanks for your hard work.

One thing I noticed is, you moved src/tools/pcp to src/utils/pcp. Is
there any reason for this?

Best regards,
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp

From: Muhammad Usama <m.usama at gmail.com>
Subject: Re: penultimate patch for exception and memory manager ( EXCEPTION_MGR branch)
Date: Tue, 11 Feb 2014 19:55:48 +0500
Message-ID: <CAEJvTzWEjDWgtJ3HGCuaSwkGpXzWBETmrM4ShrA5tcNTQFvEuA at mail.gmail.com>

> Hi Tatsuo
> 
> I have rebased the EXCEPTION_MGR branch to current master, please find the
> updated patch generated after rebasing.
> 
> Thanks
> Muhammad Usama
> 
> 
> On Tue, Feb 11, 2014 at 11:17 AM, Tatsuo Ishii <ishii at postgresql.org> wrote:
> 
>> Usama,
>>
>> I have checked EXCEPTION_MGR branch and it seems it is far behind main
>> branch. The last patch brought to the EXCEPTION_MGR was dated
>> 2013/12/19. Can you please rebase the branch?
>>
>> Best regards,
>> --
>> Tatsuo Ishii
>> SRA OSS, Inc. Japan
>> English: http://www.sraoss.co.jp/index_en.php
>> Japanese: http://www.sraoss.co.jp
>>
>> From: Muhammad Usama <m.usama at gmail.com>
>> Subject: penultimate patch for exception and memory manager (
>> EXCEPTION_MGR branch)
>> Date: Mon, 10 Feb 2014 19:53:41 +0500
>> Message-ID: <CAEJvTzUZHKKHjz-ZvYaAsviwVVBKSoX-ByNq9rhANO1zPJYR=
>> w at mail.gmail.com>
>>
>> > Please find the penultimate patch for integration of memory and exception
>> > manager APIs into pgpool.
>> > The patch is generated on the EXCEPTION_MGR development branch and
>> > integrates the said APIs in query processing portion of pgpool-II code.
>> >
>> > Patch summary for review
>> > ======================
>> > The patch is very large mainly because of replacing of pool_error(),
>> > pool_debug() and pool_log() function calls to ereport(ERROR,..)
>> > ereport(DEBUG,..) and ereport(LOG,..) and it would be little time
>> consuming
>> > to review the patch. Although all the above also needs the thorough
>> review,
>> > as replacing pool_error() with ereport(ERROR) or ereport(FATAL) is just
>> not
>> > a cosmetic but changes the flow of code, but there are few things, that
>> > requires the more attention than others.
>> >
>> > Lifecycle of objects.
>> > ================
>> > Since with integration of memory manager, it is very important to manage
>> > the life cycle of each object and making sure the object is created in
>> the
>> > proper MemoryContext. Here is the summary of some important objects and
>> > their residences.
>> >
>> > Some Important Memory Contexts
>> > ============================
>> >
>> > ProcessLoopContext:
>> > This context resets at every main loop iteration of a process
>> >
>> > TopMemoryContext:
>> > This context is same as PostgreSQL's TopMemoryContext and lives for a
>> life
>> > time of process.
>> >
>> > ErrorContext:
>> > Same as PostgreSQL's ErrorContext and used for error processing.
>> >
>> > QueryContext:
>> > Used by pgpool child process and it lives for single query cycle.
>> >
>> >
>> > Lifecycle of Important Objects
>> > ========================
>> >
>> > SessionContext:
>> > The session context consists of a backend and frontend connection and it
>> > lives for a single iteration of main child process loop.
>> > So the session context object is created in ProcessLoopContext.
>> >
>> > ProcessContext:
>> > ConnectionPool:
>> > BackendConnection:
>> > The above objects are required for the life of process hence are created
>> in
>> > TopMemoryContext which lives throughout the process life time.
>> >
>> > Child Frontend connection:
>> > Frontend connection object is placed in the ProcessLoopContext since new
>> > frontend connection is created at every main loop iteration of pgpool
>> child
>> > process.
>> >
>> > PCP Frontend connection:
>> > This object can live till the pcp process so the object for pcp frontend
>> is
>> > created in TopMemoryContext.
>> >
>> > new configuration parameters
>> > ========================
>> >
>> > The patch also adds three new configuration parameters to control the
>> error
>> > reporting style and details.
>> > Parameters names and their behaviours are borrowed from PostgreSQL.
>> >
>> > int log_error_verbosity;    /* controls how much detail about error
>> should
>> > be emitted */
>> >
>> > int client_min_messages;    /*controls which message should be sent to
>> > client */
>> >
>> > int log_min_messages;       /*controls which message should be emitted to
>> > server log */
>> >
>> >
>> > Some other notable changes introduced by the patch.
>> > ==========================================
>> >
>> > The patch changes the error reporting behaviour and now pgpool-II tries
>> to
>> > send all the error and log messages to the connected frontend clients
>> > (controllable by "client_min_messages" configuration parameter ).
>> >
>> > The patch added an extra call back "on_system_exit" to the elog API,
>> which
>> > is not present in the PostgreSQL.
>> > The call back is called at system exit by elog API just before calling
>> the
>> > "on_shmem_exit" call backs.
>> > "on_system_exit" is a single call back and can be used to clean up things
>> > which are dependent on shared memory.
>> >
>> > "repalloc()" behaviour in PostgreSQL palloc API is little different from
>> > C-API counterpart "realloc()".
>> > ("realloc()" behaves as "malloc()" if ptr argument is NULL. While
>> > "repalloc()" does not works on NULL pointers)
>> > So I have changed the behaviour of "repalloc()" to make it more
>> consistent
>> > with C-API.
>> >
>> > The patch makes the libpcp exclusive for the tool binaries only (compile
>> > with POOL_PRIVATE flag ). pgpool binary no longer has the dependancy on
>> the
>> > lib, and uses the pcp_stream.c function directly.
>> > For this new arrangement the patch also rearranges the source code
>> > organisation and moves the pcp_stream.c and pcp_error.c files to
>> > src/utils/pcp directory from src/lib/pcp directory.
>> >
>> > What is still remaining.
>> > ==================
>> > 1- Need to decide the MemoryContext for memcache objects.
>> > 2- Watchdog and few rewrite function are still without the managers.
>> > 3- Replace all remaining occurrences of  pool_error, pool_debug, and
>> > pool_log with appropriate ereport() calls
>> > 4 -Code cleanup
>> >
>> > Reviews comments and suggestions are most welcome
>> >
>> > Thanks
>> > Usama
>>


More information about the pgpool-hackers mailing list