[pgpool-hackers: 349] Import Postgres memory management API in pgpool

Muhammad Usama m.usama at gmail.com
Sat Aug 24 00:11:36 JST 2013


Currently pgpool does not contains any local memory management API, it
relies on standard C Library memory allocation functions for acquiring and
releasing the memory. Absence of a memory manager especially

with the growth of source code and feature makes it vulnerable to serious
memory issues/leakages. A proper implementation of memory manager for
pgpool can also aid in debugging and could improve the overall

performance of the system. Postgres already contains the memory manager API
and this document proposes to adopt the memory management used by PG in
pgpool.


*Some notes about the PG's memory management API.
*

The core of memory management in the Postgres is a MemoryContext and
Postgres do almost all memory allocation in the "memory contexts", which has
a defined lifecycle.

*The basic operations on a memory context are:
** create a context
* allocate a chunk of memory within a context (equivalent of standard C
library's malloc())
* delete a context (including freeing all the memory allocated therein)
* reset a context (free all memory allocated in the context, but not
the context object itself)

Given a chunk of memory previously allocated from a context, one can free
it or reallocate it larger or smaller (corresponding to standard library's
free() and realloc() routines).
These operations return memory to or get more memory from the same context
the chunk was originally allocated in.
At all times there is a "current" context denoted by
the CurrentMemoryContext global variable.  The backend macro palloc()
 implicitly allocates space in that context.  The
MemoryContextSwitchTo() operation selects a new current context (and
returns the previous context, so that the caller can restore the previous
context before exiting).

The main advantage of memory contexts over plain use of malloc/free is that
the entire contents of a memory context can be freed easily, this saves the
trouble of freeing individual chunks of memory.
This is both faster and more reliable than per-chunk bookkeeping.
PG already use this fact to clean up at transaction end, The memory is
reclaimed by resetting all the active contexts.

*Some Notes About the PG memory API(palloc) Versus Standard C Library*

The behaviour of palloc and friends is similar to the standard C
library's malloc and friends, but there are some deliberate differences
too.  Here are some notes to clarify the behaviour.

* If out of memory, palloc and repalloc exit via elog(ERROR).  They
never return NULL, and it is not necessary or useful to test for such a
result.

* palloc(0) is explicitly a valid operation.  It does not return a
NULL pointer, but a valid chunk of which no bytes may be used.  (However,
the chunk might later be repalloc'd larger; it can also be pfree'd
without error.)  disallowed palloc(0).  It seems more consistent to allow
it, however.) Similarly, repalloc allows realloc'ing to zero size.

* pfree and repalloc do not accept a NULL pointer.

*Using the palloc API in pgpool*

As mentioned above, I propose to following the memory management approach
used in PG in PGPOOL. I am planning to do the following to hook the
Postgres memory management API in pgpool.


1- Copy the the memory management source files from
PGSRC/src/backend/utils/mmgr/ (mcxt.c and aset.c) int the pgpool source.

2- Change the behaviour in case of allocation errors, as currently pgpool
does not contains elog API.

3- Create a single TopMemoryContext global memory context.

4- Replace all malloc, free and related memory management functions calls
with palloc and friends.

4- Stabilise the code.

5- Make more memory contexts, inline with the lifecycle of components of
pgpool

6- Stabilise the code.

*References*
PGSRC/src/backend/utils/mmgr/README

Thanks
Muhammad Usama
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.sraoss.jp/pipermail/pgpool-hackers/attachments/20130823/e6fbbabf/attachment.html>


More information about the pgpool-hackers mailing list