[pgpool-hackers: 2355] Re: segfault with pg_md5

Muhammad Usama m.usama at gmail.com
Mon May 22 19:30:17 JST 2017


I am looking into this

Thanks 
Best regards


> On 22-May-2017, at 1:49 PM, Tatsuo Ishii <ishii at sraoss.co.jp> wrote:
> 
> Usama,
> 
> From commit:
> https://git.postgresql.org/gitweb/?p=pgpool2.git;a=commit;h=18f4d2ea08c2562361858b09534490a6fa09d496
> "Allow to configure health check parameters for each individual node."
> 
> I have been experincing regression failure on my Ubuntu 17 laptop.
> Actually it is caused by pg_md5 segfault. For example following
> command reliably segfaults.
> 
> $ temp/installed/bin/pg_md5 -m -f tests/001.load_balance/testdir/etc/pgpool.conf -u aaa aaa
> 
> Here's the stack trace info.
> 
> $ gdb temp/installed/bin/pg_md5
> GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.04) 7.11.1
> Copyright (C) 2016 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
> and "show warranty" for details.
> This GDB was configured as "x86_64-linux-gnu".
> Type "show configuration" for configuration details.
> For bug reporting instructions, please see:
> <http://www.gnu.org/software/gdb/bugs/>.
> Find the GDB manual and other documentation resources online at:
> <http://www.gnu.org/software/gdb/documentation/>.
> For help, type "help".
> Type "apropos word" to search for commands related to "word"...
> Reading symbols from temp/installed/bin/pg_md5...done.
> (gdb) run  -m -f tests/001.load_balance/testdir/etc/pgpool.conf -u aaa aaa
> Starting program: /home/t-ishii/tmp/pgpool2-18f4d2e/src/test/regression/temp/installed/bin/pg_md5 -m -f tests/001.load_balance/testdir/etc/pgpool.conf -u aaa aaa
> *** Error in `/home/t-ishii/tmp/pgpool2-18f4d2e/src/test/regression/temp/installed/bin/pg_md5': free(): invalid next size (fast): 0x0000000000652c60 ***
> ======= Backtrace: =========
> /lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7ffff7a857e5]
> /lib/x86_64-linux-gnu/libc.so.6(+0x7fe0a)[0x7ffff7a8de0a]
> /lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7ffff7a9198c]
> /lib/x86_64-linux-gnu/libc.so.6(+0xe0224)[0x7ffff7aee224]
> /lib/x86_64-linux-gnu/libc.so.6(regcomp+0xbf)[0x7ffff7af569f]
> /home/t-ishii/tmp/pgpool2-18f4d2e/src/test/regression/temp/installed/bin/pg_md5[0x408b6b]
> /home/t-ishii/tmp/pgpool2-18f4d2e/src/test/regression/temp/installed/bin/pg_md5[0x404e84]
> /home/t-ishii/tmp/pgpool2-18f4d2e/src/test/regression/temp/installed/bin/pg_md5[0x407507]
> /home/t-ishii/tmp/pgpool2-18f4d2e/src/test/regression/temp/installed/bin/pg_md5[0x40887e]
> /home/t-ishii/tmp/pgpool2-18f4d2e/src/test/regression/temp/installed/bin/pg_md5[0x401d6c]
> /home/t-ishii/tmp/pgpool2-18f4d2e/src/test/regression/temp/installed/bin/pg_md5[0x401aa8]
> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7ffff7a2e830]
> /home/t-ishii/tmp/pgpool2-18f4d2e/src/test/regression/temp/installed/bin/pg_md5[0x401c29]
> ======= Memory map: ========
> 00400000-00411000 r-xp 00000000 08:05 16694994                           /home/t-ishii/tmp/pgpool2-18f4d2e/src/test/regression/temp/installed/bin/pg_md5
> 00610000-00611000 r--p 00010000 08:05 16694994                           /home/t-ishii/tmp/pgpool2-18f4d2e/src/test/regression/temp/installed/bin/pg_md5
> 00611000-00617000 rw-p 00011000 08:05 16694994                           /home/t-ishii/tmp/pgpool2-18f4d2e/src/test/regression/temp/installed/bin/pg_md5
> 00617000-00655000 rw-p 00000000 00:00 0                                  [heap]
> 7ffff0000000-7ffff0021000 rw-p 00000000 00:00 0 
> 7ffff0021000-7ffff4000000 ---p 00000000 00:00 0 
> 7ffff77f8000-7ffff780e000 r-xp 00000000 08:05 12062567                   /lib/x86_64-linux-gnu/libgcc_s.so.1
> 7ffff780e000-7ffff7a0d000 ---p 00016000 08:05 12062567                   /lib/x86_64-linux-gnu/libgcc_s.so.1
> 7ffff7a0d000-7ffff7a0e000 rw-p 00015000 08:05 12062567                   /lib/x86_64-linux-gnu/libgcc_s.so.1
> 7ffff7a0e000-7ffff7bcd000 r-xp 00000000 08:05 12183870                   /lib/x86_64-linux-gnu/libc-2.23.so
> 7ffff7bcd000-7ffff7dcd000 ---p 001bf000 08:05 12183870                   /lib/x86_64-linux-gnu/libc-2.23.so
> 7ffff7dcd000-7ffff7dd1000 r--p 001bf000 08:05 12183870                   /lib/x86_64-linux-gnu/libc-2.23.so
> 7ffff7dd1000-7ffff7dd3000 rw-p 001c3000 08:05 12183870                   /lib/x86_64-linux-gnu/libc-2.23.so
> 7ffff7dd3000-7ffff7dd7000 rw-p 00000000 00:00 0 
> 7ffff7dd7000-7ffff7dfd000 r-xp 00000000 08:05 12183871                   /lib/x86_64-linux-gnu/ld-2.23.so
> 7ffff7fd2000-7ffff7fd5000 rw-p 00000000 00:00 0 
> 7ffff7ff5000-7ffff7ff8000 rw-p 00000000 00:00 0 
> 7ffff7ff8000-7ffff7ffa000 r--p 00000000 00:00 0                          [vvar]
> 7ffff7ffa000-7ffff7ffc000 r-xp 00000000 00:00 0                          [vdso]
> 7ffff7ffc000-7ffff7ffd000 r--p 00025000 08:05 12183871                   /lib/x86_64-linux-gnu/ld-2.23.so
> 7ffff7ffd000-7ffff7ffe000 rw-p 00026000 08:05 12183871                   /lib/x86_64-linux-gnu/ld-2.23.so
> 7ffff7ffe000-7ffff7fff000 rw-p 00000000 00:00 0 
> 7ffffffde000-7ffffffff000 rw-p 00000000 00:00 0                          [stack]
> ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
> 
> Program received signal SIGABRT, Aborted.
> 0x00007ffff7a43428 in __GI_raise (sig=sig at entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
> 54    ../sysdeps/unix/sysv/linux/raise.c: そのようなファイルやディレクトリはありません.
> (gdb) bt
> #0  0x00007ffff7a43428 in __GI_raise (sig=sig at entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
> #1  0x00007ffff7a4502a in __GI_abort () at abort.c:89
> #2  0x00007ffff7a857ea in __libc_message (do_abort=do_abort at entry=2, 
>    fmt=fmt at entry=0x7ffff7b9e2e0 "*** Error in `%s': %s: 0x%s ***\n")
>    at ../sysdeps/posix/libc_fatal.c:175
> #3  0x00007ffff7a8de0a in malloc_printerr (ar_ptr=<optimized out>, ptr=<optimized out>, 
>    str=0x7ffff7b9e358 "free(): invalid next size (fast)", action=3) at malloc.c:5004
> #4  _int_free (av=<optimized out>, p=<optimized out>, have_lock=0) at malloc.c:3865
> #5  0x00007ffff7a9198c in __GI___libc_free (mem=<optimized out>) at malloc.c:2966
> #6  0x00007ffff7aee224 in analyze (preg=0x7fffffff7e10) at regcomp.c:1166
> #7  re_compile_internal (preg=preg at entry=0x7fffffff7e10, pattern=pattern at entry=0x651e90 "^nextval$", 
>    length=<optimized out>, syntax=syntax at entry=4436732) at regcomp.c:791
> #8  0x00007ffff7af569f in __regcomp (preg=preg at entry=0x7fffffff7e10, pattern=0x651e90 "^nextval$", 
>    cflags=<optimized out>) at regcomp.c:498
> #9  0x0000000000408b6b in add_regex_pattern (type=<optimized out>, s=<optimized out>)
>    at pool_config.l:177
> #10 0x0000000000404e84 in initialize_variables_with_default (
>    gconf=0x6141b0 <ConfigureNamesStringList+304>) at pool_config_variables.c:2363
> #11 0x0000000000407507 in InitializeConfigOptions () at pool_config_variables.c:2449
> #12 0x000000000040887e in pool_init_config () at pool_config.l:119
> #13 0x0000000000401d6c in update_pool_passwd (
>    conf_file=conf_file at entry=0x7fffffffc110 "tests/001.load_balance/testdir/etc/pgpool.conf", 
>    username=username at entry=0x7fffffffbff0 "aaa", password=0x7fffffffe6b8 "aaa") at pg_md5.c:199
> #14 0x0000000000401aa8 in main (argc=7, argv=0x7fffffffe238) at pg_md5.c:179
> (gdb)
> 
> Also valgrind shows some errors.
> 
> t-ishii at localhost: valgrind temp/installed/bin/pg_md5 -m -f tests/001.load_balance/testdir/etc/pgpool.conf -u aaa aaa
> ==16881== Memcheck, a memory error detector
> ==16881== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
> ==16881== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
> ==16881== Command: temp/installed/bin/pg_md5 -m -f tests/001.load_balance/testdir/etc/pgpool.conf -u aaa aaa
> ==16881== 
> ==16881== Invalid write of size 8
> ==16881==    at 0x4074A2: build_variable_groups (pool_config_variables.c:2075)
> ==16881==    by 0x4074A2: build_config_variables (pool_config_variables.c:2016)
> ==16881==    by 0x4074A2: InitializeConfigOptions (pool_config_variables.c:2441)
> ==16881==    by 0x40887D: pool_init_config (pool_config.l:119)
> ==16881==    by 0x401D6B: update_pool_passwd (pg_md5.c:199)
> ==16881==    by 0x401AA7: main (pg_md5.c:179)
> ==16881==  Address 0x5227018 is 0 bytes after a block of size 40 alloc'd
> ==16881==    at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==16881==    by 0x4094B4: pg_malloc (fe_memutils.c:33)
> ==16881==    by 0x4094DD: pg_malloc0 (fe_memutils.c:47)
> ==16881==    by 0x407432: build_variable_groups (pool_config_variables.c:2064)
> ==16881==    by 0x407432: build_config_variables (pool_config_variables.c:2016)
> ==16881==    by 0x407432: InitializeConfigOptions (pool_config_variables.c:2441)
> ==16881==    by 0x40887D: pool_init_config (pool_config.l:119)
> ==16881==    by 0x401D6B: update_pool_passwd (pg_md5.c:199)
> ==16881==    by 0x401AA7: main (pg_md5.c:179)
> ==16881== 
> ==16881== Invalid write of size 8
> ==16881==    at 0x4074B4: build_variable_groups (pool_config_variables.c:2077)
> ==16881==    by 0x4074B4: build_config_variables (pool_config_variables.c:2016)
> ==16881==    by 0x4074B4: InitializeConfigOptions (pool_config_variables.c:2441)
> ==16881==    by 0x40887D: pool_init_config (pool_config.l:119)
> ==16881==    by 0x401D6B: update_pool_passwd (pg_md5.c:199)
> ==16881==    by 0x401AA7: main (pg_md5.c:179)
> ==16881==  Address 0x5227020 is 8 bytes after a block of size 40 alloc'd
> ==16881==    at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==16881==    by 0x4094B4: pg_malloc (fe_memutils.c:33)
> ==16881==    by 0x4094DD: pg_malloc0 (fe_memutils.c:47)
> ==16881==    by 0x407432: build_variable_groups (pool_config_variables.c:2064)
> ==16881==    by 0x407432: build_config_variables (pool_config_variables.c:2016)
> ==16881==    by 0x407432: InitializeConfigOptions (pool_config_variables.c:2441)
> ==16881==    by 0x40887D: pool_init_config (pool_config.l:119)
> ==16881==    by 0x401D6B: update_pool_passwd (pg_md5.c:199)
> ==16881==    by 0x401AA7: main (pg_md5.c:179)
> ==16881== 
> ==16881== Invalid write of size 8
> ==16881==    at 0x4074C6: build_variable_groups (pool_config_variables.c:2079)
> ==16881==    by 0x4074C6: build_config_variables (pool_config_variables.c:2016)
> ==16881==    by 0x4074C6: InitializeConfigOptions (pool_config_variables.c:2441)
> ==16881==    by 0x40887D: pool_init_config (pool_config.l:119)
> ==16881==    by 0x401D6B: update_pool_passwd (pg_md5.c:199)
> ==16881==    by 0x401AA7: main (pg_md5.c:179)
> ==16881==  Address 0x5227028 is 16 bytes after a block of size 40 alloc'd
> ==16881==    at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==16881==    by 0x4094B4: pg_malloc (fe_memutils.c:33)
> ==16881==    by 0x4094DD: pg_malloc0 (fe_memutils.c:47)
> ==16881==    by 0x407432: build_variable_groups (pool_config_variables.c:2064)
> ==16881==    by 0x407432: build_config_variables (pool_config_variables.c:2016)
> ==16881==    by 0x407432: InitializeConfigOptions (pool_config_variables.c:2441)
> ==16881==    by 0x40887D: pool_init_config (pool_config.l:119)
> ==16881==    by 0x401D6B: update_pool_passwd (pg_md5.c:199)
> ==16881==    by 0x401AA7: main (pg_md5.c:179)
> ==16881== 
> ==16881== 
> ==16881== HEAP SUMMARY:
> ==16881==     in use at exit: 159,848 bytes in 3,700 blocks
> ==16881==   total heap usage: 5,872 allocs, 2,172 frees, 483,653 bytes allocated
> ==16881== 
> ==16881== LEAK SUMMARY:
> ==16881==    definitely lost: 4,586 bytes in 1,034 blocks
> ==16881==    indirectly lost: 0 bytes in 0 blocks
> ==16881==      possibly lost: 0 bytes in 0 blocks
> ==16881==    still reachable: 155,262 bytes in 2,666 blocks
> ==16881==         suppressed: 0 bytes in 0 blocks
> ==16881== Rerun with --leak-check=full to see details of leaked memory
> ==16881== 
> ==16881== For counts of detected and suppressed errors, rerun with: -v
> ==16881== ERROR SUMMARY: 3 errors from 3 contexts (suppressed: 0 from 0)
> 
> Any idea?
> 
> Best regards,
> --
> Tatsuo Ishii
> SRA OSS, Inc. Japan
> English: http://www.sraoss.co.jp/index_en.php
> Japanese:http://www.sraoss.co.jp
> _______________________________________________
> pgpool-hackers mailing list
> pgpool-hackers at pgpool.net
> http://www.pgpool.net/mailman/listinfo/pgpool-hackers


More information about the pgpool-hackers mailing list