DM8内存占用问题的一种分析方法

• 前言
• 内存组成
• • 操作系统内存
  • 用户内存
• 内存工具
• valgrind使用
• • 检测内存泄漏
  • 检测内存分配
  • 内存分配分析
  • 一个简单的例子
• callgrind使用
• 总结

代码的好坏，很大程度上可以反映在资源占用和执行效率上。不仅是CPU，内存亦是如此。例如经常被提到的SWAP内存使用问题，其本质思路并不在于如何禁止系统使用SWAP，而是为何其需要使用SWAP以及其内存开销来自于何处，本篇希望从内存的角度提供一种方法窥探数据库做了什么，以及其占用的内存是如何被消耗的，进而帮助理解内存分配行为。

• cache(page cache)

cache是为了协调CPU和MEMORY之间工作机制而产生的，其主要解决了工作速率不一致和共享的问题
CPU在发起读写的指令后就通过异步IO进行，提升CPU时间片利用率
通过将io单元块在MEMORY缓存，多个进程要访问某个文件时，直接从cache读取即可。

• buffer

buffer是为了协调DISK与MEMORY之间的工作机制而产生的，其主要解决了重复寻址，离散读写，磁盘碎片的问题
通过将短期内突发IO进行缓冲，起到流量整形的作用，使得在下层更为平稳，提升整体性能，也可以对磁盘起到保护作用。

只要采用常规读写机制，则必定会产生并消耗这部分内存，这也是linux对文件系统的读写优化机制之一。
尽管可以通过一些手段，例如drop_caches方法进行清理，但暴力清理的风险性是存在的，从原理上如果清理的page cache并非inactive，则使用到的进程就会出现问题，这一点在Oracle RAC的一些案例中已经得到了验证
此外如果清理的page cache为一些工作流程本身需要加载的文件，则这部分文件也会重新加载，不但没有性能提升，反而会使其下降，更为优雅的方式可以通过vfs_cache_pressure来增加频率进行回收，还可以通过min_free_kbytes来选择较好的触发回时机。

• swap

除了以上两种，linux 还引入了 swap 技术，大部分人可能会将swap单纯理解为Windows系统下的“虚拟内存”，当物理内存不足时，拿出部分硬盘空间当SWAP分区（虚拟成内存）使用，从而解决内存容量不足的情况，而实际上并不是这样，其使用的机制并非单纯与free容量相关，也并非只影响最后使用的那个人。

内存分配时需要连续且满足大小需求，否则则会申请失败。
例如一个内存100M的电脑，有四个程序。a（20M）b（10M）c（60M）d（30M）
依次启动a,b,c三个程序，这时如果向启动d程序，自然启动不起来，因为内存不够了。那么把a关掉，剩余内存就够30M了，但其实依然无法起来，因为现在虽然有30M的空闲，但是并不连续，无法分配的空间就是内存碎片。
这时想要解决的话，根据上边的四个程序，我们需要将b c程序的内存依次前移，使得空间空间连续。但是内存并不支持一大块空间直接移动，所以就出现了swap。如果将b程序先放交换空间上，再把交换空间上的东西挪到我们想要前移的地址上，就完成了内存的整理，也就解决了内存碎片的问题。但是由于硬盘的操作比内存慢的多，所以大内存进程自然会受到更严重性能影响。
此外内核对swap的使用机制是将不活跃的内存移动到swap中，在需要的时候再换回来，尽管这种倾向性可以调整（vm.swappiness），但规则本身是无法改变的，除非彻底关闭了swap。所以只要开着swap，看到使用了swap也不必惊讶，如果内存充足换进去的也不会是某个进程用到的内存页，当然由于磁盘和内存的性能差异，如果swap很大则会将这个问题放大，带来更多隐患。

用户内存也就是我们程序使用的内存，从系统FREE中进行申请，主要包括堆，栈，静态存储区，静态存储区主要存储全局变量与 static 变量，会在程序结束时自动释放，这里不做讨论，主要针对堆和栈进行对比

• 管理方式
  栈内存是由编译器自动管理，通常会在调用结束自动释放，无需人工干涉
  堆内存是由释由程序员控制，需要手动进行申请和释放，如果未进行对应free，写入内存越界损坏内存结构导致无法free或特殊原因则可能产生memory leakage（此处不讨论基于RAII或GC机制的场景）
• 存放内容
  栈内存主要存放函数，形参和局部变量，以及函数调用开辟的栈帧（这部分因语言存在差异，）
  堆内存主要存放malloc/realloc的内存
• 空间大小
  栈内存Windows一般大小是1M或2M，Linux下,默认栈空间大小为8MB，可通过ulimit -s来设置，栈空间不足时会触发overflow
  堆内存获得空间根据系统的物理内存，有效虚拟内存有关，同时也与系统位数，内核参数允许申请的内存段大小等诸多因素有关，例如常见的32位系统最大支持4G内存空间
• 碎片问题
  栈内存由于其LIFO的操作方式，不会产生碎片，空间连续
  堆内存是由链表组成，其地址可能并不连续，会产生碎片，分为内部碎片和外部碎片
  内部碎片：主要是因为处理器内存对齐，可能会分配超过实际需求的内存，因此由所需大小四舍五入而产生的多余空间就叫内部碎片
  外部碎片： 频繁的分配与回收物理页面会导致大量的、连续且小的页面块夹杂在已分配的页面中间，就会产生外部碎片
• 生长方向
  栈内存是从顶部向低地址方向扩展
  堆内存是从底部向高地址方向扩展
• 分配方式
  栈内存有2种分配方式，静态分配和动态分配
  静态由编译器完成，如声明局部变量，动态由alloca函数完成
  堆内存只能动态分配，如malloc
• 分配效率
  栈内存是系统提供的数据结构，计算机会在底层对栈提供支持，分配专门的寄存器存放栈的地址，压栈出栈等操作都有汇编命令（PUSH POP MOV等）所以效率很高
  栈内存是C/C++函数库提供的，它的机制较为复杂，为了分配一块内存，库函数会按照一定的算法在堆内存中搜索足够大小的连续空间，如果没有足够大小的连续空间，就有可能通过其他系统手段来增加数据段的内存空间使其有机会分到足够大小的内存，其机制的复杂决定了堆的效率比栈要低得多

内存工具最主要是用来帮助开发人员检测程序本身是否存在内存泄露，内存分配行为检验以及其对句柄/指针的管理是否及时关闭，个人比较常用的是valgrind，它最主要包含下面这些模块
Valgrind一般包含下列工具：

• Memcheck
  最常用的工具，用来检测程序中出现的内存问题，所有对内存的读写都会被检测到，一切对malloc()/free()/new/delete的调用都会被捕获。所以，它能检测以下问题：
  （1） 使用未初始化的内存(Use of uninitialised memory)
  （2） 使用已经释放了的内存(Reading/writing memory after it has been free’d)
  （3） 使用超过malloc分配的内存空间(Reading/writing off the end of malloc’d blocks)
  （4） 对堆栈的非法访问(Reading/writing inappropriate areas on the stack)
  （5） 申请的空间是否有释放(Memory leaks – where pointers to malloc’d blocks are lost forever)
  （6） malloc/free/new/delete申请和释放内存的匹配(Mismatched use of malloc/new/new [] vs free/delete/delete [])
  （7） src和dst的重叠(Overlapping src and dst pointers in memcpy() and related functions)
• Callgrind
  和另一篇中提到的perf/gprof类似的分析工具，可以建立函数调用关系图，帮助找到瓶颈点或者热点调用，还可以有选择地进行cache模拟。
• Cachegrind
  Cache分析器，它模拟CPU中的一级缓存I1，Dl和二级缓存，能够精确地指出程序中cache的丢失和命中。如果需要，它还能够为我们提供cache丢失次数，内存引用次数，以及每行代码，每个函数，每个模块，整个程序产生的指令数。
• Helgrind
  它主要用来检查多线程程序中出现的竞争问题。Helgrind寻找内存中被多个线程访问，而又没有一贯加锁的区域，这些区域往往是线程之间失去同步的地方，而且会导致难以发掘的错误
• Massif
  堆栈分析器，它能测量程序在堆栈中使用了多少内存，告诉我们堆块，堆管理块和栈的大小。

不同平台上的内存工具多种多样，个人原因对linux平台相对熟悉一些，其他平台的工具大体思路相同

这里对内存检测相关的用法进行展示，并讲解其参数作用

检测内存分配通过模块memcheck进行，这里以一个例子展示用法

[dmdba@dmdsc0 bin]$ valgrind --tool=memcheck  --leak-check=full --show-leak-kinds=all --trace-children=yes --track-fds=yes --log-file=leak.log --show-reachable=yes --keep-debuginfo=yes ./dmserver /opt/dsc/dmdbms/data/DAMENG/dm.ini 
file dm.key not found, use default license!
version info: develop
DM Database Server 64 V8 03134283890-20220304-158322-10045 startup...
Normal of FAST
Normal of DEFAULT
Normal of RECYCLE
Normal of KEEP
Normal of ROLL
Database mode = 0, oguid = 0
License will expire on 2023-03-04
file lsn: 31526495
ndct db load finished
ndct second level fill fast pool finished
ndct third level fill fast pool finished
ndct second level fill fast pool finished
ndct third level fill fast pool finished
ndct fill fast pool finished
iid page's trxid[417002]
NEXT TRX ID = 417003
pseg_collect_mgr_items, total collect 0 active_trxs, 0 cmt_trxs, 0 pre_cmt_trxs, 0 to_release_trxs, 0 active_pages, 0 cmt_pages, 0 pre_cmt_pages, 0 to_release_pages, 0 mgr pages, 0 mgr recs!
iid page's trxid[418004]
NEXT TRX ID = 419006.
total 0 active crash trx, pseg_crash_trx_rollback sys_only(0) begin ...
pseg_crash_trx_rollback end, total 0 active crash trx, include 0 empty_trxs, 0 empty_pages which only need to delete mgr recs.
pseg_crash_trx_rollback end
pseg recv finished
nsvr_startup end.
aud sys init success.
aud rt sys init success.
systables desc init success.
ndct_db_load_info success.
nsvr_process_before_open begin.
nsvr_process_before_open success.
total 0 active crash trx, pseg_crash_trx_rollback sys_only(0) begin ...
pseg_crash_trx_rollback end, total 0 active crash trx, include 0 empty_trxs, 0 empty_pages which only need to delete mgr recs.
pseg_crash_trx_rollback end
SYSTEM IS READY.

–tools=memcheck
指定内存检测模块

–leak-check=
要求对leak给出详细信息? 如summary，只反馈一些总结信息，告诉你有多少个malloc，多少个free 等；如果是full将输出所有的leaks，也就是定位到某一个malloc/free。 [default: summary]

–trace-children=
跟踪子线程? [default: no]

–track-fds=
跟踪打开的文件描述？[default: no]

–show-reachable=
如果为no，只输出没有引用的内存leaks，或指向malloc返回的内存块中部某处的leaks [default: no]

–show-leak-kinds= [default: definite,possible]
valgrind 有4种泄漏类型，这个参数决定显示哪些类型泄漏。definite indirect possible reachable 这4种可以设置多个，以逗号相隔，也可以用 all 表示全部类型，none 表示啥都不显示。

–keep-debuginfo= [default: no]
如果程序有使用 动态加载库（dlopen），在动态库卸载时（dlclose），debug信息都会被清除。使能这个选项后，即使动态库被卸载，也会保留调用栈信息。

–log-file=leak.log
valgrind 打印日志转存到指定文件或者文件描述符。如果没有这个参数，valgrind 的日志会连同用户程序的日志一起输出，对于大多数使用者来说，会显得非常乱。

把日志输出到文件的话，还支持一些特殊动态变量，可以实现按进程ID或者序号保存到不同文件。我之前没留意到有这个功能，结果发现不同进程写入到同一个文件，后面写入的检查结果把其他进程的检查结果覆盖了。以下是输出到文件支持的一些动态变量：

%n：会重置为一个进程唯一的文件序列号
%p：表示当前进程的 ID 。多进程时且使能了 trace-children=yes 跟踪子进程时会非常实用
%q{FOO}：实用环境变量 FOO 的值。适用于那种不同进程会设置不同变量的情况。
%%：转意成一个百分号。

必须正常退出程序才能生成检测结果，KILL -9之类异常会破坏捕获流程
结束检测

exit
Server is stopping...
listener closed  and all sessions disconnected
adjust undo_retention & wakeup purge thread...full check point starting...
generate force checkpoint, rlog free space[536721920], used space[140800]
checkpoint begin, used_space[140800], free_space[536721920]...
checkpoint end, 0 pages flushed, used_space[140800], free_space[536721920].
full check point end.
shutdown audit subsystem...OK
shutdown schedule subsystem...OK
shutdown timer successfully.
pre-shutdown MAL subsystem...OK
shutdown worker threads subsystem...OK
shutdown local parallel threads pool successfully.
shutdown replication subsystem...OK
shutdown sequence cache subsystem...OK
wait for mtsk link worker to exit..OK
shutdown mpp session subsystem...OK
wait for rapply is all over... OK
rapply worker threads exit successfully.
pre ending task & worker threads...OK
shutdown dblink subsystem...OK
shutdown pthd_pools...OK
shutdown session subsystem...OK
shutdown rollback segments purging subsystem...OK
shutdown rps subsystem...OK
shutdown transaction subsystem...OK
shutdown locking subsystem...OK
shutdown dbms_lock subsystem...OK
ending tsk and worker threads...OK
ckpt2_exec_immediately begin.
checkpoint begin, used_space[140800], free_space[536721920]...
checkpoint end, 6 pages flushed, used_space[12800], free_space[536849920].
checkpoint begin, used_space[12800], free_space[536849920]...
checkpoint end, 0 pages flushed, used_space[12800], free_space[536849920].
checkpoint begin, used_space[12800], free_space[536849920]...
checkpoint end, 0 pages flushed, used_space[0], free_space[536862720].
checkpoint begin, used_space[0], free_space[536862720]...
checkpoint end, 0 pages flushed, used_space[0], free_space[536862720].
shutdown archive subsystem...OK
shutdown redo log subsystem...OK
shutdown MAL subsystem...OK
shutdown message compress subsystem successfully.
shutdown task subsystem...OK
shutdown trace subsystem...OK
shutdown svr_log subsystem...OK
shutdown plan cache subsystem...OK
shutdown database dictionary subsystem...OK
shutdown file subsystem...OK
shutdown mac cache subsystem...OK
shutdown dynamic login cache subsystem...OK
shutdown ifun/bifun/sfun/afun cache subsystem...OK
shutdown crypt subsystem...OK
shutdown pipe subsystem...OK
shutdown compress component...OK
shutdown slave redo subsystem...OK
shutdown kernel buffer subsystem...OK
shutdown SQL capture subsystem...OK
shutdown control file system...OK
shutdown dtype subsystem...OK
shutdown huge buffer and memory pools...OK
close lsnr socket
DM Database Server shutdown successfully.

检测文件

-rw-r--r-- 1 dmdba dinstall  2066173 5月   4 20:25 leak.log

该文件内部会按线程标识处所有存在泄露可能的位置，此处由于我设置了FULL，所以会未初始化也会打印出来

单就泄露而言，可以直接看LEAK SUMMARY中的definitely lost，这表示代码存在逻辑上的泄露，需要对照源码进行修正，对于成熟的产品而言一般这部分会在代码提交之前就解决掉，而其他类型的泄露从严格规范上来说应当尽量避免，可能会在某些BUG中触发，但也可能并不会对资源使用和释放产生影响

检测内存分配通过模块massif进行，这里也以一个例子展示

[dmdba@dmdsc0 bin]$ valgrind -v --tool=massif --time-unit=B --detailed-freq=1 --keep-debuginfo=yes -s --trace-children=yes --track-fds=yes  --massif-out-file=./massif.out   ./dmserver /opt/dsc/dmdbms/data/DAMENG/dm.ini
==35740== Massif, a heap profiler
==35740== Copyright (C) 2003-2017, and GNU GPL'd, by Nicholas Nethercote
==35740== Using Valgrind-3.19.0-8d3c8034b8-20220411 and LibVEX; rerun with -h for copyright info
==35740== Command: ./dmserver /opt/dsc/dmdbms/data/DAMENG/dm.ini
==35740== 
--35740-- Valgrind options:
--35740--    -v
--35740--    --tool=massif
--35740--    --time-unit=B
--35740--    --detailed-freq=1
--35740--    --keep-debuginfo=yes
--35740--    -s
--35740--    --trace-children=yes
--35740--    --track-fds=yes
--35740--    --massif-out-file=./massif.out
--35740-- Contents of /proc/version:
--35740--   Linux version 3.10.0-957.el7.x86_64 (mockbuild@x86-040.build.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-36) (GCC) ) #1 SMP Thu Oct 4 20:48:51 UTC 2018
--35740-- 
--35740-- Arch and hwcaps: AMD64, LittleEndian, amd64-cx16-lzcnt-rdtscp-sse3-ssse3-avx-avx2-bmi-f16c-rdrand-rdseed
--35740-- Page sizes: currently 4096, max supported 4096
--35740-- Valgrind library directory: /usr/local/libexec/valgrind
--35740-- Massif: alloc-fns:
--35740-- Massif:   malloc
--35740-- Massif:   __builtin_new
--35740-- Massif:   operator new(unsigned)
--35740-- Massif:   operator new(unsigned long)
--35740-- Massif:   __builtin_vec_new
--35740-- Massif:   operator new[](unsigned)
--35740-- Massif:   operator new[](unsigned long)
--35740-- Massif:   calloc
--35740-- Massif:   realloc
--35740-- Massif:   memalign
--35740-- Massif:   posix_memalign
--35740-- Massif:   valloc
--35740-- Massif:   operator new(unsigned, std::nothrow_t const&)
--35740-- Massif:   operator new[](unsigned, std::nothrow_t const&)
--35740-- Massif:   operator new(unsigned long, std::nothrow_t const&)
--35740-- Massif:   operator new[](unsigned long, std::nothrow_t const&)
--35740-- Massif: ignore-fns:
--35740-- Massif:   
--35740-- Reading syms from /opt/dsc/dmdbms/bin/dmserver
--35740-- Reading syms from /usr/lib64/ld-2.17.so
--35740-- Reading syms from /usr/local/libexec/valgrind/massif-amd64-linux
--35740--    object doesn't have a dynamic symbol table
--35740-- Scheduler: using generic scheduler lock implementation.
==35740== embedded gdbserver: reading from /tmp/vgdb-pipe-from-vgdb-to-35740-by-dmdba-on-dmdsc0
==35740== embedded gdbserver: writing to   /tmp/vgdb-pipe-to-vgdb-from-35740-by-dmdba-on-dmdsc0
==35740== embedded gdbserver: shared mem   /tmp/vgdb-pipe-shared-mem-vgdb-35740-by-dmdba-on-dmdsc0
==35740== 
==35740== TO CONTROL THIS PROCESS USING vgdb (which you probably
==35740== don't want to do, unless you know exactly what you're doing,
==35740== or are doing some strange experiment):
==35740==   /usr/local/libexec/valgrind/../../bin/vgdb --pid=35740 ...command...
==35740== 
==35740== TO DEBUG THIS PROCESS USING GDB: start GDB like this
==35740==   /path/to/gdb ./dmserver
==35740== and then give GDB the following command
==35740==   target remote | /usr/local/libexec/valgrind/../../bin/vgdb --pid=35740
==35740== --pid is optional if only one valgrind process is running
==35740== 
--35740-- Reading syms from /usr/local/libexec/valgrind/vgpreload_core-amd64-linux.so
--35740-- Reading syms from /usr/local/libexec/valgrind/vgpreload_massif-amd64-linux.so
--35740-- Reading syms from /usr/lib64/librt-2.17.so
--35740-- Reading syms from /usr/lib64/libpthread-2.17.so
--35740-- Reading syms from /usr/lib64/libdl-2.17.so
--35740-- Reading syms from /usr/lib64/libstdc++.so.6.0.19
--35740--    object doesn't have a symbol table
--35740-- Reading syms from /usr/lib64/libm-2.17.so
--35740-- Reading syms from /usr/lib64/libc-2.17.so
==35740== WARNING: new redirection conflicts with existing -- ignoring it
--35740--     old: 0x07553a20 (memalign            ) R-> (1011.0) 0x06698ea2 memalign
--35740--     new: 0x07553a20 (memalign            ) R-> (1017.0) 0x06698e72 aligned_alloc
==35740== WARNING: new redirection conflicts with existing -- ignoring it
--35740--     old: 0x07553a20 (memalign            ) R-> (1011.0) 0x06698ea2 memalign
--35740--     new: 0x07553a20 (memalign            ) R-> (1017.0) 0x06698e45 aligned_alloc
==35740== WARNING: new redirection conflicts with existing -- ignoring it
--35740--     old: 0x07553a20 (memalign            ) R-> (1011.0) 0x06698ea2 memalign
--35740--     new: 0x07553a20 (memalign            ) R-> (1017.0) 0x06698e72 aligned_alloc
==35740== WARNING: new redirection conflicts with existing -- ignoring it
--35740--     old: 0x07553a20 (memalign            ) R-> (1011.0) 0x06698ea2 memalign
--35740--     new: 0x07553a20 (memalign            ) R-> (1017.0) 0x06698e45 aligned_alloc
--35740-- Reading syms from /usr/lib64/libgcc_s-4.8.5-20150702.so.1
--35740--    object doesn't have a symbol table
--35740-- REDIR: 0x7553160 (libc.so.6:malloc) redirected to 0x66941c0 (malloc)
--35740-- REDIR: 0xffffffffff600400 (???:???) redirected to 0x58094291 (vgPlain_amd64_linux_REDIR_FOR_vtime)
--35740-- REDIR: 0x7553580 (libc.so.6:free) redirected to 0x6696746 (free)
--35740-- REDIR: 0xffffffffff600000 (???:???) redirected to 0x58094287 (vgPlain_amd64_linux_REDIR_FOR_vgettimeofday)
--35740-- REDIR: 0x7553b80 (libc.so.6:calloc) redirected to 0x669890e (calloc)
--35740-- Reading syms from /usr/lib64/libnss_files-2.17.so
file dm.key not found, use default license!
--35740-- REDIR: 0x7553660 (libc.so.6:realloc) redirected to 0x6698b2b (realloc)
--35740-- Reading syms from /opt/dsc/dmdbms/bin/libcrypto.so
--35740-- Reading syms from /opt/dsc/dmdbms/bin/external_crypto_libs/libopenssl_crypto.so
==35742== embedded gdbserver: reading from /tmp/vgdb-pipe-from-vgdb-to-35742-by-dmdba-on-dmdsc0
==35742== embedded gdbserver: writing to   /tmp/vgdb-pipe-to-vgdb-from-35742-by-dmdba-on-dmdsc0
==35742== embedded gdbserver: shared mem   /tmp/vgdb-pipe-shared-mem-vgdb-35742-by-dmdba-on-dmdsc0
==35742== 
==35742== TO CONTROL THIS PROCESS USING vgdb (which you probably
==35742== don't want to do, unless you know exactly what you're doing,
==35742== or are doing some strange experiment):
==35742==   /usr/local/libexec/valgrind/../../bin/vgdb --pid=35742 ...command...
==35742== 
==35742== TO DEBUG THIS PROCESS USING GDB: start GDB like this
==35742==   /path/to/gdb ./dmserver
==35742== and then give GDB the following command
==35742==   target remote | /usr/local/libexec/valgrind/../../bin/vgdb --pid=35742
==35742== --pid is optional if only one valgrind process is running
==35742== 
==35742== Massif, a heap profiler
==35742== Copyright (C) 2003-2017, and GNU GPL'd, by Nicholas Nethercote
==35742== Using Valgrind-3.19.0-8d3c8034b8-20220411 and LibVEX; rerun with -h for copyright info
==35742== Command: /bin/sh -c getconf PAGESIZE
==35742== 
--35742-- Valgrind options:
--35742--    -v
--35742--    --tool=massif
--35742--    --time-unit=B
--35742--    --detailed-freq=1
--35742--    --keep-debuginfo=yes
--35742--    -s
--35742--    --trace-children=yes
--35742--    --track-fds=yes
--35742--    --massif-out-file=./massif.out
--35742-- Contents of /proc/version:
--35742--   Linux version 3.10.0-957.el7.x86_64 (mockbuild@x86-040.build.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-36) (GCC) ) #1 SMP Thu Oct 4 20:48:51 UTC 2018
--35742-- 
--35742-- Arch and hwcaps: AMD64, LittleEndian, amd64-cx16-lzcnt-rdtscp-sse3-ssse3-avx-avx2-bmi-f16c-rdrand-rdseed
--35742-- Page sizes: currently 4096, max supported 4096
--35742-- Valgrind library directory: /usr/local/libexec/valgrind
--35742-- Massif: alloc-fns:
--35742-- Massif:   malloc
--35742-- Massif:   __builtin_new
--35742-- Massif:   operator new(unsigned)
--35742-- Massif:   operator new(unsigned long)
--35742-- Massif:   __builtin_vec_new
--35742-- Massif:   operator new[](unsigned)
--35742-- Massif:   operator new[](unsigned long)
--35742-- Massif:   calloc
--35742-- Massif:   realloc
--35742-- Massif:   memalign
--35742-- Massif:   posix_memalign
--35742-- Massif:   valloc
--35742-- Massif:   operator new(unsigned, std::nothrow_t const&)
--35742-- Massif:   operator new[](unsigned, std::nothrow_t const&)
--35742-- Massif:   operator new(unsigned long, std::nothrow_t const&)
--35742-- Massif:   operator new[](unsigned long, std::nothrow_t const&)
--35742-- Massif: ignore-fns:
--35742-- Massif:   
--35742-- Reading syms from /usr/bin/bash
--35742--    object doesn't have a symbol table
--35742-- Reading syms from /usr/lib64/ld-2.17.so
--35742-- Reading syms from /usr/local/libexec/valgrind/massif-amd64-linux
--35742--    object doesn't have a dynamic symbol table
--35742-- Scheduler: using generic scheduler lock implementation.
==35742== embedded gdbserver: reading from /tmp/vgdb-pipe-from-vgdb-to-35742-by-dmdba-on-dmdsc0
==35742== embedded gdbserver: writing to   /tmp/vgdb-pipe-to-vgdb-from-35742-by-dmdba-on-dmdsc0
==35742== embedded gdbserver: shared mem   /tmp/vgdb-pipe-shared-mem-vgdb-35742-by-dmdba-on-dmdsc0
==35742== 
==35742== TO CONTROL THIS PROCESS USING vgdb (which you probably
==35742== don't want to do, unless you know exactly what you're doing,
==35742== or are doing some strange experiment):
==35742==   /usr/local/libexec/valgrind/../../bin/vgdb --pid=35742 ...command...
==35742== 
==35742== TO DEBUG THIS PROCESS USING GDB: start GDB like this
==35742==   /path/to/gdb /bin/sh
==35742== and then give GDB the following command
==35742==   target remote | /usr/local/libexec/valgrind/../../bin/vgdb --pid=35742
==35742== --pid is optional if only one valgrind process is running
==35742== 
--35742-- Reading syms from /usr/local/libexec/valgrind/vgpreload_core-amd64-linux.so
--35742-- Reading syms from /usr/local/libexec/valgrind/vgpreload_massif-amd64-linux.so
--35742-- Reading syms from /usr/lib64/libtinfo.so.5.9
--35742--    object doesn't have a symbol table
--35742-- Reading syms from /usr/lib64/libdl-2.17.so
--35742-- Reading syms from /usr/lib64/libc-2.17.so
==35742== WARNING: new redirection conflicts with existing -- ignoring it
--35742--     old: 0x052e5a20 (memalign            ) R-> (1011.0) 0x04c2dea2 memalign
--35742--     new: 0x052e5a20 (memalign            ) R-> (1017.0) 0x04c2de72 aligned_alloc
==35742== WARNING: new redirection conflicts with existing -- ignoring it
--35742--     old: 0x052e5a20 (memalign            ) R-> (1011.0) 0x04c2dea2 memalign
--35742--     new: 0x052e5a20 (memalign            ) R-> (1017.0) 0x04c2de45 aligned_alloc
==35742== WARNING: new redirection conflicts with existing -- ignoring it
--35742--     old: 0x052e5a20 (memalign            ) R-> (1011.0) 0x04c2dea2 memalign
--35742--     new: 0x052e5a20 (memalign            ) R-> (1017.0) 0x04c2de72 aligned_alloc
==35742== WARNING: new redirection conflicts with existing -- ignoring it
--35742--     old: 0x052e5a20 (memalign            ) R-> (1011.0) 0x04c2dea2 memalign
--35742--     new: 0x052e5a20 (memalign            ) R-> (1017.0) 0x04c2de45 aligned_alloc
--35742-- REDIR: 0x52e5160 (libc.so.6:malloc) redirected to 0x4c291c0 (malloc)
--35742-- REDIR: 0x52e5580 (libc.so.6:free) redirected to 0x4c2b746 (free)
--35742-- REDIR: 0xffffffffff600400 (???:???) redirected to 0x58094291 (vgPlain_amd64_linux_REDIR_FOR_vtime)
--35742-- REDIR: 0x52e5b80 (libc.so.6:calloc) redirected to 0x4c2d90e (calloc)
--35742-- REDIR: 0xffffffffff600000 (???:???) redirected to 0x58094287 (vgPlain_amd64_linux_REDIR_FOR_vgettimeofday)
==35742== Massif, a heap profiler
==35742== Copyright (C) 2003-2017, and GNU GPL'd, by Nicholas Nethercote
==35742== Using Valgrind-3.19.0-8d3c8034b8-20220411 and LibVEX; rerun with -h for copyright info
==35742== Command: /bin/getconf PAGESIZE
==35742== 
--35742-- Valgrind options:
--35742--    -v
--35742--    --tool=massif
--35742--    --time-unit=B
--35742--    --detailed-freq=1
--35742--    --keep-debuginfo=yes
--35742--    -s
--35742--    --trace-children=yes
--35742--    --track-fds=yes
--35742--    --massif-out-file=./massif.out
--35742-- Contents of /proc/version:
--35742--   Linux version 3.10.0-957.el7.x86_64 (mockbuild@x86-040.build.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-36) (GCC) ) #1 SMP Thu Oct 4 20:48:51 UTC 2018
--35742-- 
--35742-- Arch and hwcaps: AMD64, LittleEndian, amd64-cx16-lzcnt-rdtscp-sse3-ssse3-avx-avx2-bmi-f16c-rdrand-rdseed
--35742-- Page sizes: currently 4096, max supported 4096
--35742-- Valgrind library directory: /usr/local/libexec/valgrind
--35742-- Massif: alloc-fns:
--35742-- Massif:   malloc
--35742-- Massif:   __builtin_new
--35742-- Massif:   operator new(unsigned)
--35742-- Massif:   operator new(unsigned long)
--35742-- Massif:   __builtin_vec_new
--35742-- Massif:   operator new[](unsigned)
--35742-- Massif:   operator new[](unsigned long)
--35742-- Massif:   calloc
--35742-- Massif:   realloc
--35742-- Massif:   memalign
--35742-- Massif:   posix_memalign
--35742-- Massif:   valloc
--35742-- Massif:   operator new(unsigned, std::nothrow_t const&)
--35742-- Massif:   operator new[](unsigned, std::nothrow_t const&)
--35742-- Massif:   operator new(unsigned long, std::nothrow_t const&)
--35742-- Massif:   operator new[](unsigned long, std::nothrow_t const&)
--35742-- Massif: ignore-fns:
--35742-- Massif:   
--35742-- Reading syms from /usr/bin/getconf
--35742--    object doesn't have a symbol table
--35742-- Reading syms from /usr/lib64/ld-2.17.so
--35742-- Reading syms from /usr/local/libexec/valgrind/massif-amd64-linux
--35742--    object doesn't have a dynamic symbol table
--35742-- Scheduler: using generic scheduler lock implementation.
==35742== embedded gdbserver: reading from /tmp/vgdb-pipe-from-vgdb-to-35742-by-dmdba-on-dmdsc0
==35742== embedded gdbserver: writing to   /tmp/vgdb-pipe-to-vgdb-from-35742-by-dmdba-on-dmdsc0
==35742== embedded gdbserver: shared mem   /tmp/vgdb-pipe-shared-mem-vgdb-35742-by-dmdba-on-dmdsc0
==35742== 
==35742== TO CONTROL THIS PROCESS USING vgdb (which you probably
==35742== don't want to do, unless you know exactly what you're doing,
==35742== or are doing some strange experiment):
==35742==   /usr/local/libexec/valgrind/../../bin/vgdb --pid=35742 ...command...
==35742== 
==35742== TO DEBUG THIS PROCESS USING GDB: start GDB like this
==35742==   /path/to/gdb /bin/getconf
==35742== and then give GDB the following command
==35742==   target remote | /usr/local/libexec/valgrind/../../bin/vgdb --pid=35742
==35742== --pid is optional if only one valgrind process is running
==35742== 
--35742-- Reading syms from /usr/local/libexec/valgrind/vgpreload_core-amd64-linux.so
--35742-- Reading syms from /usr/local/libexec/valgrind/vgpreload_massif-amd64-linux.so
--35742-- Reading syms from /usr/lib64/libc-2.17.so
==35742== WARNING: new redirection conflicts with existing -- ignoring it
--35742--     old: 0x04eb7a20 (memalign            ) R-> (1011.0) 0x04c2dea2 memalign
--35742--     new: 0x04eb7a20 (memalign            ) R-> (1017.0) 0x04c2de72 aligned_alloc
==35742== WARNING: new redirection conflicts with existing -- ignoring it
--35742--     old: 0x04eb7a20 (memalign            ) R-> (1011.0) 0x04c2dea2 memalign
--35742--     new: 0x04eb7a20 (memalign            ) R-> (1017.0) 0x04c2de45 aligned_alloc
==35742== WARNING: new redirection conflicts with existing -- ignoring it
--35742--     old: 0x04eb7a20 (memalign            ) R-> (1011.0) 0x04c2dea2 memalign
--35742--     new: 0x04eb7a20 (memalign            ) R-> (1017.0) 0x04c2de72 aligned_alloc
==35742== WARNING: new redirection conflicts with existing -- ignoring it
--35742--     old: 0x04eb7a20 (memalign            ) R-> (1011.0) 0x04c2dea2 memalign
--35742--     new: 0x04eb7a20 (memalign            ) R-> (1017.0) 0x04c2de45 aligned_alloc
--35742-- REDIR: 0x4eb7160 (libc.so.6:malloc) redirected to 0x4c291c0 (malloc)
--35742-- REDIR: 0x4eb7580 (libc.so.6:free) redirected to 0x4c2b746 (free)
==35742== 
==35742== FILE DESCRIPTORS: 3 open (3 std) at exit.
==35742== 
version info: develop
==35744== embedded gdbserver: reading from /tmp/vgdb-pipe-from-vgdb-to-35744-by-dmdba-on-dmdsc0
==35744== embedded gdbserver: writing to   /tmp/vgdb-pipe-to-vgdb-from-35744-by-dmdba-on-dmdsc0
==35744== embedded gdbserver: shared mem   /tmp/vgdb-pipe-shared-mem-vgdb-35744-by-dmdba-on-dmdsc0
==35744== 
==35744== TO CONTROL THIS PROCESS USING vgdb (which you probably
==35744== don't want to do, unless you know exactly what you're doing,
==35744== or are doing some strange experiment):
==35744==   /usr/local/libexec/valgrind/../../bin/vgdb --pid=35744 ...command...
==35744== 
==35744== TO DEBUG THIS PROCESS USING GDB: start GDB like this
==35744==   /path/to/gdb ./dmserver
==35744== and then give GDB the following command
==35744==   target remote | /usr/local/libexec/valgrind/../../bin/vgdb --pid=35744
==35744== --pid is optional if only one valgrind process is running
==35744== 
==35744== Massif, a heap profiler
==35744== Copyright (C) 2003-2017, and GNU GPL'd, by Nicholas Nethercote
==35744== Using Valgrind-3.19.0-8d3c8034b8-20220411 and LibVEX; rerun with -h for copyright info
==35744== Command: /bin/sh -c chattr -R +A /opt/dsc/dmdbms/data/DAMENG/dm.ctl
==35744== 
--35744-- Valgrind options:
--35744--    -v
--35744--    --tool=massif
--35744--    --time-unit=B
--35744--    --detailed-freq=1
--35744--    --keep-debuginfo=yes
--35744--    -s
--35744--    --trace-children=yes
--35744--    --track-fds=yes
--35744--    --massif-out-file=./massif.out
--35744-- Contents of /proc/version:
--35744--   Linux version 3.10.0-957.el7.x86_64 (mockbuild@x86-040.build.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-36) (GCC) ) #1 SMP Thu Oct 4 20:48:51 UTC 2018
--35744-- 
--35744-- Arch and hwcaps: AMD64, LittleEndian, amd64-cx16-lzcnt-rdtscp-sse3-ssse3-avx-avx2-bmi-f16c-rdrand-rdseed
--35744-- Page sizes: currently 4096, max supported 4096
--35744-- Valgrind library directory: /usr/local/libexec/valgrind
--35744-- Massif: alloc-fns:
--35744-- Massif:   malloc
--35744-- Massif:   __builtin_new
--35744-- Massif:   operator new(unsigned)
--35744-- Massif:   operator new(unsigned long)
--35744-- Massif:   __builtin_vec_new
--35744-- Massif:   operator new[](unsigned)
--35744-- Massif:   operator new[](unsigned long)
--35744-- Massif:   calloc
--35744-- Massif:   realloc
--35744-- Massif:   memalign
--35744-- Massif:   posix_memalign
--35744-- Massif:   valloc
--35744-- Massif:   operator new(unsigned, std::nothrow_t const&)
--35744-- Massif:   operator new[](unsigned, std::nothrow_t const&)
--35744-- Massif:   operator new(unsigned long, std::nothrow_t const&)
--35744-- Massif:   operator new[](unsigned long, std::nothrow_t const&)
--35744-- Massif: ignore-fns:
--35744-- Massif:   
--35744-- Reading syms from /usr/bin/bash
--35744--    object doesn't have a symbol table
--35744-- Reading syms from /usr/lib64/ld-2.17.so
--35744-- Reading syms from /usr/local/libexec/valgrind/massif-amd64-linux
--35744--    object doesn't have a dynamic symbol table
--35744-- Scheduler: using generic scheduler lock implementation.
==35744== embedded gdbserver: reading from /tmp/vgdb-pipe-from-vgdb-to-35744-by-dmdba-on-dmdsc0
==35744== embedded gdbserver: writing to   /tmp/vgdb-pipe-to-vgdb-from-35744-by-dmdba-on-dmdsc0
==35744== embedded gdbserver: shared mem   /tmp/vgdb-pipe-shared-mem-vgdb-35744-by-dmdba-on-dmdsc0
==35744== 
==35744== TO CONTROL THIS PROCESS USING vgdb (which you probably
==35744== don't want to do, unless you know exactly what you're doing,
==35744== or are doing some strange experiment):
==35744==   /usr/local/libexec/valgrind/../../bin/vgdb --pid=35744 ...command...
==35744== 
==35744== TO DEBUG THIS PROCESS USING GDB: start GDB like this
==35744==   /path/to/gdb /bin/sh
==35744== and then give GDB the following command
==35744==   target remote | /usr/local/libexec/valgrind/../../bin/vgdb --pid=35744
==35744== --pid is optional if only one valgrind process is running
==35744== 
--35744-- Reading syms from /usr/local/libexec/valgrind/vgpreload_core-amd64-linux.so
--35744-- Reading syms from /usr/local/libexec/valgrind/vgpreload_massif-amd64-linux.so
--35744-- Reading syms from /usr/lib64/libtinfo.so.5.9
--35744--    object doesn't have a symbol table
--35744-- Reading syms from /usr/lib64/libdl-2.17.so
--35744-- Reading syms from /usr/lib64/libc-2.17.so
==35744== WARNING: new redirection conflicts with existing -- ignoring it
--35744--     old: 0x052e5a20 (memalign            ) R-> (1011.0) 0x04c2dea2 memalign
--35744--     new: 0x052e5a20 (memalign            ) R-> (1017.0) 0x04c2de72 aligned_alloc
==35744== WARNING: new redirection conflicts with existing -- ignoring it
--35744--     old: 0x052e5a20 (memalign            ) R-> (1011.0) 0x04c2dea2 memalign
--35744--     new: 0x052e5a20 (memalign            ) R-> (1017.0) 0x04c2de45 aligned_alloc
==35744== WARNING: new redirection conflicts with existing -- ignoring it
--35744--     old: 0x052e5a20 (memalign            ) R-> (1011.0) 0x04c2dea2 memalign
--35744--     new: 0x052e5a20 (memalign            ) R-> (1017.0) 0x04c2de72 aligned_alloc
==35744== WARNING: new redirection conflicts with existing -- ignoring it
--35744--     old: 0x052e5a20 (memalign            ) R-> (1011.0) 0x04c2dea2 memalign
--35744--     new: 0x052e5a20 (memalign            ) R-> (1017.0) 0x04c2de45 aligned_alloc
--35744-- REDIR: 0x52e5160 (libc.so.6:malloc) redirected to 0x4c291c0 (malloc)
--35744-- REDIR: 0x52e5580 (libc.so.6:free) redirected to 0x4c2b746 (free)
--35744-- REDIR: 0xffffffffff600400 (???:???) redirected to 0x58094291 (vgPlain_amd64_linux_REDIR_FOR_vtime)
--35744-- REDIR: 0x52e5b80 (libc.so.6:calloc) redirected to 0x4c2d90e (calloc)
--35744-- REDIR: 0xffffffffff600000 (???:???) redirected to 0x58094287 (vgPlain_amd64_linux_REDIR_FOR_vgettimeofday)
==35744== Massif, a heap profiler
==35744== Copyright (C) 2003-2017, and GNU GPL'd, by Nicholas Nethercote
==35744== Using Valgrind-3.19.0-8d3c8034b8-20220411 and LibVEX; rerun with -h for copyright info
==35744== Command: /bin/chattr -R +A /opt/dsc/dmdbms/data/DAMENG/dm.ctl
==35744== 
--35744-- Valgrind options:
--35744--    -v
--35744--    --tool=massif
--35744--    --time-unit=B
--35744--    --detailed-freq=1
--35744--    --keep-debuginfo=yes
--35744--    -s
--35744--    --trace-children=yes
--35744--    --track-fds=yes
--35744--    --massif-out-file=./massif.out
--35744-- Contents of /proc/version:
--35744--   Linux version 3.10.0-957.el7.x86_64 (mockbuild@x86-040.build.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-36) (GCC) ) #1 SMP Thu Oct 4 20:48:51 UTC 2018
--35744-- 
--35744-- Arch and hwcaps: AMD64, LittleEndian, amd64-cx16-lzcnt-rdtscp-sse3-ssse3-avx-avx2-bmi-f16c-rdrand-rdseed
--35744-- Page sizes: currently 4096, max supported 4096
--35744-- Valgrind library directory: /usr/local/libexec/valgrind
--35744-- Massif: alloc-fns:
--35744-- Massif:   malloc
--35744-- Massif:   __builtin_new
--35744-- Massif:   operator new(unsigned)
--35744-- Massif:   operator new(unsigned long)
--35744-- Massif:   __builtin_vec_new
--35744-- Massif:   operator new[](unsigned)
--35744-- Massif:   operator new[](unsigned long)
--35744-- Massif:   calloc
--35744-- Massif:   realloc
--35744-- Massif:   memalign
--35744-- Massif:   posix_memalign
--35744-- Massif:   valloc
--35744-- Massif:   operator new(unsigned, std::nothrow_t const&)
--35744-- Massif:   operator new[](unsigned, std::nothrow_t const&)
--35744-- Massif:   operator new(unsigned long, std::nothrow_t const&)
--35744-- Massif:   operator new[](unsigned long, std::nothrow_t const&)
--35744-- Massif: ignore-fns:
--35744-- Massif:   
--35744-- Reading syms from /usr/bin/chattr
--35744--    object doesn't have a symbol table
--35744-- Reading syms from /usr/lib64/ld-2.17.so
--35744-- Reading syms from /usr/local/libexec/valgrind/massif-amd64-linux
--35744--    object doesn't have a dynamic symbol table
--35744-- Scheduler: using generic scheduler lock implementation.
==35744== embedded gdbserver: reading from /tmp/vgdb-pipe-from-vgdb-to-35744-by-dmdba-on-dmdsc0
==35744== embedded gdbserver: writing to   /tmp/vgdb-pipe-to-vgdb-from-35744-by-dmdba-on-dmdsc0
==35744== embedded gdbserver: shared mem   /tmp/vgdb-pipe-shared-mem-vgdb-35744-by-dmdba-on-dmdsc0
==35744== 
==35744== TO CONTROL THIS PROCESS USING vgdb (which you probably
==35744== don't want to do, unless you know exactly what you're doing,
==35744== or are doing some strange experiment):
==35744==   /usr/local/libexec/valgrind/../../bin/vgdb --pid=35744 ...command...
==35744== 
==35744== TO DEBUG THIS PROCESS USING GDB: start GDB like this
==35744==   /path/to/gdb /bin/chattr
==35744== and then give GDB the following command
==35744==   target remote | /usr/local/libexec/valgrind/../../bin/vgdb --pid=35744
==35744== --pid is optional if only one valgrind process is running
==35744== 
--35744-- Reading syms from /usr/local/libexec/valgrind/vgpreload_core-amd64-linux.so
--35744-- Reading syms from /usr/local/libexec/valgrind/vgpreload_massif-amd64-linux.so
--35744-- Reading syms from /usr/lib64/libe2p.so.2.3
--35744--    object doesn't have a symbol table
--35744-- Reading syms from /usr/lib64/libcom_err.so.2.1
--35744--    object doesn't have a symbol table
--35744-- Reading syms from /usr/lib64/libc-2.17.so
==35744== WARNING: new redirection conflicts with existing -- ignoring it
--35744--     old: 0x052c3a20 (memalign            ) R-> (1011.0) 0x04c2dea2 memalign
--35744--     new: 0x052c3a20 (memalign            ) R-> (1017.0) 0x04c2de72 aligned_alloc
==35744== WARNING: new redirection conflicts with existing -- ignoring it
--35744--     old: 0x052c3a20 (memalign            ) R-> (1011.0) 0x04c2dea2 memalign
--35744--     new: 0x052c3a20 (memalign            ) R-> (1017.0) 0x04c2de45 aligned_alloc
==35744== WARNING: new redirection conflicts with existing -- ignoring it
--35744--     old: 0x052c3a20 (memalign            ) R-> (1011.0) 0x04c2dea2 memalign
--35744--     new: 0x052c3a20 (memalign            ) R-> (1017.0) 0x04c2de72 aligned_alloc
==35744== WARNING: new redirection conflicts with existing -- ignoring it
--35744--     old: 0x052c3a20 (memalign            ) R-> (1011.0) 0x04c2dea2 memalign
--35744--     new: 0x052c3a20 (memalign            ) R-> (1017.0) 0x04c2de45 aligned_alloc
--35744-- Reading syms from /usr/lib64/libpthread-2.17.so
--35744-- REDIR: 0x52c3160 (libc.so.6:malloc) redirected to 0x4c291c0 (malloc)
--35744-- REDIR: 0x52c3580 (libc.so.6:free) redirected to 0x4c2b746 (free)
==35744== 
==35744== FILE DESCRIPTORS: 3 open (3 std) at exit.
==35744== 
--35740-- Reading syms from /opt/dsc/dmdbms/bin/libz.so
--35740-- Reading syms from /opt/dsc/dmdbms/bin/libsnappy.so
--35740-- Reading syms from /opt/dsc/dmdbms/bin/libregex.so
DM Database Server 64 V8 03134283890-20220304-158322-10045 startup...
Normal of FAST
Normal of DEFAULT
Normal of RECYCLE
Normal of KEEP
Normal of ROLL
--35740-- Reading syms from /opt/dsc/dmdbms/bin/libgeos_c.so
--35740-- Reading syms from /opt/dsc/dmdbms/bin/libgeos.so
--35740-- REDIR: 0x6f23eb0 (libstdc++.so.6:operator new(unsigned long)) redirected to 0x66948e1 (operator new(unsigned long))
--35740-- Reading syms from /opt/dsc/dmdbms/bin/libxqilla.so
--35740-- Reading syms from /opt/dsc/dmdbms/bin/libxerces-c-3.1.so
--35740-- Reading syms from /usr/lib64/libnsl-2.17.so
--35740-- Reading syms from /usr/lib64/libcurl.so.4.3.0
--35740--    object doesn't have a symbol table
--35740-- Reading syms from /usr/lib64/libidn.so.11.6.11
--35740--    object doesn't have a symbol table
--35740-- Reading syms from /usr/lib64/libssh2.so.1.0.1
--35740--    object doesn't have a symbol table
--35740-- Reading syms from /usr/lib64/libssl3.so
--35740--    object doesn't have a symbol table
--35740-- Reading syms from /usr/lib64/libsmime3.so
--35740--    object doesn't have a symbol table
--35740-- Reading syms from /usr/lib64/libnss3.so
--35740--    object doesn't have a symbol table
--35740-- Reading syms from /usr/lib64/libnssutil3.so
--35740--    object doesn't have a symbol table
--35740-- Reading syms from /usr/lib64/libplds4.so
--35740--    object doesn't have a symbol table
--35740-- Reading syms from /usr/lib64/libplc4.so
--35740--    object doesn't have a symbol table
--35740-- Reading syms from /usr/lib64/libnspr4.so
--35740--    object doesn't have a symbol table
--35740-- Reading syms from /usr/lib64/libgssapi_krb5.so.2.2
--35740--    object doesn't have a symbol table
--35740-- Reading syms from /usr/lib64/libkrb5.so.3.3
--35740--    object doesn't have a symbol table
--35740-- Reading syms from /usr/lib64/libk5crypto.so.3.1
--35740--    object doesn't have a symbol table
--35740-- Reading syms from /usr/lib64/libcom_err.so.2.1
--35740--    object doesn't have a symbol table
--35740-- Reading syms from /usr/lib64/liblber-2.4.so.2.10.7
--35740--    object doesn't have a symbol table
--35740-- Reading syms from /usr/lib64/libldap-2.4.so.2.10.7
--35740--    object doesn't have a symbol table
--35740-- Reading syms from /usr/lib64/libz.so.1.2.7
--35740--    object doesn't have a symbol table
--35740-- Reading syms from /usr/lib64/libssl.so.1.0.2k
--35740--    object doesn't have a symbol table
--35740-- Reading syms from /usr/lib64/libcrypto.so.1.0.2k
--35740--    object doesn't have a symbol table
--35740-- Reading syms from /usr/lib64/libkrb5support.so.0.1
--35740--    object doesn't have a symbol table
--35740-- Reading syms from /usr/lib64/libkeyutils.so.1.5
--35740--    object doesn't have a symbol table
--35740-- Reading syms from /usr/lib64/libresolv-2.17.so
--35740-- Reading syms from /usr/lib64/libsasl2.so.3.0.0
--35740--    object doesn't have a symbol table
--35740-- Reading syms from /usr/lib64/libselinux.so.1
--35740--    object doesn't have a symbol table
--35740-- Reading syms from /usr/lib64/libcrypt-2.17.so
--35740-- Reading syms from /usr/lib64/libpcre.so.1.2.0
--35740--    object doesn't have a symbol table
--35740-- Reading syms from /usr/lib64/libfreebl3.so
--35740--    object doesn't have a symbol table
--35740-- Reading syms from /usr/lib64/gconv/UTF-16.so
--35740-- REDIR: 0x6f22020 (libstdc++.so.6:operator delete(void*)) redirected to 0x6696c8c (operator delete(void*))
--35740-- REDIR: 0x6f23fc0 (libstdc++.so.6:operator new[](unsigned long)) redirected to 0x66959bf (operator new[](unsigned long))
Database mode = 0, oguid = 0
License will expire on 2023-03-04
file lsn: 31527671
ndct db load finished
ndct second level fill fast pool finished
ndct third level fill fast pool finished
ndct second level fill fast pool finished
ndct third level fill fast pool finished
ndct fill fast pool finished
iid page's trxid[418004]
NEXT TRX ID = 418005
pseg_collect_mgr_items, total collect 0 active_trxs, 0 cmt_trxs, 0 pre_cmt_trxs, 0 to_release_trxs, 0 active_pages, 0 cmt_pages, 0 pre_cmt_pages, 0 to_release_pages, 0 mgr pages, 0 mgr recs!
iid page's trxid[419006]
NEXT TRX ID = 420008.
total 0 active crash trx, pseg_crash_trx_rollback sys_only(0) begin ...
pseg_crash_trx_rollback end, total 0 active crash trx, include 0 empty_trxs, 0 empty_pages which only need to delete mgr recs.
pseg_crash_trx_rollback end
pseg recv finished
nsvr_startup end.
aud sys init success.
aud rt sys init success.
systables desc init success.
ndct_db_load_info success.
nsvr_process_before_open begin.
nsvr_process_before_open success.
total 0 active crash trx, pseg_crash_trx_rollback sys_only(0) begin ...
pseg_crash_trx_rollback end, total 0 active crash trx, include 0 empty_trxs, 0 empty_pages which only need to delete mgr recs.
pseg_crash_trx_rollback end
SYSTEM IS READY.

–tool=massif
指定堆内存检测模块

–time-unit=B
设置Massif分析的时间单位。有三种：按照程序运行的指令数、按照时间（毫秒）、按照申请、释放的字节数变化。最后一种对于小程序和测试时非常有用，因为它的复现性比较好

–detailed-freq=1
设置详细快照的频率。当设置为1（–detailed-freq=1）时，每个快照都被记录成详细的

–massif-out-file=massif.out.%p
将Massif分析数据写入指定file 而不是默认输出文件massif.out。文件名中可以添加 %p和%q用来加入进程ID和%q后指定环境变量的内容

其他一些和memcheck通用

结束检测

exit
Server is stopping...
listener closed  and all sessions disconnected
adjust undo_retention & wakeup purge thread...full check point starting...
generate force checkpoint, rlog free space[536675840], used space[186880]
checkpoint begin, used_space[186880], free_space[536675840]...
checkpoint end, 0 pages flushed, used_space[186880], free_space[536675840].
full check point end.
shutdown audit subsystem...OK
shutdown schedule subsystem...OK
shutdown timer successfully.
pre-shutdown MAL subsystem...OK
shutdown worker threads subsystem...OK
shutdown local parallel threads pool successfully.
shutdown replication subsystem...OK
shutdown sequence cache subsystem...OK
wait for mtsk link worker to exit..OK
shutdown mpp session subsystem...OK
wait for rapply is all over... OK
rapply worker threads exit successfully.
pre ending task & worker threads...OK
shutdown dblink subsystem...OK
shutdown pthd_pools...OK
shutdown session subsystem...OK
shutdown rollback segments purging subsystem...OK
shutdown rps subsystem...OK
shutdown transaction subsystem...OK
shutdown locking subsystem...OK
shutdown dbms_lock subsystem...OK
ending tsk and worker threads...OK
ckpt2_exec_immediately begin.
checkpoint begin, used_space[186880], free_space[536675840]...
checkpoint end, 6 pages flushed, used_space[12800], free_space[536849920].
checkpoint begin, used_space[12800], free_space[536849920]...
checkpoint end, 0 pages flushed, used_space[12800], free_space[536849920].
checkpoint begin, used_space[12800], free_space[536849920]...
checkpoint end, 0 pages flushed, used_space[0], free_space[536862720].
checkpoint begin, used_space[0], free_space[536862720]...
checkpoint end, 0 pages flushed, used_space[0], free_space[536862720].
shutdown archive subsystem...OK
shutdown redo log subsystem...OK
shutdown MAL subsystem...OK
--35740-- Archiving syms at 0x172f83b0-0x1730de78 in /opt/dsc/dmdbms/bin/libz.so (have_dinfo 1)
--35740-- Scanning and archiving ExeContexts ...
--35740-- Scanned 2,809 ExeContexts, archived 0 ExeContexts
--35740-- Archiving syms at 0x17517430-0x1751cfe8 in /opt/dsc/dmdbms/bin/libsnappy.so (have_dinfo 1)
--35740-- Scanning and archiving ExeContexts ...
--35740-- Scanned 2,809 ExeContexts, archived 0 ExeContexts
shutdown message compress subsystem successfully.
shutdown task subsystem...OK
shutdown trace subsystem...OK
shutdown svr_log subsystem...OK
shutdown plan cache subsystem...OK
shutdown database dictionary subsystem...OK
shutdown file subsystem...OK
shutdown mac cache subsystem...OK
shutdown dynamic login cache subsystem...OK
shutdown ifun/bifun/sfun/afun cache subsystem...OK
--35740-- Archiving syms at 0xa0f01f0-0xa0f2a08 in /opt/dsc/dmdbms/bin/external_crypto_libs/libopenssl_crypto.so (have_dinfo 1)
--35740-- Scanning and archiving ExeContexts ...
--35740-- Scanned 2,815 ExeContexts, archived 0 ExeContexts
shutdown crypt subsystem...OK
shutdown pipe subsystem...OK
shutdown compress component...OK
shutdown slave redo subsystem...OK
shutdown kernel buffer subsystem...OK
shutdown SQL capture subsystem...OK
--35740-- Archiving syms at 0x1ee2aac0-0x1ee3f8c8 in /opt/dsc/dmdbms/bin/libregex.so (have_dinfo 1)
--35740-- Scanning and archiving ExeContexts ...
--35740-- Scanned 2,829 ExeContexts, archived 29 ExeContexts
shutdown control file system...OK
shutdown dtype subsystem...OK
shutdown huge buffer and memory pools...OK
close lsnr socket
DM Database Server shutdown successfully.
--35740-- Archiving syms at 0x3eaad620-0x3eaaed66 in /usr/lib64/gconv/UTF-16.so (have_dinfo 1)
--35740-- Scanning and archiving ExeContexts ...
--35740-- Scanned 2,842 ExeContexts, archived 2 ExeContexts
--35740-- Archiving syms at 0x84bb130-0x84c2481 in /usr/lib64/libnss_files-2.17.so (have_dinfo 1)
--35740-- Scanning and archiving ExeContexts ...
--35740-- Scanned 2,842 ExeContexts, archived 2 ExeContexts
==35740== 
==35740== FILE DESCRIPTORS: 3 open (3 std) at exit.
==35740== 

检测文件
这里根据进程，生成了3个文件，表示有3个程序被调用

[dmdba@dmdsc0 bin]$ ls -lrt massif.out.*
-rw-r--r-- 1 dmdba dinstall  43595 5月   4 16:39 massif.out.25491
-rw-r--r-- 1 dmdba dinstall  50457 5月   4 16:39 massif.out.25493
-rw-r--r-- 1 dmdba dinstall 251161 5月   4 16:40 massif.out.25489

25491文件其实是启动中调用了/bin/getconf PAGESIZE所生成的

25493文件是执行chattr -R +A生成

25489是dmserver程序生成的

直接阅读堆栈和信息比较麻烦，所以需要借助一些图形化工具的帮助，这里主要介绍两种

• ms_print

[dmdba@dmdsc0 bin]$ ms_print massif.out.25489 >2

其内容包括这样几个部分
首先是内存分配和时间的坐标图，横轴为快照，纵轴为内存
之后会依次打印出每个快照时刻内存分配总量，及其来自于哪个调用和占比，缩进相同的部分表示同一时间占用内存

由于文本难以依次比较快照来了解内存变化和峰值时刻，所以介绍第二个方式

• massif-visualizer

该工具同时依赖QT5.12和KF5框架的runtime，且存在大量其他底层依赖，安装流程略纠结，在此不展开，但建议还是用centos或者ubuntu安装，RHEL实在是...orz T-T
生成图形

[root@myOracle01 massif]# massif-visualizer massif.out.25489 
QXcbConnection: Failed to initialize XRandr
Qt: XKEYBOARD extension not present on the X server.
libGL error: No matching fbConfigs or visuals found
libGL error: failed to load driver: swrast

界面如下

大体上左侧为内存开销的柱状加折线图，会明确标注出特定拐点的内存实际数值，右侧是每个快照时刻内存的总量，并可以进行展开，上方的几个标签可以帮助调整左侧模型显示，快速选中右侧PEAK快照和调整统计调用的个数

右侧未展开阶段主要帮助我们快速看出内存分配的趋势以及数值变化，便于结合时间点定位到特定的操作行为，展开后大体如下，可以帮助快速定位内存开销来源于某个调用，以及内存开销的具体组成

左侧标签页可以切换到allocators，会按照调用占用最大内存进行逆序排列，右侧下层展开后可以根据调用名对照其具体功能来判断其内存使用合理性

这里我们选择一个简单的实例来稍微讲解如何借助valgrind了解DM8内存的分配及释放，这里我们启动一个实例，执行了一个CTAS，执行了一个分组SQL，进行一次备份后正常关闭，其整体图如下

那么我们来分段观察

第一部分内存变化如下

可以观察到通过os_malloc_ex分配了大约500MB内存，那么这部分是啥呢？其实很清晰就可以从调用名知道，这部分是share_pool的申请，也正对应到我们的参数MEMORY_POOL，当然也会有其他一些小的内存分配，可能为某些变量结构申请，但是由于其大小未观察到扩展，所以暂时不用太关注

由于我测试环境的MEMORY_N_POOLS是1，所以只观察到1个，实际可能会有多个，这部分在此不展开，着重在讲解这种分析的思路上

第二部分内存变化如下

可以观察到通过调用os_malloc_with_chk申请了大约120MB内存，其组成包括hbuf_pool的80MB 也就是参数HUGE_BUFFER，rt_memobj_vpool（run time memory object pool）的36MB，其中包含了pthd_pool的10MB，disql_binddata_history的10MB，dthred_item_memobj的9MB，同时也开始申请BUFFER POOL相关内存，此处的24MB应当为FAST POOL

第三部分内存变化如下

这部分主要是通过buf4_pools_init继续完成了各种BUFFER内存的申请，其中1000MB对应于参数BUFFERS，300应当为RECYCLE，24为此前申请的FAST POOL

第四部分内存变化如下

这部分实际上是在此前申请的rt_memobj_vpool 36MB基础上扩展到86MB，并额外通过mem2_extend_if_necessary申请了110MB内存

第五部分内存变化如下

该SNAP达到了我们内存使用率的PEAK，也就是我们执行所有命令的时刻，此时可以观察到执行sql相关的内存结构，如vm_run_pln,hash_create等等以及trx4_cmtarr_create申请的约80MB的内存


此后随着我们执行关闭命令，内存开始释放，trx4_cmtarr_create内存伴随着其相关的rt_mem_obj_mem2和mem2_extend_if_necessary开始下降

BUFFER 结构依次陆续释放

直到最后将首先申请的MEMORY_POOL释放，并释放各种对象变量

实际上对于内存分析，也可以结合callgrind来进行调用层的查看，使用时只需要在valgrind中加入–xtree feture的参数即可，例如–xtree-memory=full --xtree-memory-file=./xtmemory.kcg， 如果不指定file会自动根据pid生成形如xtmemory.kcg.22955这样的文件，可以通过qcachegrind来对其进行图形化展示，整体如下

其审阅的角度抛开时间维度，单纯从调用维度进行，当我们发现某一个调用占用内存较高可以对齐进行下钻操作，例如

我们想知道os_malloc_with_chk为何占用这么多内存，可以对其进行层层下钻分析



对于调用中的任意一个堆栈，都可以进行完整的内存统计

本文主要针对DM8通过valgrind/callgrind展示一种对其内存资源使用进行窥探的方法，并针对一个简单的启动，执行操作，关闭的简单场景展示了进行分析的最基本方法。需要特别注意的是，内存的申请和释放基于内部机制的逻辑而进行，并非某个操作一定会产生，这一点必须严格遵循代码逻辑进行审查。同时实际处理问题时可能会出现及其复杂的情况，应当尽可能分析每一个步骤前后内存的变化，并结合数据库内部例如v$BUFFERPOOL,v$mem_pool等内存相关视图来进行对应，对于可能存在泄露的匿名内存，应当结合调用和其地址范围配合代码逻辑来排查具体问题。

达梦云适配技术社区
https://eco.dameng.com/