Heap usage in pthread allocates> 100 MB of RAM
A simple test case pthread allocates the following RAM, measured by the VIRT column at the top:
No use of pthread / heap: 4224 kB
Use pthread, but no heap: 14716 kB
Use pthread and heap in main (but not in thread): 23632 kB
Use pthread and heap in main and thread: 89168 kB
Why will so much RAM be allocated? valgrind --page-as-heap = yes shows 127MB heap heap allocation.
I read that threads share the heap, so why a big jump when using the heap from pthread? The code will then target an embedded system with very limited resources, so understanding these allocations is essential.
Compiled with the following (g ++ version 5.4.0 on Ubuntu 16.04):
g++ -O2 -pthread thread.cpp
thread.cpp:
#include <iostream>
#include <pthread.h>
#include <unistd.h>
#define USE_THREAD
#define USE_HEAP_IN_MAIN
#define USE_HEAP_IN_THREAD
int *arr_thread;
int *arr_main;
void *use_heap(void *arg){
#ifdef USE_HEAP_IN_THREAD
arr_thread=new int[10];
arr_thread[0]=1234;
#endif
usleep(10000000);
}
int main() {
pthread_t t1;
#ifdef USE_HEAP_IN_MAIN
arr_main=new int[10];
arr_main[0]=5678;
#endif
#ifdef USE_THREAD
pthread_create(&t1, NULL, &use_heap, NULL);
pthread_join(t1,NULL);
#else
usleep(10000000);
#endif
return 0;
}
- edited to use global ptrs to demonstrate the effect with -O2.
source to share
I read that threads share a bunch
Right.
so why a big jump when using heap from pthread?
GLIBC (and many others) implementations of malloc use per-thread arenas to avoid all the problems associated with a single heap lock.
The very first one malloc(1)
in each thread will create a whole new arena (usually through mmap
). The arena size is an implementation detail, 2 or 4 MB is not unreasonable.
RAM measured by the top-level VIRT column
This is not a very good measurement. A program can have a huge virt size, but use very little actual memory. Just try mmap 1GB of memory, but don't touch it - your program VIRT
will go through the roof, but the memory available on the system will not be reduced at all.
The code will then target an embedded system with very limited resources, so understanding these allocations is very important.
In this case, you need better tools than top
. Accurately accounting for actual memory usage in Linux is a little more complicated. You can start by searching /proc/$pid/smaps
.
source to share