Using LibUMem to find memory leaks

LibUMem:

Will not be supported in all versions of Solaris. Support starts from Solaris 10.

Usage: Finding memory leaks
UseCase: Let us suppose we have one executable process is causing memory leak which is observed through prstat or by any other means.


Steps to be followed to find the same:

Program: 1.C

===========================

bash-3.00$ cat 1.C
#include
#include
#include
#include

using namespace std;

char * functionName()
{

time_t current,c6;
struct tm *before6Days;
char result[11];

current = time(NULL);
c6 = current - (6 * 3600 *24);
before6Days = localtime(&c6);
sprintf(result,"%02d/%02d/%04d", before6Days->tm_mon+1, before6Days->tm_mday, before6Days->tm_year+1900);
printf("\n Result : [%s]", result);
return result;
}

int main()
{

char *dateRequired ;
dateRequired = (char *) malloc(11);
dateRequired = functionName();
printf("\n Result in main [%s]", dateRequired);
int test;
scanf("%d", &test);
return 0;
}
======================
$ CC test.C -o Test.TASK &

bash-3.00$ ./a.out &
[1] 9988
bash-3.00$
Result : [11/02/2009]
Result in main [11/02/2009]

[1]+ Stopped ./Test.TASK


**************The program is stopped because it is waiting to accet the value for the scanf statement ****


1) Before executing the process . We should set the environment. for example we are setting up the environment through bash shell

bash$export LD_PRELOAD=libumem.so.1
bash$export UMEM_DEBUG=default
bash$export UMEM_LOGGING=transaction
bash$

2) We will execute the use case which is causing the memory leak. after that
bash$ gcore `pgrep Test.Task` // Dumping core for the task which is causing memory leak in the same shell

3) We will get core file for the Test.task . some times in the same directory from where we executed. or some times it is asper $coreadm command settings (where the core files path is configured)

4) lets us suppose the core file name is core.9988(generally core gets genertaed with core. )

5)
bash-3.00$ mdb core.9988
Loading modules: [ libumem.so.1 libc.so.1 ld.so.1 ] ***Here we can obsver it loaded libumem.so.1****
> $G
C++ symbol demangling enabled
> $C //to get back trace
ffbff7a8 libc.so.1`_read+8(ff06c7bc, 82, ffffffff, ff3ee0f8, 1, 0)
ffbff808 libc.so.1`__doscan_u+0x6dc(82, ff06c06a, ffbff990, 64, ff06c7bc, 0)
ffbffa18 libc.so.1`vscanf+0x88(1115d, ffbffb20, ff06c7bc, 3faf0, ff069288, ff068284)
ffbffa78 libc.so.1`scanf+0x1c(1115d, ffbffb3c, 0, ff06c7da, ff068284, 1d)
ffbffad8 main+0x30(1, ffbffbac, ffbffbb4, 21000, ff390740, ffbffabd)
ffbffb48 _start+0x108(0, 0, 0, 0, 0, 0)
> ::findleaks
CACHE LEAKED BUFCTL CALLER
0004e508 1 000cc4b0 main+4
----------------------------------------------------------------------
Total 1 buffer, 24 bytes
/* Here it is showing a buffer at 000cc4b0 called by main function is giving leak */
> 000cc4b0$ ::umalog
T-0.000000000 addr=7d100 umem_alloc_12288
libumem.so.1`umem_cache_alloc+0x13c
libumem.so.1`umem_alloc+0x60
libumem.so.1`malloc+0x28
libc.so.1`get_zone+0x78
libc.so.1`getsystemTZ+0x10c
libc.so.1`localtime_r+0x1c
char*functionName+0x20
main+0xc
_start+0x108

T-0.000193583 addr=c7fc0 umem_alloc_24
libumem.so.1`umem_cache_alloc+0x13c
libumem.so.1`umem_alloc+0x60
libumem.so.1`malloc+0x28
main+4
_start+0x108
> ::umem_log
CPU ADDR BUFADDR TIMESTAMP THREAD
1 00032b54 0007d100 bd1486f5f64da 00000001
1 00032af0 000c7fc0 bd1486f5c70ab 00000001
1 00032a8c 000c5f00 bd1486f4f4158 00000001
1 00032a28 0008dda8 bd1486f4d9790 00000001
1 000329c4 0008ddd8 bd1486f4d8e1f 00000001
1 00032960 0008de08 bd1486f4cfafc 00000001
1 000328fc 0008de38 bd1486f4cf18c 00000001
1 00032898 0008de68 bd1486f4b7409 00000001
1 00032834 0008de98 bd1486f4b694b 00000001
1 000327d0 00099fe0 bd1486f475077 00000001
1 0003276c 00099fe0 bd1486f45f7bd 00000001
1 00032708 0006bf18 bd1486f3d580e 00000001
1 000326a4 0006bf50 bd1486f3d4486 00000001
1 00032640 00095f88 bd1486f3d07fa 00000001
1 000325dc 0008dec8 bd1486f398c0d 00000001
1 00032578 0008def8 bd1486f39843d 00000001
1 00032514 00080140 bd1486f397bc6 00000001
1 000324b0 0008df28 bd1486f394182 00000001
1 0003244c 0008df58 bd1486f393a05 00000001
1 000323e8 00083180 bd1486f392fee 00000001
1 00032384 0008df88 bd1486f38f020 00000001
1 00032320 0008dfb8 bd1486f38e609 00000001
1 000322bc 000861c0 bd1486f34093e 00000001
1 00032258 0006bf88 bd1486f28bf98 00000001
1 000321f4 00065f80 bd1486f28b1ec 00000001
1 00032190 0005bf28 bd1486f28a346 00000001
1 0003212c 00071f88 bd1486f2842ec 00000001
1 000320c8 0006bfc0 bd1486f24c4b7 00000001
1 00032064 00065fc0 bd1486f1ffc1b 00000001
1 00032000 0005bf88 bd1486f1ba6bb 00000001
0002abb8 00000000 0 00000000
0002ac1c 00000000 0 00000000
0002ac80 00000000 0 00000000
0002ace4 00000000 0 00000000
0002ad48 00000000 0 00000000
0002adac 00000000 0 00000000
0002ae10 00000000 0 00000000
0002ae74 00000000 0 00000000
0002aed8 00000000 0 00000000
0002af3c 00000000 0 00000000
0002afa0 00000000 0 00000000
0002b004 00000000 0 00000000
0002b068 00000000 0 00000000
---
In above we can see the number of transactions in which CPU1 is executing around 30.
From : 1 00032b54 0007d100 bd1486f5f64da 00000001
:
:
To : 1 00032000 0005bf88 bd1486f1ba6bb 00000001---
---
> 000cc4b0$ c7fc0/10X
0xc7fc0: 13 3a10bfed baddcafe baddcafe baddcabb
baddcafe feedface 12a2 cc4b0 a11c0c5d

/* The below shows there is no memory corruption as such in our program, but only memory leak)
> ::umem_verify
Cache Name Addr Cache Integrity
umem_magazine_1 4c008 clean
umem_magazine_3 4c288 clean
umem_magazine_7 4c508 clean
umem_magazine_15 4c788 clean
umem_magazine_31 4ca08 clean
umem_magazine_47 4cc88 clean
umem_magazine_63 4cf08 clean
umem_magazine_95 4d188 clean
umem_magazine_143 4d408 clean
umem_slab_cache 4d688 clean
umem_bufctl_cache 4d908 clean
umem_bufctl_audit_cache 4db88 clean
umem_alloc_8 4e008 clean
umem_alloc_16 4e288 clean
umem_alloc_24 4e508 clean
umem_alloc_32 4e788 clean
umem_alloc_40 4ea08 clean
umem_alloc_48 4ec88 clean
umem_alloc_56 4ef08 clean
umem_alloc_64 4f188 clean
umem_alloc_80 4f408 clean
umem_alloc_96 4f688 clean
umem_alloc_112 4f908 clean
umem_alloc_128 4fb88 clean
umem_alloc_160 54008 clean
umem_alloc_192 54288 clean
umem_alloc_224 54508 clean
umem_alloc_256 54788 clean
umem_alloc_320 54a08 clean
umem_alloc_384 54c88 clean
umem_alloc_448 54f08 clean
umem_alloc_512 55188 clean
umem_alloc_640 55408 clean
umem_alloc_768 55688 clean
umem_alloc_896 55908 clean
umem_alloc_1152 55b88 clean
umem_alloc_1344 58008 clean
umem_alloc_1600 58288 clean
umem_alloc_2048 58508 clean
umem_alloc_2688 58788 clean
umem_alloc_4096 58a08 clean
umem_alloc_8192 58c88 clean
umem_alloc_12288 58f08 clean
umem_alloc_16384 59188 clean

/* Prints the status of the umem indicating if the logging features have been turned on or off*/
> ::umem_status
Status: ready and active
Concurrency: 4
Logs: transaction=128k
Message buffer:

> ::findleaks
CACHE LEAKED BUFCTL CALLER
0004e508 1 000cc4b0 char*functionName+0x94
----------------------------------------------------------------------
Total 1 buffer, 24 bytes

> 000cc4b0$ c7fc0/20X
0xc7fc0: 13 3a10bfed baddcafe baddcafe baddcabb
baddcafe feedface 12a2 cc4b0 a11c0c5d
baddcafe baddcafe baddcafe baddcafe baddcafe
baddcafe c8078 c6c10 5de00 4e508


/* Now Caliculate (gets after feedface 12a2 and divide by 251 and subtract 8 from the value, so 0x12a2 = 4770, 4770/251 = 19 , 19-8 = 11, So this tells we allocated some where 11 bytes and not released */

SO WE GOT THE NEAREST POINT OF THE MEMORY LEAK :)