Java memory allocation issues

This page is out of date

Recently an issue was discovered when trying to run a java with a large heap allocation.

In this case we were trying to allocate 2Gb to the java VM instance. Even when specifying large memory allocations in SGE java would always complain there was not enough space to allocate the heap.

A few people online have reported similar issues with SGE and java, it seems to stem from differences in the way java and SGE allocate memory to threads. Their solution is included on this page, but for more details see https://wiki.york.ac.uk/pages/viewpage.action?pageId=66126798

Set MALLOC_ARENA_MAX

Firstly we need to set an enviromental variable, there is two ways to do this. Either set it in your SGE script, or as an environmental variable. The first is probably the best option so that it doesn't cause issues with other scripts.

Set as job variable

Add the following to your SGE script

 #$ -v MALLOC_ARENA_MAX=1

Set as environmental variable

To do it in your .bashrc

 export MALLOC_ARENA_MAX=1

Then add this to your SGE script

 #$ -V

Limit GC threads

Java will spawn multiple GC threads which take up memory, limiting these can help

 -XX:ParallelGCThreads=1

SGE Memory Allocation

To get the 2Gb heap to work I had to allocate 4Gb of Memory in sun grid engine

 #$ -l h_vmem=4G