Java memory allocation issues
This page is out of date
Recently an issue was discovered when trying to run a java with a large heap allocation.
In this case we were trying to allocate 2Gb to the java VM instance. Even when specifying large memory allocations in SGE java would always complain there was not enough space to allocate the heap.
A few people online have reported similar issues with SGE and java, it seems to stem from differences in the way java and SGE allocate memory to threads. Their solution is included on this page, but for more details see https://wiki.york.ac.uk/pages/viewpage.action?pageId=66126798
Firstly we need to set an enviromental variable, there is two ways to do this. Either set it in your SGE script, or as an environmental variable. The first is probably the best option so that it doesn't cause issues with other scripts.
Set as job variable
Add the following to your SGE script
#$ -v MALLOC_ARENA_MAX=1
Set as environmental variable
To do it in your .bashrc
Then add this to your SGE script
Limit GC threads
Java will spawn multiple GC threads which take up memory, limiting these can help
SGE Memory Allocation
To get the 2Gb heap to work I had to allocate 4Gb of Memory in sun grid engine
#$ -l h_vmem=4G