Complex submissions

In Submitting your job using SGE we looked at the following script;

   
#specify the shell type
#$ -S /bin/sh

#run in the current working directory
#$ -cwd

#specify that the job requires 40G of total memory and run for no more than 24hours.
#$ -l h_vmem=40G
#$ -l s_rt=360
#$ -l h_rt=3600

#specify exactly which node you wish to use (this isn't really recommended)
#the only time you may wish to use this is if there is specific hardware/software
#available on that node. In the case of Bert and Ernie, we don't have that.
#$ -l h=node006

#specify the number of cores required
#$ -pe mpich 1

#specify which queue you wish to use
#$ -q amd.q

#run a program command to print hostname and uptime
hostname && uptime
    

Most of this was not really required for the actual job (e.g. getting the hostname and uptime from each machine). However it is good practice to include limits for sun grid engine to work well.

Sun Grid Engine and Limits

Sun Grid Engine does not know what you're attempting to do until you tell it. There are three pieces of information that the scheduler needs to best load balance your job in the queue. It needs to know how much memory you need, how many CPU cores (also known as slots) and how long the job is expected to run for. This information is passed to the scheduler from your grid engine script. If you fail to specify the number of cpu cores, memory or time required, it will use the scheduler defaults, which may not be the best for your task.

Requesting Memory

Requesting memory is achieved by using the h_rt limit in your script, like so;

   
#$ -l h_vmem=40G
    

This will request 40G of RAM from the available nodes in the queue you've submitted it to. For information about the specification of the nodes available, please see Bert and Ernie - An Overview