Difference between revisions of "Use scratch space"

From IBERS Bioinformatics and HPC Wiki
Jump to: navigation, search
(Created page with "Scratch space is fast access disk that is not backed up. It is located; /ibers/ernie/scratch/username where username is your username. It is here that you should have your...")
 
 
Line 15: Line 15:
 
Number 2) is probably the most important. Consider the following;
 
Number 2) is probably the most important. Consider the following;
  
If you're running a job that is creating lots of temporary files to your home directory, each night these files will be backed up. The backup server has a snapshot functionality which allows the retrival of past deleted files. If these temporary files that your job is creating are deleted often, it means that you might not see them, but they still exist on the backup server. This limits the ability to make lenghy backups of home directories on the HPC.
+
If you're running a job that is creating lots of temporary files to your home directory, each night these files will be backed up. The backup server has a snapshot functionality which allows the retrieval of past deleted files. If these temporary files that your job is creating are deleted often, it means that you might not see them, but they still exist on the backup server. This limits the ability to make lengthy backups of home directories on the HPC.

Latest revision as of 11:36, 23 August 2013

Scratch space is fast access disk that is not backed up.

It is located;

/ibers/ernie/scratch/username

where username is your username.

It is here that you should have your working data that you're reading from and writing to. This is important for two reasons;

1) It's faster

2) It doesn't get included in the nightly backup

Number 2) is probably the most important. Consider the following;

If you're running a job that is creating lots of temporary files to your home directory, each night these files will be backed up. The backup server has a snapshot functionality which allows the retrieval of past deleted files. If these temporary files that your job is creating are deleted often, it means that you might not see them, but they still exist on the backup server. This limits the ability to make lengthy backups of home directories on the HPC.