I/O Resources

Many users must run jobs that read datasets that are too large to transfer to the /scratch directory.   When large numbers of these jobs run simultaneously, it can have an adverse affect on the performance of the disks serving the data causing your jobs to run very inefficiently.   We have configured a resource in SGE that should help manage the amount of concurrent access.  These are called IOresources, and they depend on the bandwidth available from the storage hardware.

Submit your jobs as follows for files on the various elizas:

qsub -l eliza<#>io=1 <job script>

The <#> should be replaced with a number corresponding to the volume that contains your data.  For example, if your data resides on /eliza15, use eliza15io as your resource.  All PDSF volumes now have these resources as well as NGF (projectio) and HPSS (hpssio).  We recommend setting your resource to 1 as shown in the example above and if that turns out to be too low it can be raised. 

If you use /project there is also an io resource for that, used as follows:

qsub -l projectio=1 <job script>

And if you use both an eliza and /project you should specify both:

qsub -l projectio=1, eliza<#>io=1 <job script>

The following command can be used to see how many io units are assigned to your resource and how many are being used at any given time:

/usr/common/nsg/bin/getIOlimits.pl

SGE GlobalResource: Total hpssio = 400   	 Available=345    	   Used= 13.8 %
SGE GlobalResource: Total eliza7io = 200   	 Available=200    	   Used= 0.0 %
SGE GlobalResource: Total eliza4io = 250   	 Available=185    	   Used= 26.0 %
SGE GlobalResource: Total scpidl = 25   	 Available=25    	   Used= 0.0 %
SGE GlobalResource: Total eliza12io = 0   	 Available=0    	   Used= 100.0 %
SGE GlobalResource: Total eliza11io = 0   	 Available=0    	   Used= 100.0 %
SGE GlobalResource: Total projectio = 400   	 Available=306    	   Used= 23.5 %
SGE GlobalResource: Total eliza17io = 550   	 Available=319    	   Used= 42.0 %
SGE GlobalResource: Total eliza18io = 550   	 Available=485    	   Used= 11.8 %
SGE GlobalResource: Total eliza3io = 250   	 Available=228    	   Used= 8.8 %
SGE GlobalResource: Total eliza8io = 175   	 Available=175    	   Used= 0.0 %
SGE GlobalResource: Total eliza5io = 400   	 Available=400    	   Used= 0.0 %
SGE GlobalResource: Total eliza14io = 400   	 Available=162    	   Used= 59.5 %
SGE GlobalResource: Total eliza16io = 300   	 Available=300    	   Used= 0.0 %
SGE GlobalResource: Total starhpssio = 15   	 Available=15    	   Used= 0.0 %
SGE GlobalResource: Total eliza2io = 200   	 Available=135    	   Used= 32.5 %
SGE GlobalResource: Total eliza13io = 0   	 Available=0    	   Used= 100.0 %
SGE GlobalResource: Total eliza6io = 400   	 Available=277    	   Used= 30.8 %
SGE GlobalResource: Total eliza1io = 250   	 Available=185    	   Used= 26.0 %
SGE GlobalResource: Total eliza10io = 100   	 Available=91    	   Used= 9.0 %
SGE GlobalResource: Total eliza15io = 400   	 Available=334    	   Used= 16.5 %
SGE GlobalResource: Total snapidl = 768   	 Available=768    	   Used= 0.0 %
SGE GlobalResource: Total eliza9io = 200   	 Available=118    	   Used= 41.0 %