Storage Resource Unit (SRU) Formula Coefficients

The coefficients in the Storage Resource Unit (SRU) formula were arrived at from the following considerations:

• The formula should help influence user behavior towards efficient use of the storage resource.
• The formula should reflect the relative costs of "doing business".

From these considerations, we adopted 1) file counts, 2) bytes stored, and 3) I/O transfers as the minimum factors that needed to be included. Hardware costs are related to these three areas in the following ways:

1. Costs driven by number of files:
   • Metadata CPUs, disks and backup systems
   • Additional tape drives required to overcome the latency of small file sizes
2. Costs driven by the amount of space used:
   • Library
   • Media
   • Repack CPUs and drives
3. Costs driven by bandwidth requirements:
   • Multiple tape drives
   • Large capacity, high speed disks
   • High speed networking and network switches
   • Data transfer CPUs ("movers")

We considered NERSC's costs of storage operation and roughly assigned them to these three areas. Ignoring file counts, we found that storage and I/O accounted for roughly 40% and 60% of our costs. We decided that storage and I/O would have these rough proportions, and that we would introduce file counts to be 10% to 20% of the overall formula.

At the time, the cost of monitoring space usage was very high, requiring a complete directory listing. Therefore, we limited these operations to one or two per month. This led to an accounting granularity of one month. Based on this granularity, the overall coefficient in the formula were set to generate 1 SRU per month for the average user with 1 GB stored in the system.

The formula

At the time the formula was set (1999), there were about 20 to 50 TB in NERSC storage. The amount of I/O, on a monthly basis, was observed to be about 10% of the amount stored (as of 2003 it is 13% to 14%). So, the formula was

   SRUs = (GB stored) + 10 x (GB of I/O)  

The average file size was about 10 MB, so there were about 100 files per GB stored. To cause this to account for 1/4 as much as the space charge, the factor times 100 must equal about 0.25. This leads to a file factor of 0.0025. This was subsequently raised slightly to 0.003 to have more influence on users. So the hypothetical user, who stores 1 GB in 100 files, and does 0.12 GB of I/O per month sees the following charges:

    no. of files * .003 = 0.3
    space stored        = 1.0
    I/O * 10            = 1.2
                   TOTAL  2.5

Finally, it was decided that it would be convenient if the "average" user, with 1 GB stored would accrue 1 SRU (per month), so the above rates were scaled by 0.4, leading to

   monthly user SRUs = 0.0012 * files + 0.4 * GB_stored + 4 * GB_I/O

Later, when the accounting granularity became daily, the above rates were divided by 30.5 days per month (366 days per year) to get

   daily user SRUs  =  0.0000393 x files  
		       +  0.0131147 x space (GB)  
		       +  4.0 x I/O (GB)

Multiplying by a typical 365 day year, we get:

   yearly user SRUs  =  0.01436 x Avg files  
		        +  4.787 x Avg space (GB)  
                        +  4.0 x I/O (GB)