NERSCPowering Scientific Discovery Since 1974

Q & A

July 15, 2013

Question/Issue 1

What is the more precise meaning of “On-site System Delivery and Build Complete” for Trinity by Q3CY15 and NERSC 8 by Q4CY15 as referenced on page 6 within the Trinity-NERSC-8-Draft technical requirements document?

Project Response 1

These dates indicate when the systems are to be completely delivered and bootable. The actual subcontract schedules will be negotiated, but Offerors should propose based on the schedule within the Technical Requirements Document.

Question/Issue 2

Capacity benchmarks – In the Trinity and NERSC-8 Draft Technical Requirements document, paragraphs 3.5.4 and 3.5.5, it states these benchmarks are to be used AT ACCEPTANCE. Does this mean Offerors do not need to provide information on the Capacity benchmarks for the proposal?

Project Response 2

3.5.4 and 3.5.5 are the Capability Improvement requirements, not Capacity. Offerors do not have to provide Capability Improvement results for individual applications at RFP response time, but they do have to state what improvement factor they will achieve. The Capability Improvement will be measured at acceptance.  Please refer to the following website for more information: http://www.nersc.gov/systems/trinity-nersc-8-rfp/trinity-nersc-8-capability-improvement/

Question/Issue 3

Contradiction in IOR Run instructions; The ‘How to Run’ section states that changes may only be made to the following four parameters when running IOR; [testFile, hintsFileName, collective, segmentCount].

However, the Required Runs section states: The vendor shall determine the transfer size to be used in the tests.

Please clarify.

Project Response 3

An error existed in the wording of the IOR instructions. The second paragraph under Required Runs should read: "The vendor shall determine the file size to be used in the tests." See the description of file size in the "How to Run" section. This change will be updated in the instructions on the benchmarking website. http://www.nersc.gov/systems/trinity-nersc-8-rfp/draft-nersc-8-trinity-benchmarks/

Question/Issue 4

The benchmark instructions and benchmark results template indicate that the small benchmarks must run on one node. But the instructions within the benchmark codes require numbers of MPI ranks which are not practical on a single node – for example, 96 MPI ranks for AMG, 64 for GTC.

Please clarify how the small benchmarks are to be run, for 1) MPI-Only and 2) MPI+X.

Project Response 4

The number of MPI ranks reported in the run rules and the spreadsheet correspond to how the baseline data was collected on NERSC's Hopper platform. The number of MPI ranks for the Trinity/NERSC8 projections can be varied to suit the proposed architecture. The intent is to report the best performance using the optimal decomposition. This applies to all problem sizes, MPI-only and MPI+X reporting. If the small problem does not fit on the proposed node due to memory limitations, 2 or more nodes may be used. The large problem shall be weak scaled based on the node count used for the small problem. For example, using Hopper the large problem for the miniFE used 512 times more nodes than the small problem (2048/4).

Question/Issue 5

In the June 13 update to the SNAP benchmark information (located at http://www.nersc.gov/systems/trinity-nersc-8-rfp/draft-nersc-8-trinity-benchmarks/snap/), several details of the problem description have changed significantly from the previous version. The new README requires a weakly scaling results for both the small and large cases. The examples used to describe scaling the physical grid (nx, ny, nz) and process grid (npey, npez) are straightforward, i.e. multiply the grid dimensions by 2 in a round robin fashion when increasing the MPI task count by 2 (at least for factors of 2 counts). However, for the large case, the 'ichunk' parameter is also changed with increasing task count, but it is not clear how this parameter should be changed for intermediate values not shown (e.g. 32-1024, 4096-16384 tasks). Currently, the assumption is the ichunk parameter should be doubled from previous value when nx = 64, or 256, e.g. ichunk=16 for 64 <= nx < 256, and 256 <= nx. Please clarify how one should modify the ichunk parameter or if it is a tuning parameter subject to vendor's discretion.

Project Response 5

The intent for providing weak scaling examples is to show how to vary the parameters, the "small", "large" problems are the only two results required for reporting. You are allowed to change the decomposition to fit your architecture, but for these two problems the number of cells and other parameters that affect the "size" of the problem (e.g. number of groups and number of angles) must stay the same.

We use a 2-D spatial decomposition over the y and z directions. "ichunk" is the number of x-planes swept before communication, and can be considered a tuning parameter, the number we give is just a suggestion. If it is changed, it should be called out specifically.

Question/Issue 6

Are Offerors anticipated to provide storage for home directories as part of the responses? If so, what capacity and performance characteristics are desired for this file system? 

Project Response 6

Both the Trinity and NERSC-8 platforms do not require storage to be provided for anything other than the parallel file system, which is nominally used as /scratch. The specified external network requirements are meant to provide connections to ACES and NERSC provided storage using NFS for file systems such as /home.

Question/Issue 7

Will it be sufficient for Offerors to reference specific, controlled documents currently available under existing NDA agreements in proposals, or will the contents of those documents also need to be provided?

Project Response 7

All information necessary to support the Offeror's proposal should be included in the RFP response.

Question/Issue 8

Can portions of Offeror's responses be marked as "Vendor Confidential"?

Project Response 8

Yes. There will be more specific instructions in the RFP when it is issued.

June 20, 2013 

Question/Issue 1 

Requirement 3.4.4 specifies that the Offeror provide the time to perform the metadata operations "enumerate and retrieve" on one million objects. However, the "Required Runs" section of the mdtest web page,http://www.nersc.gov/systems/trinity-nersc-8-rfp/draft-nersc-8-trinity-benchmarks/mdtest/ specifies that only the times to perform creation and removal of objects be provided. Should we abide by the web page? 

Project Response 1 

The mdtest benchmark is not to be used for the "enumerate and retrieve" portion of 3.4.4. We anticipate using another method and/or benchmark to test this at acceptance. You do NOT need to address this aspect of the requirement at proposal time. 

Question/Issue 2 

The "Required Runs" section of the mdtest benchmark web page describes the syntax for an mdtest run that will cause N processes to operate on a single file:

aprun -n <#pes> N <#pes-per-node> ./mdtest -S -C -T -r -n 1 -d <path-to-pfs>/<n1_shared-dir>/ -F

However, when running with this command line, it appears to operate on 1 file per process, not a single file: 

mdtest-1.8.3 was launched with 256 total task(s) on 8 nodes Command line used: ./mdtest -S -C -T -r -n 1 -d /lus/scratch/u3186/mdtest/mdtest-1.8.4/4121812.sdb/test4 -F Path: /lus/nid00030/u3186/mdtest/mdtest-1.8.4/4121812.sdb FS: 25.8 TiB Used FS: 20.8% Inodes: 87.2 Mi Used Inodes: 2.4%
256 tasks, 256 files 

Project Response 2 

Mdtest does report writing to N files on stdout, but if you check the actual directory for the resultant file you should find a single file. We confirmed this on our Cielo test bed.

Question/Issue 3 

Note that the mdtest.c file in the 1.8.4 tar file contains the string: #define RELEASE_VERS "1.8.3" Can we get confirmation that the mdtest.c in the tar file is the correct version? 

Project Response 3 

The version of mdtest is 1.8.4, the release string was not updated. We'll correct that in the next distribution. 

Question/Issue 4 

Is the target SSP increase over Hopper (10x-30x for NERSC-8, 20x-60x for Trinity) using the optimized MPI+X SSP or base case (MPI-only) SSP? 

Project Response 4 

The Optimized MPI+X Case 

Question/Issue 5 

Is there an example calculation for the Capability Improvement metric? 

Project Response 5 

An example calculation for the Capability Improvement metric has been provided for the three NERSC-8 applications on the NERSC Capability Improvement web page

Question/Issue 6 

We are thinking of proposing a system with two different node types. In our response do we need to include benchmark times and an SSP calculation for both node types? How is the SSP calculated for a system with more than one type of node? 

Project Response 6 

Yes, your response should include benchmark times and an SSP calculation for both types of nodes. 

We have provided an example SSP calculation for a system with two different kinds of nodes via a spreadsheet that can be downloaded on the SSP web page. This example shows that the SSP is calculated using all benchmarks and across all node types. The resulting SSP is the sum of the SSPs for each node type. 

Question/Issue 7 

The RFP Technical Specifications specifies some benchmarks for "RFP Response" and others for "Acceptance". What does this mean? 

Project Response 7 

Results of benchmarks labeled “RFP Response” should be submitted as part of the RFP response. Results of benchmarks labeled “Acceptance” must be provided by the selected vendor at the start of negotiations for inclusion in the Statement of Work. 

Question/Issue 8 

In reference to Question and Answer Set 1, Project Response 12, it appears that Moab is mandatory to be used as the only scheduler for all Trinity systems. Some of the more advanced capabilities from the RFP such as controlling the power use of the cluster, burst buffer operation, or system scale rely on functionality of the scheduler. 

1. Please confirm if Moab is required as the scheduler for the clusters or can an alternate be proposed.

2. If Moab is mandatory, how will enhancements for the complex scheduling requirements needed for advanced power management and burst buffer data management be implemented? 

Project Response 8 

Since the requirement for Moab is a Target Requirement, it is not mandatory that Moab be the scheduler. Per the definition provided for a Target Requirement, failure to meet a Target Design requirement does NOT make the proposal non-responsive. However, if a requirement cannot be met the Offeror should provide the technical rational and it is desirable that they propose an alternative solution. 

Question/Issue 9 

Please clarify in-situ visualization. What is the role of the visualization nodes with in-situ visualization? For example, are they running the simulation as well as the visualization/analysis? What is meant by “requires use of the main compute resources”? 

Project Response 9 

In-situ visualization and/or data analysis can take different forms, but in general, it requires that at least 2 jobs running on the system are able to communicate with each other over the high speed interconnect. It could be part of the visualization analysis is being done on the compute nodes and some reduced form of the data is being sent to the visualization nodes. It may also be the simulation and all of the in-situ visualization analysis is being done on the compute nodes, with viz and analysis dumps written to the file system as needed. 

Question/Issue 10 

How is does Section of the Draft Technical Requirements 4.1 workload #4 “Analysis of large ensembles of data” different from #1 and #3? Is this analysis that does not include visualization? 

Project Response 10 

To clarify, 4.1 #1, #3 and #4 could be non-viz data analysis. Number 4 is a special case of 1 and 3, called out because of the parallelism involved, and also because ensembles are a Sandia use case for simulation. 

Question/Issue 11 

In Section 4.1, what are the networking requirements for remote display? What bandwidth and latency is sufficient? What access is needed from the remote display, for example, do the visualization nodes need to have IP addresses on the same network as the remote displays? Figure A1 in Appendix A shows a 10 GB/s connection between the Visualization partition and DISCOM (Distance Computing WAN) at LANL. Does that imply that the visualization nodes share a 10 GB/s connection to the network outside the cluster? 

Project Response 11 

Yes, the 10 GB/s connection shown in Figure A1 is for the visualization nodes. We have not specified a latency requirement because it is dependent on the LANL external network configuration. The visualization nodes do need to be able to access an IP address off of the platform. 

Question/Issue 12 

Does the customer have information about the roadmaps of these visualization packages that they can share? Does the statement that ACES and NERSC will provide porting support indicate that there is ACES and NERSC will provide support to adapt these packages to the upcoming technologies? 

Project Response 12 

The ports are expected to be done by the respective visualization software vendor. We need to know that these packages will work on the given architecture. We expect the System vendor to provide porting assistance to the software vendors. We also expect that the System vendor will have discussed these issues with the software vendors and will be able to provide some level of assurance that the software packages will work on the proposed architecture. We recommend the following contacts: 

ParaView: Berk Geveci, berk.geveci@kitware.com
VisIt: Dave Pugmire (ORNL), pugmire@ornl.gov
EnSight: Anders Grimsrud (CEI), ang@ensight.com 

Question/Issue 13 

If the visualization nodes are the same architecture as the compute nodes then this is straightforward. If they are different, then how do we figure 5%? 

Project Response 13 

If the visualization nodes are of a different architecture, the baseline proposal should be sized for 5% of the main compute partitions aggregate memory. For example. if the aggregate memory of the main compute partition is 2 PB, the visualization partition shall provide 0.1 PB. In the requirements, we also ask for the option of doubling the memory capacity, this would also apply to the visualization partition. 

Question/Issue 14 

In Section 4.1, #1, a possible interpretation is GPUs are not essential for visualization nodes. 

Project Response 14 

GPUs can help rendering, but the visualization and data analysis needs are much greater than just rendering. These nodes require high performance, general-purpose capabilities. 

Question/Issue 15 

One of the requirements in the Draft RFP that has potential significant impact on system design (HW and SW) is the Max Power Rate of Change: 

"The hourly average in platform power should not exceed the 2MW wide power band negotiated at least 2 hours in advance." 

Would it be possible for you to elaborate on this requirement? How do you envision the negotiated parameter to be presented to the system? How often will the negotiated power band be changed? Is the hourly average arollinghour or delineated by wall clock hours? Is the system really permitted to reach the peak power consumption (15 MW?) within the hourly window if the power band is significantly lower than that? 

Project Response 15 

Please recognize that advanced power management is one of the areas targeted for further development for DoE’s Advanced Technology systems. 

Industrial electricity isn't priced by kW*hr. Other components of the pricing formula include demand charge (basically, maximum power demand measured by some contractually specified formula), power factor, and other variables designed to estimate how hard is it for the utility to deliver the electricity. These factors are specified in the rate charts for industrial electricity and usually approved by relevant regulatory commissions. For example in NM, it may be that variable electricity costs about 65% more to deliver than steady electricity (a rough estimate based on a single data point for NM and not by a proper study of national averages). There are power fluctuation constraints arising from data center cooling infrastructure as well (e.g. sequencing of chillers/pumps/cooling towers).

More specific to the situation at Los Alamos is the pricing formula we currently have. We are supposed to stay within the power band negotiated in advance. Details of this may change when power contracts change, but basically they are designed to capture the power grid's costs of producing and delivering power. Currently, demand charge is computed based on hourly averages, i.e. not rolling averages. Those hourly averages should stay within the agreed upon power band, or else these things happen: 

* Above the power band, must pay the electrical utility provider emergency power rates, or buy power on the spot market (either way, power is much more expensive) 

* Below the power band, must pay for unused power or sell it on the spot market (thus dramatically reducing the cost benefit of energy savings) 

* If peak hourly average power occurs in the wrong hour of the month, that extra MW could cost over 300 times more than normal; that normally means that mid-day peak hourly power should be watched carefully 

We are looking to minimize operatingcosts, so power variability has to be considered. We hope that there is an opportunity to reduce the operating cost impact of power variability through advance warnings, which would allow site electrical utility people to trade energy on the spot market and thus avoid extra costs of emergency power rates or unused power. This normally implies at least an hour (preferably two) advance notice of major changes in power demand. The specific formula we have now has hourly averages, but this may change depending on terms in future power contracts. We'll be looking for advances in power management which can make large platforms behave as a good citizen on the power grid, where "good" is defined by cost incentives pushed to us by the power companies through rate structure we've got to pay. Our expectation is that for all data centers, power companies will find ways to pass on the cost of delivering variable power; that those costs may add about 65% to the cost of steady power (per the NM example rate formula), and that any energy saving technique which converts steady power to visibly variable power (meaning larger than something the power company can ignore) may have to achieve more than 65% energy efficiency improvement in order to be cost justified. 

Grossly simplified, just think of variable power costing substantially more than steady power; even on hour-by-hour average basis. 

Question/Issue 16 

In the Draft benchmark run rules posted at
http://www.nersc.gov/assets/Trinity--NERSC-8-RFP/Documents/N8BmkInstructJune13.pdf,
five of the benchmarks have an extra-large dataset defined. Can you provide scaling data from the reference platform, Hopper, for these data sets? 

Project Response 16 

We are not providing scaling data for the applications specified in Table 2 of the document, only the reference values used for the SSP calculation.

June 7, 2013 

Question / Issue 1 

Respondents to the RFI have expressed a concern regarding the sharing of proprietary and/or intellectual property (IP) information with Offerors of fully integrated systems. 

Project Response 1 

RFP language will require that each proposal be an offer for complete system integration for two separate systems, Trinity and NERSC-8. RFP language will allow the direct submission to the LANS Procurement Specialist of proprietary and/or IP information that is a supplement to an offer for complete system integration. The Instructions to Offerors will provide guidance for the submittal of such supplemental proprietary and/or IP information. Each Offeror for full system integration will be responsible for timely submission of the supplemental information and for each supplement’s clear reference to the offer that is being supplemented. Submissions of supplemental information must comply with proposal preparation instructions (i.e. format and page limitation requirements) and LANS will assume no responsibility/liability for any failure to comply with the instructions. 

Question / Issue 2  

Processor technology is controlled by the processor suppliers. If projected performance isn’t realized, how is the system vendor protected? 

Project Response 2

Subcontracts will be executed with the system vendor, presumably with the processor technology suppliers as lowertier suppliers to that system vendor. It is the responsibility of the system vendor to determine reliable sustained system performance and to propose accordingly. Any failure to meet proposed performance by the successful offeror is subject to usual contractual provisions, which will be stated in the RFP. 

Question / Issue 3 

Please provide more information on NRE funding availability for Trinity. Primarily provide how much NRE funding will be available and what rules may apply for those seeking it.

Project Response 3

We are not specifying how NRE will be funded or how much NRE will be available. It depends on the proposals and their value add to the projects.  

Question / Issue 4

Is there any information available on what the acceptance process would entail? 

Project Response 4 

Information on the acceptance process has been posted with the DRAFT Technical Requirements. 

Question / Issue 5

Extrapolating benchmark numbers from a smaller system to a system of this magnitude has many unknowns. Also, the technology is some number of months out, further complicating the estimates.  

Project Response 5 

Providing benchmark estimates has always been a challenge. However, we have used this process for most of the past ASC systems in the last 15 years. We have asked for risk mitigation for specific components such as processors in case the estimated performance doesn't measure up. 

Question / Issue 6

The recent benchmark instructions and application README's state that the 'Base case' shall be run with "no OpenMP" threads or "any existing APIs in the codes which exploit additional parallelism, e.g. OpenMP, may not be enabled". However, I have not been able to build the provided SNAP, or MILC codes without using the OpenMP enabling compiler flag. I can provide details of the code why this is the case, but just to be safe should the documents say something like "if OpenMP is enabled only 1 thread may be specified"? 

Project Response 6 

Yes, it is acceptable to run with the OpenMP enabled version and use OMP_NUM_THREADS=1 for the MPI-only base case. 

Question / Issue 7

In the May06 update to the README and web page there is a section that states : “Required Runs We require the weak scaling results, where the problem size per processor is constant regardless of the number of processors(sockets) with 1MPI rank per core, no OpenMP threads. Focus on 16x16x16 per processor on hopper. In *addition* to the above you can adjust the # MPI ranks and OpenMP threads that give the best performance for your architecture and report the configuration and results.” 

Does the customer expect the offerer to provide weakly scaling results i.e. modifying ny and nz proportional to npey and npez? Can the customer clarify?  

Project Response 7

The description provided on the SNAP benchmarking web page describes how the baseline results were collected on NERSC's Hopper system, and how to modify parameters to best fit the vendors architecture. The vendor is allowed to run the required problem sizes at a different number of MPI ranks while preserving the total number of cells. The run rules for SNAP will be updated to provide more clarity.

Question / Issue 8

MILC 

On the web site and in the documentation the problem size is described as follows. "The single node case is designed to have 8x8x8x8 sites per MPI task and is sized to fit on a current node of NERSC's hopper system (24 core/node Cray XE6). The large test case has the same number of sites per core but is sized to fit on 1024 nodes of hopper." Suggesting that the inputs should are scaled weakly based on a 8x8x8x8 site lattice per MPI task. This is consistent with all the sample input files and output files. In the "Capability Improvement Runs" section it says that "For the 24,576MPI rank large problem, the size of the fourdimensional spacetime lattice is controlled by the following parameters in the input deck: 

nx 64 
ny 64 
nz 128 
nt 192" 
... 
In general, to weak scale a 4x4x4x4 (nx x ny x ny x nt) problem, one begins by multiplying nt by 2.. " 

Suggesting that a 4x4x4x4 base lattice size should be used, but the parameters above is consistent with the 8x8x8x8 size lattice per MPI task. There seems to be some inconsistencies between the different statements and it would help if they could be reconciled or clarified.  

Project Response 8

The 4x4x4x4 lattice size in the Capability Improvement description was meant to be an example. For the Capability Improvement metric, it is not a requirement to weak scale the "large" problem defined for the SSP metric. The vendor is allowed to choose the parameters that best fit their architecture.  The run rules for MILC will be updated to provide more clarity. 

Question / Issue 9 

UMT (The following is also relevant to AMG) 

With the instructions in the new README file we have been able to run UMT with different MPI rank counts while keeping the problem size constant. As for the variation in the iteration counts, we have recently discovered the same phenomenon is at work for AMG. The number of iterations changes with both MPI count and with the compiler used, with different compilers having the lowest iteration count at different processor counts. However, it does appear that the time per iteration is rather constant. It seems that it is important for the purposes of this benchmark and specifically the SSP calculation that the number of iterations be a constant. 

The current SSP calculation does not take into account a change in convergence behavior due to a change in problem size; it is based on a reference time and a reference, fixed FLOP count. So while we agree it is possible and very useful for end users to examine the time per iteration, that is currently not part of the SSP calculation. Since we don’t actually know how many iterations were completed during the reference runs, it is not clear to us how to project to a final time, which is dependent on the number of iterations. Furthermore it is not clear to us that we can guarantee that we will be executing the same number of iterations on a future machine with different processors and compilers. 

Perhaps an alternative would be to base the SSP calculation off of a Tflops/iteration and a time/iteration. That way the program can be allowed to converge but timings and SSP calculations can account for variations in the number of iterations. Please advise on how we should handle this situation.  

Project Response 9

The Benchmarking Run Rules document and README files have been updated to account for the differences in the number of iterations to convergence. Make sure to provide the number of iterations along with the benchmark time.

Question / Issue 10

We have observed that in the subroutine src/Teton/transport/Teton/snac/snflwxyz.F90 arrays are allocated by each MPI rank to be used in subsequent calls to calculate fluxes. Most of the computational work is in this section. Below are the allocate statements we are referring to: 

allocate( omega(ndim,n_cpuL) ) 
allocate( abdym(nbelem,n_cpuL) ) 
allocate( siginv(npart,ncornr) ) }
allocate( sigvol(npart,ncornr) ) 
allocate( tphic(npart,ncornr,n_cpuL) ) 
allocate( tpsic(npart,ncornr,n_cpuL) ) 
allocate( qc(npart,ncornr,n_cpuL) ) 
allocate( psifp(npart,maxcf,ncornr,n_cpuL) ) 
allocate( source(npart,ncornr,n_cpuL) ) 

By our calculation, these allocates would total about 20 Mbytes of data per MPI rank on average, assuming 1 OpenMP thread, and 36 Mbytes of data per rank assuming 2 OMP threads. We have two questions regarding the size of these arrays: 

a) Is there an input parameter than can control and change the size of ncornr while keeping the global problem size and science problem fixed? If one exists that would allow one to impact the total size of those allocates per rank

b) Conversely, do the size of ncornr and npart change depending on the science problem? If yes, are they typically larger or smaller than the benchmark problem?  

Project Response 10 

a) No, ncornr is not a controllable parameter and is not to be changed. 

b) Ncornr and npart are independent of the size of the problem, i.e., number of zones. 

Question / Issue 11  

Our benchmarking center isn't equipped with systems that large. Can you provide options for benchmarks on smaller subsets? 

Project Response 11

The draft technical requirements we posted in December stated that performance benchmarks, application and microbenchmarks, can be "actual, predicted and/or extrapolated". In addition, the benchmarking run rules posted on the web site provide further guidance under the "Submission Guidelines" section for performance projections (predicted and/or extrapolated). With a 2015 delivery, we expect performance projections are necessary as many key technologies are not available at this time.  

Question / Issue 12 

The 1st draft of the RFP Technical Specifications released in December 2012 stated the system shall provide resource management functionality including checkpoint/restart, Moab Compatible. 

a) Does this mean application level checkpoint restart, not system C/R support? 

b) What does it mean for Moab compatible? Does it mean Moab as scheduler, vendor provided resource management and job launching system? Or Moab only plays as meta scheduler to forward job to the vendor scheduler. Please clarify this.  

Project Response 12

This requirement has been revised in the second draft of the RFP Technical Specifications released June 3, 2013 and the language for checkpoint/restart has been removed. For Moab capability, the requirement is to provide Moab as the scheduler, with a vendor provided resource manager and job launching system. 

Question / Issue 13

The draft RFP Technical Specifications state that a job interrupt shall not require a complete resource re-allocation. Does this mean if one of hosts fails during job run time, system will allocate a new resource to replace the failure one for a running job? Please help clarify this. 

Project Response 13

The requirement intent is for the job to not have to resubmit to the scheduler, and hence wait in the queue, to allocate resources to continue. 

Question / Issue 14

The draft RFP technical specifications state a complete system initialization shall take no more than 30 minutes. How will this measured and please indicate what constitutes the start and stop time? 

Project Response 14

The requirement is to describe the sytem’s "full system initialize sequence and timings." How it is measured is system dependent and the method will be determined in the statement of work. 

Question / Issue 15

What Globus toolkit version do you plan to use? GT4 supports our LSF product. However, GT5 changed the interface and now it does not work with LSF. As GT is a community program, historically it is has been the community who does integration work. While [we] can try to influence the the community, we cannot guarantee what actions will be taken. 

Project Response 15

There is not a requirement for a specific version of the Globus toolkit. Please describe what version you will support. 

Question / Issue 16

The draft RFP technical specifications state job launch in 30 seconds. How will this be measured and please indicate what constitutes the start and stop time?  

Project Response 16 

See Table 2 in the RFP Technical Requirements.  

Question/Issue 17 

Question: How will I/O performance be measured? 

Project Response 17 

See Table 2 in the technical requirements. 

Question/Issue 18

The 1st draft of the RFP Technical Specifications released in December 2012 stated JMTTI/DELTA > 30 and also stated that the JMTTI > 30, from which it follows that DELTA must be less than 30/30 = 1 (hour). However, Table 4 gives values of 20 and 35 minutes. And the Burst Buffer Use Scenarios gave significantly different values as well. Please clarify the IO bandwidth performance requirement.

Project Response 18

The PFS needs to be sized based on the performance requirements specified in Table 4. Delta for the PFS and Burst Buffer are separate requirements. This has been clarified in the second draft of the RFP Technical Specifications released June 3, 2013.  

Question/Issue 19

Please clarify the External and Local client requirements in Table 4. For example, are these to be provided via nfs? Are these external and local clients on known networks that we need to bridge to? Are you requesting Gateway nodes? 

Project Response 19

The external bandwidth requirements have been further clarified in the second draft of the RFP Technical Specifications released June 3, 2013. In particular, a facilities interface diagram has been provided in the appendix.