Back to Survey Results

FY 2003 User Survey Results

Many thanks to the 326 users who responded to this year's User Survey -- this represents the highest response level yet in the six years we have conducted the survey. The respondents represent all five DOE Science Offices and a variety of home institutions: see Respondent Demographics.

The survey responses provide feedback about every aspect of NERSC's operation, help us judge the quality of our services, give DOE information on how well NERSC is doing, and point us to areas we can improve. The survey results are listed below.

You can see the FY 2003 User Survey text, in which users rated us on a 7-point satisfaction scale. Some areas were also rated on a 3-point importance scale or a 3-point usefulness scale.

Satisfaction Score Meaning 7 Very Satisfied 6 Mostly Satisfied 5 Somewhat Satisfied 4 Neutral 3 Somewhat Dissatisfied 2 Mostly Dissatisfied 1 Very Dissatisfied	Importance Score Meaning 3 Very Important 2 Somewhat Important 1 Not Important
	Usefulness Score Meaning 3 Very Useful 2 Somewhat Useful 1 Not at All Useful

The average satisfaction scores from this year's survey ranged from a high of 6.61 (very satisfied) to a low of 4.67 (somewhat satisfied). See All Satisfaction Questions. Areas with the highest user satisfaction were:

Topic	Avg Score	No. of Responses
HPSS reliability	6.61	126
Consulting - timely response	6.55	207
Consulting - technical advice	6.54	200
HPSS uptime	6.54	126
Local Area Network	6.54	114

Areas with the lowest user satisfaction were:

Topic	Avg Score	No. of Responses
Access Grid classes	4.67	27
Escher visualization software	4.75	8
Visualization services	4.81	97
NERSC training classes	4.88	24
Training	5.04	94

The largest increases in satisfaction over last year's survey came from the IBM SP, Seaborg, HPSS uptime, network connectivity, and available hardware:

Topic	Avg Score	Increase from 2002	No. of Responses
SP Applications	6.00	0.30	94
SP Libraries	6.27	0.18	131
SP Disk Configuration and I/O Performance	6.15	0.18	156
HPSS Uptime	6.54	0.17	126
Network Connectivity	6.23	0.16	241
Available Hardware	6.13	0.16	255

The areas rated significantly lower this year were:

Topic	Avg Score	Decrease from 2002	No. of Responses
PDSF Fortran Compilers	6.03	-0.42	29
PDSF Ability to Run Interactively	5.77	-0.41	64
PDSF Applications	5.87	-0.34	39
SP Queue Structure	5.69	-0.23	177
SP Uptime	6.42	-0.14	191

Survey Results Lead to Changes at NERSC

Every year we institute changes based on the survey. NERSC took a number of actions in response to suggestions from the 2002 user survey.

SP resource scheduling:

Could longer run time limits be implemented across the board?

NERSC response: In March 2003 limits were extended from 8 to 48 hours for jobs running on 32 or more nodes, and from 8 to 12 hours for jobs run on 31 or fewer nodes. The "regular long" class, which provides a 24 hour limit for jobs run on 31 or fewer nodes, was preserved but with restrictions on the number of jobs that can run simultaneously.

Could more services be devoted to interactive jobs?

NERSC response: In March 2003 interactive jobs were given an additional system priority boost (placing them ahead of debug jobs).

Could there be a serial queue?

NERSC response: Two new classes to facilitate pre-and-post data processing and data transfers to HPSS were introduced in November, 2003. Jobs run in these classes are charged for one processor's wall clock time.

Could more resources be devoted to the "small node-long runtime" class (more nodes, a longer run time, better throughput)?

NERSC response: Resources were not increased for "regular long" types of jobs; rather the priority has been to increase resources for jobs running on more than 32 nodes. This is in line with the DOE Office of Science's goal that 1/4 of all batch resources be applied to jobs that use 1/8 of the available processors. For FY 2004 this goal has been increased to target 1/2 of the batch resources. Perhaps because of this resource prioritization, satisfaction with the SP queue structure dropped by 0.2 points.

SP software enhancements:

Could the Unix environment be more user-friendly (e.g. more editors and shells in the default path)?

NERSC response: The most recent versions of vim, nano, nedit, gvim, pico, xemacs are now in in all users' paths by default, as well as the compression utilities zip and bunzip2. Two new utilities help make the batch environment easier to use: llhist shows recently completed jobs and ll_check_script gives warnings/advice on crafting batch scripts. This year's rating for SP applications went up by 0.3 points..

Could there be more data analysis software, including matlab?

NERSC response: Matlab and Mathematica are available on the math server, newton. Matlab is not available on the IBM SP because big Matlab jobs can severely affect other users on the interactive nodes. The IDL (Interactive Data Language) package is available on Seaborg for interactive data analysis and visualization of data.

Computing resources:

NERSC needs more computational power overall.
Could a vector resource be provided?
Could mid-range computing or cluster resources be provided?

NERSC response: All the above are excellent suggestions and we certainly understand the desire for more computational resources. The FY 2004 Seaborg allocation requests were for 2.4 times the amount available to allocate. The reality is that there is no budget for additional hardware acquisitions. Last year we were able to double the number of nodes on Seaborg and this year's rating for available computing hardware increased by 0.2 points.

Documentation:

Provide better searching, navigation, organization of the information.

NERSC response: The NERSC user web site (http://www.nersc.gov/nusers/) has been restructured with new navigation links that should make finding information faster and easier. Related information has been consolidated. Printer-friendly links have been added to consolidate multi-page documents into a single one. The final phase of the update will be to encode descriptions for each page to increase the effectiveness of the search engine.

Enhance SP documentation.

NERSC response: We have made an effort to keep up-to-date on a wide range of SP topics: IBM compilers, the LoadLeveler batch system, IBM SP specific APIs, and links to IBM redbooks. In addition the presentation of SP information has been streamlined; hopefully information is easier to find now. In August 2003 we received positive comments from ScicomP 8 attendees in regard to how we present IBM documentation.

Training

Provide more training on performance analysis, optimization and debugging.

NERSC response: Since last year's survey NERSC has emphasized these topics in our training classes, for example: CPU performance analysis on Seaborg, Scaling I/O and Communication, Debugging Parallel Programs with Totalview. See http://www.nersc.gov/nusers/services/training/.

Provide more information in the New Users Guide.

NERSC response: More information on initial account setup was added to the New User Guide, which was also reformatted for ease of use. See http://www.nersc.gov/nusers/help/new_user/.

This year's survey included several new questions:

1. How useful were the DOE and NERSC scaling initiatives? [Read the Scaling Initiatives Response Page]

   In FY 2003 NERSC implemented initiatives aimed at promoting highly scalable applications as part of the DOE emphasis on large scale computing. For the first time, DOE had in FY 2003 an explicit goal that "25% of the usage will be accounted for by computations that require at least 1/8 of the total [compute] resource." (Note: for FY 2004 this goal is for 50% of the usage, rather than 25%.)

   The 24 respondents who had participated in the Large Scale Jobs Reimbursement Program and the 32 respondents who had worked on scaling their codes with the NERSC consultants rated these initiatives as "very useful" on average. poe+, used to measure code performance characteristics, had been used by 104 respondents and was also rated "very useful" on average. The 115 respondents who rated Seaborg's new batch class structure, designed to give preference to high concurrency jobs, gave it an average rating of "somewhat useful".

   20 users wrote comments in support of the scaling initiatives, for example:

   Please push this project as much as you can. This type of consulting is very important if one goes to the limit of a system in terms of #processors and sustained performance.

   11 users stated why they thought these initiatives are misguided. The general theme behind these comments was that it is science output that is important, not scaling per se. Some representative comments here:

   I believe that they are totally misguided. The emphasis should be on maximizing the SCIENTIFIC output from NERSC. If the best way to do this is for the user to run 100 1-node jobs at a time rather than 1 100-node job, every effort should be made to accommodate him/her. ... In the final analysis, it should be up to the users to decide how they use their allocations. Most, if not all of us, will choose a usage pattern which maximizes our scientific output. Remember that most of us are in computational science, not in computer science. We are interested in advancing our own fields of research, not in obtaining Gordon Bell awards.

   Don't freeze out the small-to-moderate user --- the science/CPU hour is often higher for the moderate user.

   There is always a tension between massive users and those who want to run smaller jobs. While many researchers use a single node (16 processors), I think it would not be cost effective for DOE to pay them to run on their own machines.

2. Why do you compute at NERSC? (What are the reasons NERSC is important to you?) [Read All 229 Responses]

   Many of the answers were along the lines of "to run my codes in order to get my science done". Users pointed out that they need powerful compute resources that they can't get elsewhere. Many users specifically mentioned large numbers of processors or parallel computing as a reason to compute at NERSC. Turnaround time (getting results fast) is very important. Data analysis, especially in the context of PDSF computing is also a common theme. One user even pointed out that the time is "free".

3. Has security gotten in the way of your work at NERSC?

   Ninety percent of the respondents (217 users) answered no to this question.

4. If security has gotten in the way of your work at NERSC, how? [Read All 25 Responses]

   25 users answered this question:

   • 10 pointed to difficulties accessing NERSC (the change to ssh version 2, FTP retirement, difficulties with tunneling and ports).
   • 6 reported password or login attempt problems.
   • 3 encountered difficulties with accessing HPSS
   • 3 had grid/distributed computing concerns,
   • 3 said "it's inconvenient".

5. How do you compare NERSC openness and access to your home site and others? [Read All 146 Responses]
   • 49% stated that NERSC has similar or greater openness than other sites they access
   • 28% said that NERSC's openness or security measures are good (without making a comparison)
   • 9% said that NERSC is less open or too secure

Users are invited to provide overall comments about NERSC:

• 119 users answered the question What does NERSC do well?   69 respondents pointed specifically to NERSC's good hardware management practices which provide users with excellent access to HPC resources; 62 mentioned User Support and NERSC's responsive staff; 17 highlighted documentation and 13 job scheduling and batch throughput. Some representative comments are:

  Powerful and well maintained machines, great mass storage facility, and helpful and responsive staff. What more could you want?

  As Apple would put it .... "it just works". I get my work done and done fast. Seaborg is up and working nearly all the time. Network, storage, it's all there when I need it. That is what matters most and NERSC delivers.

  NERSC simply is the best run centralized computer center on the planet. I have interacted with many central computer centers and none are as responsive, have people with the technical knowledge available to answer questions and have the system/software as well configured as does NERSC.

• 75 users responded to What should NERSC do differently?. The area of greatest concern is job scheduling; 14 users expressed concerns with favoring large jobs at the expense of smaller ones; six wanted more resources devoted to interactive computing and debugging. Next in concern is the need for more hardware: more compute power overall, different architectures, mid-range computing support, vector architectures. Eight users pointed out the need for better documentation and six wanted more training. Some of the comments from this section are:

  NERSC's new emphasis favoring large (1024+ processor) jobs runs contrary to its good record of catering to the scientific community. It needs to remember the community it is serving --- the customer is always right. The queue configuration should be returned to a state where it no longer favours jobs using large numbers of processors.

  I'm not in favor of giving highest priority to the extremely large jobs on all nodes of seaborg. I think that NERSC must accommodate capacity computing for energy research that cannot be performed anywhere else, in addition to providing capability computing for the largest simulations.

  NERSC should move more aggressively to upgrade its high end computing facilities. It might do well to offer a wider variety of architectures. For example, the large Pentium 4 clusters about to become operational at NCSA provide a highly cost effective resources for some problems, but not for others. If NERSC had a greater variety of machines, it might be able to better serve all its users. However, the most important improvement would be to simply increase the total computing power available to users.

  It would be great if NERSC could again acquire a supercomputer with excellent vector-processing capability, like the CRAY systems which existed for many years. The success of the Japanese "Earth Simulator" will hopefully cause a re-examination of hardware purchase decisions. Strong vector processors make scientific programming easier and more productive.

  Measure success on science output and not on size of budgets or quantity of hardware.

  The overhead on account managers still seems a bit much for what we're getting. I still find the ERCAP process onerous (i.e., more information requested than should be necessary). Also, most of the codes we are using are changing much more from year to year in a scientific sense than a computational sense, it becomes repetitious to have to keep evaluating them computationally each year. You need to keep in mind that most of us are being funded to do science rather than computational research.

• 65 users answered the question How does NERSC compare to other centers you have used?   63% of the respondents stated that NERSC was a good center (no comparison made) or was better than other centers they used. Reasons given for preferring NERSC include good hardware, networking and software management, good user support, and better job throughput. 11% of the respondents said that NERSC was not as good as another center they used. The most common reason for finding dissatisfaction with NERSC is job scheduling.

Here are the survey results:

1. Respondent Demographics
2. Overall Satisfaction and Importance; Why do you use NERSC?; Security and Flexible Work Option
3. All Satisfaction Questions and Changes from Previous Years
4. DOE and NERSC Scaling Initiatives
5. Web, NIM, and Communications
6. Hardware
7. Software
8. Training
9. User Services
10. Comments about NERSC

Respondent Demographics

Number of respondents to the survey: 326

• Respondents by DOE Office and User Role
• Respondents by Organization
• Which NERSC resources do you use?
• How long have you used NERSC?
• What desktop systems do you use to connect to NERSC?
• Web Browser Used to Take Survey
• Operating System Used to Take Survey

Respondents by DOE Office and User Role:

Office Respondents Percent ASCR 20 6 BER 32 10 BES 89 27 FES 35 11 HENP 146 45 guests 4 1

User Role Number Percent Principal Investigators 43 13 PI Proxies 37 11 Project Managers 17 5 Users 229 70

Respondents by Organization:

Organization Type Number Percent Universities 182 55.8 DOE Labs 120 36.8 Other Govt Labs 12 3.7 Industry 6 1.8 Private Labs 6 1.8 Organization Number Percent Berkeley Lab 59 18.1 UC Berkeley 17 5.2 Livermore 11 3.4 Oak Ridge 9 2.8 Argonne 8 2.5 Brookhaven 8 2.5 Stanford 8 2.5 U. South Carolina 8 2.5 UC Davis 7 2.1 U. Wisconsin - Madison 7 2.1 Yale 7 2.1 NREL 6 1.8 SLAC 6 1.8 U. Colorado 6 1.8 U. Washington 6 1.8 PNNL 5 1.5 UCLA 5 1.5 Ames Lab 4 1.2 Inst National de Physique 4 1.2 Ohio State 4 1.2 Purdue 4 1.2 U. Texas - Austin 4 1.2

Organization Number Auburn U. 3 Cal Tech 3 Georgia IT 3 Harvard 3 Max Planck Inst. 3 New York U. 3 UC San Diego 3 UC Santa Cruz 3 U. Oklahoma 3 Vanderbilt 3 City College NY 2 Inst NdFN Italy 2 Iowa State 2 Johns Hopkins 2 Joint Inst Nuc Research 2 Mississippi State 2 New Mexico State 2 NCAR 2 N. Carolina State 2 Northeastern U. 2 U. Chicago 2 U. Maryland 2 U. S. California 2 U. Utah 2 UC Irvine 2 UC Santa Barbara 2 Warsaw Tech 2 Other University 38 Other Gov. Labs 8 Industry 6 Other DOE Labs 4 Other Private Labs 3

Which NERSC resources do you use?

Note that users did not always check all the resources they use -- compare the table below with How Satisfied are you? (sorted by Number of Responses).

Resource	Responses
SP	226
HPSS	145
NIM	87
Consulting	80
PDSF	79
HPCF Website	73
Account Support	55
Computer Operations and Network Support	26
CVS	16
grid	13
Newton	12
Escher	11
Visualization Services	9
Alvarez	9
The lattice gauge connection	1
SCS	1
not yet	1

How long have you used NERSC?

Time	Number	Percent
less than 6 months	61	19
6 months - 3 years	161	51
more than 3 years	95	30

What desktop systems do you use to connect to NERSC?

System	Responses
UNIX Total	338
PC Total	195
Mac Total	60
UNIX-linux	229
PC-win2000	84
PC-winXP	79
UNIX-solaris	54
MAC-OSX	41
PC-win98	27
UNIX-irix	24
MAC-macos	19
UNIX-aix	15
UNIX-tru64	10
UNIX-hpux	4
PC-win95	3
Cygwin	2
UNIX-other	2
PC-other	2
OSF1	1
iMac	1
FreeBSD	1
alpha	1

Web Browser Used to Take Survey:

Browser	Number	Percent
Mozilla	91	26.1
MS Internet Explorer 6	90	25.9
Netscape 4	59	17.0
Netscape 7	46	13.2
Safari	22	6.3
MS Internet Explorer 5	15	4.3
Galeon	9	2.6
Konqueror	7	2.0
Netscape 6	6	1.7
w3m (Text browser)	1	0.3
Opera 6.0	1	0.3
Opera 7	1	0.3

OS Used to Take Survey:

OS	Number	Percent
UNIX Total	164	47.1
Windows Total	145	41.7
MacIntosh Total	39	11.2
Linux	136	39.1
Windows XP Pro	70	20.1
Windows NT	64	18.4
Mac PowerPC	39	11.2
SunOS	19	5.5
Windows 98	10	2.9
DEC OSF	5	1.4
SGI IRIX	4	1.1
Windows 2000	1	0.3

---

Overall Satisfaction and Importance

Satisfaction Average Score Mostly Satisfied 5.5 - 6.4 Somewhat Satisfied 4.5 - 5.4	Importance Average Score Very Important 2.5 - 3 Somewhat Important 1.5 - 2.4
Significance of Change significant increase significant decrease not significant

Overall Satisfaction with NERSC:

Topic	No. of Responses	Average	Std. Dev.	Change from 2002	Change from 2001
Account Support Services	245	6.39	1.05	0.01	-0.04
Overall satisfaction with NERSC	298	6.37	0.88	0.05	0.12
Consulting Services	233	6.34	1.01	0.04	0.04
Network Connectivity	241	6.23	0.94	0.16	0.20
Available Computing Hardware	255	6.13	1.06	0.16	0.02
Mass Storage Facilities	207	6.12	1.10	0.08	0.07
HW management and configuration	210	6.07	1.07	-0.03	0.25
Available software	242	6.05	1.09	0.07	0.24
SW maintenance and configuration	213	6.04	1.20	-0.13	0.12
HPCF Website	213	6.00	1.10	-0.09	-0.18
Allocation Process	196	5.69	1.26	-0.15	-0.31
Training	94	5.04	1.26	0.05	0.12
Visualization Services	97	4.81	1.17	-0.02	0.30

Importance to Users:

Topic	No. of Responses	Average	Std. Dev.	Change from 2002	Change from 2001
Overall satisfaction with NERSC	267	2.84	0.39	0.00	0.02
Available Computing Hardware	236	2.84	0.41	-0.05	0.03
Network Connectivity	223	2.75	0.47	-0.00	-0.07
Allocation Process	183	2.68	0.56	0.11	0.01
HW management and configuration	187	2.66	0.56	-0.05	0.04
Consulting Services	227	2.59	0.61	0.01	-0.05
Account Support Services	230	2.54	0.60	0.07	0.10
Mass Storage Facilities	190	2.53	0.62	0.02	0.05
SW maintenance and configuration	194	2.52	0.62	-0.02	-0.08
Available software	222	2.50	0.64	-0.03	-0.06
HPCF Website	197	2.39	0.64	-0.13	-0.10
Training	116	1.80	0.79	0.00	0.28
Visualization Services	125	1.60	0.75	-0.02	-0.11

Why do you compute at NERSC?

[Read all 229 responses ]

Many of the answers were along the lines of "to run my codes in order to get my science done". Users pointed out that they need powerful compute resources that they can't get elsewhere. Many users specifically mentioned large numbers of processors or parallel computing as a reason to compute at NERSC. Turnaround time (getting results fast) is very important. Data analysis, especially in the context of PDSF computing is also a common theme. One user even pointed out that the time is "free".

139	Need lots of compute power to do science (focus on compute resources)
32	Focus on data analysis / PDSF
31	NERSC is a well run center / provides good services:
20	Need lots of storage and compute power
10	Need large memory (and lots of storage and compute power)
8	Need to test / install / maintain software
3	Other

Security

Question	No. of Responses	Average	Std. Dev.	Change from 2002	Change from 2001
How satisfied are you with NERSC security?	261	6.37	1.09	NA	NA

Has security gotten in the way of your work at NERSC?

	No. of Responses
No	217 (90%)
Yes	23 (10%)

If yes, how?

[Read all 26 responses]

10	Access problems (ssh, secure ftp)
6	Password problems, Failed login attempts
3	HPSS access difficult
3	Grid, Distributed computing, Collaborative computing
3	It's inconvenient
1	Security is OK

How do you compare NERSC openness and access to your home site and others?

[Read all 146 responses]

49	(34%)	Similar openness
41	(28%)	NERSC's openness / security measures are good
23	(16%)	NERSC is more open / more flexible
13	(9%)	NERSC is less open / more restrictive; too secure / too complicated
2	(1%)	Concerns about NERSC security
2	(1%)	Home site is secure
1	(0.7%)	NERSC is less secure
4	(3%)	Other

Flexible Work Option at NERSC

Beginning in May, NERSC started participating in Berkeley Lab's Flexible Work Option (FWO) Pilot. FWO means some staff work 9 hours a day and are off one day every two weeks. NERSC always has on duty qualified staff for all areas.

Have you noticed any changes specifically due to the FWO participation?

	No. of Responses
No	220
Yes	0

If so, describe your experience. 2 responses

---

All Satisfaction Topics and Changes from Previous Years

• How Satisfied are you? (sorted by Average Score)
• How Satisfied are you? (sorted by Number of Responses)

Legend

Satisfaction Average Score Very Satisfied 6.5 - 7 Mostly Satisfied 5.5 - 6.4 Somewhat Satisfied 4.5 - 5.4

Significance of Change significant increase significant decrease not significant

How Satisfied are you? (sorted by Average Score)

Topics from the Overall Satisfaction with NERSC section are indicated in bold; they tend to be answered by more users and rated lower than the topics in the ensuing sections that rate specific areas in more detail.

Topic	No. of Responses	Average	Std. Dev.	Change from 2002	Change from 2001
HPSS Reliability	126	6.61	0.77	0.10	-0.02
Consult: Timely response	207	6.55	0.73	0.04	-0.01
Consult: Technical advice	200	6.54	0.69	0.08	0.08
HPSS Uptime	126	6.54	0.79	0.17	0.21
Network LAN	114	6.54	0.67	NA	NA
Consult: Followup to initial questions	186	6.49	0.75	0.10	0.12
HPSS Overall	134	6.46	0.84	0.07	-0.04
HPSS Performance	126	6.46	0.88	0.11	0.10
PDSF C/C++ Compilers	54	6.44	0.79	-0.02	NA
SP Overall	192	6.43	0.78	0.05	0.61
SP Uptime	191	6.42	0.83	-0.14	0.89
PDSF Overall	68	6.41	0.87	0.15	NA
Account Support Services	245	6.39	1.05	0.01	-0.04
Overall satisfaction with NERSC	298	6.37	0.88	0.05	0.12
Consult: Time to solve problems	196	6.36	0.84	-0.04	NA
Consult: Response to special requests	126	6.35	1.06	-0.05	0.12
PDSF Uptime	62	6.35	1.04	-0.16	NA
Consulting Overall	233	6.34	1.01	0.04	0.04
SP Fortran Compilers	152	6.34	0.97	-0.02	0.38
PDSF User Environment	55	6.33	0.77	-0.05	NA
SP Libraries	131	6.27	0.86	0.18	0.27
New User's Guide	137	6.26	0.86	0.05	0.32
SP User Environment	169	6.24	0.90	0.12	0.17
Network Connectivity	241	6.23	0.94	0.16	0.20
SP C/C++ Compilers	103	6.22	1.10	0.11	0.50
SP Disk Configuration and I/O Performance	156	6.15	1.03	0.18	0.48
Available Computing Hardware	255	6.13	1.06	0.16	0.02
Network WAN	100	6.12	1.02	NA	NA
Mass Storage Facilities	207	6.12	1.10	0.08	0.07
NERSC Online Tutorials	121	6.07	0.99	0.10	0.10
HW management and configuration	210	6.07	1.07	-0.03	0.25
Available software	242	6.05	1.09	0.07	0.24
SW maintenance and configuration	213	6.04	1.20	-0.13	0.12
PDSF Fortran Compilers	29	6.03	1.09	-0.42	NA
RightNowWeb interface	109	6.02	1.07	0.08	NA
HPCF Website	213	6.00	1.10	-0.09	-0.18
PDSF Queue Structure	59	6.00	0.96	0.03	NA
SP Applications	94	6.00	1.04	0.30	0.33
PDSF Libraries	43	6.00	1.07	-0.24	NA
SP General Tools and Utilities	111	5.98	1.04	0.18	0.26
HPSS User Interface	127	5.98	1.24	0.03	-0.04
PDSF General Tools and Utilities	44	5.93	1.07	-0.11	NA
PDSF Batch Wait Time	61	5.93	1.12	0.19	NA
PDSF Applications	39	5.87	1.06	-0.34	NA
PDSF Bug Resolution	34	5.85	1.21	-0.15	NA
NERSC Training Web Pages	98	5.83	1.06	-0.06	NA
PDSF Ability to Run Interactively	64	5.77	1.39	-0.41	NA
PDSF Disk Configuration and I/O Performance	59	5.69	1.15	0.06	NA
SP Queue Structure	177	5.69	1.22	-0.23	0.50
Allocation Process	196	5.69	1.26	-0.15	-0.31
SP Bug Resolution	81	5.64	1.15	0.05	0.19
SP Performance and Debugging Tools	117	5.57	1.31	0.08	0.88
SP Ability to Run Interactively	162	5.57	1.49	0.10	0.86
PDSF Performance and Debugging Tools	42	5.31	1.39	0.06	NA
SP Batch Wait Time	190	5.24	1.52	-0.17	0.32
Escher	13	5.23	1.30	-0.15	0.15
Newton	15	5.20	1.37	-0.24	-0.27
SP Viz Software	37	5.08	1.46	NA	NA
Training	94	5.04	1.26	0.05	0.12
NERSC Training Classes	24	4.88	1.15	-0.25	-0.67
Visualization Services	97	4.81	1.17	-0.02	0.30
Escher Viz Software	8	4.75	1.39	NA	NA
Access Grid classes	27	4.67	1.41	NA	NA

How Satisfied are you? (sorted by Number of Responses)

This ordering helps to indicate which services are used most by users (and is probably a better indicator than the services clicked for the question What NERSC resources do you use?

Topic	No. of Responses	Average	Std. Dev.	Change from 2002	Change from 2001
Overall satisfaction with NERSC	298	6.37	0.88	0.05	0.12
Available Computing Hardware	255	6.13	1.06	0.16	0.02
Account Support Services	245	6.39	1.05	0.01	-0.04
Available software	242	6.05	1.09	0.07	0.24
Network Connectivity	241	6.23	0.94	0.16	0.20
Consulting Overall	233	6.34	1.01	0.04	0.04
HPCF Website	213	6.00	1.10	-0.09	-0.18
SW maintenance and configuration	213	6.04	1.20	-0.13	0.12
HW management and configuration	210	6.07	1.07	-0.03	0.25
Consult: Timely response	207	6.55	0.73	0.04	-0.01
Mass Storage Facilities	207	6.12	1.10	0.08	0.07
Consult: Technical advice	200	6.54	0.69	0.08	0.08
Allocation Process	196	5.69	1.26	-0.15	-0.31
Consult: Time to solve problems	196	6.36	0.84	-0.04	NA
SP Overall	192	6.43	0.78	0.05	0.61
SP Uptime	191	6.42	0.83	-0.14	0.89
SP Batch Wait Time	190	5.24	1.52	-0.17	0.32
Consult: Followup to initial questions	186	6.49	0.75	0.10	0.12
SP Queue Structure	177	5.69	1.22	-0.23	0.50
SP User Environment	169	6.24	0.90	0.12	0.17
SP Ability to Run Interactively	162	5.57	1.49	0.10	0.86
SP Disk Configuration and I/O Performance	156	6.15	1.03	0.18	0.48
SP Fortran Compilers	152	6.34	0.97	-0.02	0.38
New User's Guide	137	6.26	0.86	0.05	0.32
HPSS Overall	134	6.46	0.84	0.07	-0.04
SP Libraries	131	6.27	0.86	0.18	0.27
HPSS User Interface	127	5.98	1.24	0.03	-0.04
Consult: Response to special requests	126	6.35	1.06	-0.05	0.12
HPSS Performance	126	6.46	0.88	0.11	0.10
HPSS Reliability	126	6.61	0.77	0.10	-0.02
HPSS Uptime	126	6.54	0.79	0.17	0.21
NERSC Online Tutorials	121	6.07	0.99	0.10	0.10
SP Performance and Debugging Tools	117	5.57	1.31	0.08	0.88
Network LAN	114	6.54	0.67	NA	NA
SP General Tools and Utilities	111	5.98	1.04	0.18	0.26
RightNowWeb interface	109	6.02	1.07	0.08	NA
SP C/C++ Compilers	103	6.22	1.10	0.11	0.50
Network WAN	100	6.12	1.02	NA	NA
NERSC Training Web Pages	98	5.83	1.06	-0.06	NA
Visualization Services	97	4.81	1.17	-0.02	0.30
SP Applications	94	6.00	1.04	0.30	0.33
Training	94	5.04	1.26	0.05	0.12
SP Bug Resolution	81	5.64	1.15	0.05	0.19
PDSF Overall	68	6.41	0.87	0.15	NA
PDSF Ability to Run Interactively	64	5.77	1.39	-0.41	NA
PDSF Uptime	62	6.35	1.04	-0.16	NA
PDSF Batch Wait Time	61	5.93	1.12	0.19	NA
PDSF Disk Configuration and I/O Performance	59	5.69	1.15	0.06	NA
PDSF Queue Structure	59	6.00	0.96	0.03	NA
PDSF User Environment	55	6.33	0.77	-0.05	NA
PDSF C/C++ Compilers	54	6.44	0.79	-0.02	NA
PDSF General Tools and Utilities	44	5.93	1.07	-0.11	NA
PDSF Libraries	43	6.00	1.07	-0.24	NA
PDSF Performance and Debugging Tools	42	5.31	1.39	0.06	NA
PDSF Applications	39	5.87	1.06	-0.34	NA
SP Viz Software	37	5.08	1.46	NA	NA
PDSF Bug Resolution	34	5.85	1.21	-0.15	NA
PDSF Fortran Compilers	29	6.03	1.09	-0.42	NA
Access Grid classes	27	4.67	1.41	NA	NA
NERSC Training Classes	24	4.88	1.15	-0.25	-0.67
Newton	15	5.20	1.37	-0.24	-0.27
Escher	13	5.23	1.30	-0.15	0.15
Escher Viz Software	8	4.75	1.39	NA	NA

DOE and NERSC Scaling Initiatives

Legend:

Usefulness	Average Score
Very Useful	2.5 - 3.0
Somewhat Useful	1.5 - 2.4

How useful were the following scaling initiatives?

Initiative	No. "don't use" Responses	No. of other Responses	Average	Std. Dev.	Change from 2002	Change from 2001
large scale job reimbursement program	9	24	2.75	0.61	NA	NA
consulting scaling services	66	32	2.59	0.50	NA	NA
poe+	12	104	2.50	0.54	NA	NA
new Seaborg batch class structure	48	115	2.17	0.72	NA	NA

Comments on NERSC's scaling initiatives:

[Read all 39 comments]

20	Good scaling initiatives
11	Wrong approach
4	Didn't help / not ready to use / not interested
2	Startup projects weren't eligible
2	Users need more technical info
1	Sometimes hurts smaller jobs / sometimes OK

Did you participate in the Large Job Reimbursement Project?

	No. of Responses
No	183 (90%)
Yes	21 (10%)

Have you or your project submitted information to the Applications performance Matrix?

	No. of Responses
No	118 (71%)
Yes	48 (29%)

Have you use poe+?

	No. of Responses
Yes	104 (52%)
No	95 (48%)

Do you plan to submit information to the the Applications performance Matrix in the coming year?

	No. of Responses
Yes	85 (64%)
No	48 (36%)

If you don't plan on submitting to the Matrix next year, why not?

[Read all 36 responses]

13	Don't know about the Matrix / not my role to do this
7	Codes don't scale well / don't have large codes
6	Question doesn't apply to PDSF users:
5	It's inappropriate / don't have enough time:
2	Have already submitted and no changes expected next year:
3	Other

---

Information Technology (Web, NIM, Communications)

• Legend
• Satisfaction with the Web
• How useful are these methods for keeping you informed?
• Are you well informed of changes?
• Summary of Information Technology Comments
• Comments concerning the HPCF web site:   22 responses
• Comments concerning the PDSF web site:   6 responses
• Comments concerning NIM:   20 responses
• Comments concerning how NERSC keeps you informed of changes:   19 responses

Legend

Satisfaction Average Score Mostly Satisfied 5.5 - 6.4 Somewhat Satisfied 4.5 - 5.4	Usefulness Average Score Somewhat Useful 1.50 - 2.49 Not Useful / Not Used 0.50 - 1.49
Significance of Change significant increase significant decrease not significant

Satisfaction with the Web

Question	No. of Responses	Average	Std. Dev.	Change from 2002	Change from 2001
Accuracy	181	6.25	0.93	0.00	0.10
NIM	132	6.08	1.18	0.08	NA
Seaborg job displays	157	6.07	1.06	NA	NA
Timeliness of info	172	6.05	1.09	-0.15	-0.03
SP Pages	159	6.00	1.04	0.04	0.22
Help section	163	5.93	1.11	NA	NA
General programming info	162	5.91	1.02	0.06	0.19
Software Section	160	5.87	1.04	-0.09	NA
PDSF	84	5.83	0.99	0.02	NA
Ease of finding info on web	216	5.80	1.10	-0.00	-0.08
File Storage Pages	125	5.74	1.14	-0.10	-0.05
IBM docs	142	5.70	1.18	NA	NA
Search facilities	164	5.44	1.19	-0.12	-0.11

How useful are these methods for keeping you informed?

Question	No. of Responses	Average	Std. Dev.	Change from 2002	Change from 2001
Email	186	2.46	0.66	-0.00	0.02
Announcement web archive	174	2.40	0.65	0.06	0.35
MOTD	165	2.18	0.77	0.09	-0.09
Phone calls	142	1.77	0.90	0.09	-0.04

Are you well informed of changes?

Question	No. of Yes Responses	No. of No Responses	Percent Who Said Yes	Percent Who Said Yes in 2002	Percent Who Said Yes in 2001
Do you feel you are adequately informed?	208	12	95	96	94
Are you aware of major changes at least 1 month in advance?	162	25	87	91	81
Are you aware of software changes at least 7 days in advance?	165	22	88	86	81
Are you aware of planned outages 24 hours in advance?	170	20	89	87	91

Summary of Information Technology Comments

Comments concerning the HPCF web site

[Read all 32 responses]

9	Improve navigation / organization / presentation
8	Good website
5	Provide additional or clearer information
5	Keep info up-to-date / correct errors
4	Improve searching
4	Other

Comments concerning the PDSF web site

[Read all 12 responses]

5	Improve navigation / organization / presentation
4	Keep info up-to-date / correct errors
4	Good website
1	Provide additional or clearer information
1	Other (certificates)

Comments concerning NIM

[Read all 26 responses]

8	Issues with ERCAP or the allocations process
8	Good interface
3	Problems with reliability
1	Difficult to use
1	Violates privacy

Comments concerning how NERSC keeps you informed of changes

[Read all 22 responses]

8	Satisfied / well informed
6	Comments on system outages
5	Comments on using email
2	Comments on the MOTD
1	Software changes

---

Hardware Resources

• Legend
• Satisfaction - Compute Platforms (sorted by Average Score)
• Satisfaction - Compute Platforms (sorted by Platform)
• Max Processors Used and Max Code Can Effectively Use
• Satisfaction - HPSS
• Satisfaction - Servers
• Satisfaction - Networking
• Summary of Hardware Comments
• Comments on NERSC's IBM SP:   51 responses
• Comments on NERSC's PDSF Cluster:   17 responses
• Comments on NERSC's HPSS Storage System: 29 responses
• Comments about NERSC's auxiliary servers:   5 responses

Legend:

Satisfaction Average Score Very Satisfied 6.5 - 7 Mostly Satisfied 5.5 - 6.4 Somewhat Satisfied 4.5 - 5.4

Significance of Change significant increase significant decrease not significant

Satisfaction - Compute Platforms

Sorted by average score

Question	No. of Responses	Average	Std. Dev.	Change from 2002	Change from 2001
SP Overall	192	6.43	0.78	0.05	0.61
SP Uptime	191	6.42	0.83	-0.14	0.89
PDSF Overall	68	6.41	0.87	0.15	NA
PDSF Uptime	62	6.35	1.04	-0.16	NA
SP Disk Configuration and I/O Performance	156	6.15	1.03	0.18	0.48
PDSF Queue Structure	59	6.00	0.96	0.03	NA
PDSF Batch Wait Time	61	5.93	1.12	0.19	NA
PDSF Ability to Run Interactively	64	5.77	1.39	-0.41	NA
PDSF Disk Configuration and I/O Performance	59	5.69	1.15	0.06	NA
SP Queue Structure	177	5.69	1.22	-0.23	0.50
SP Ability to Run Interactively	162	5.57	1.49	0.10	0.86
SP Batch Wait Time	190	5.24	1.52	-0.17	0.32

Satisfaction - Compute Platforms

Sorted by Platform

Question	No. of Responses	Average	Std. Dev.	Change from 2002	Change from 2001
SP Overall	192	6.43	0.78	0.05	0.61
SP Uptime	191	6.42	0.83	-0.14	0.89
SP Disk Configuration and I/O Performance	156	6.15	1.03	0.18	0.48
SP Queue Structure	177	5.69	1.22	-0.23	0.50
SP Ability to Run Interactively	162	5.57	1.49	0.10	0.86
SP Batch Wait Time	190	5.24	1.52	-0.17	0.32
PDSF Overall	68	6.41	0.87	0.15	NA
PDSF Uptime	62	6.35	1.04	-0.16	NA
PDSF Queue Structure	59	6.00	0.96	0.03	NA
PDSF Batch Wait Time	61	5.93	1.12	0.19	NA
PDSF Ability to Run Interactively	64	5.77	1.39	-0.41	NA
PDSF Disk Configuration and I/O Performance	59	5.69	1.15	0.06	NA

Max Processors Used and Max Code Can Effectively Use

Question	No. of Responses	Average	Std. Dev.	Change from 2002	Change from 2001
SP Processors Can Use	139	609.41	1006.23	63.41	-141.59
Max SP Processors Used	161	444.84	733.31	273.84	242.84
Max PDSF Processors Used	35	13.06	43.25	-21.94	NA
PDSF Processors Can Use	34	10.26	21.37	-86.74	NA

Satisfaction - HPSS

Question	No. of Responses	Average	Std. Dev.	Change from 2002	Change from 2001
Reliability	126	6.61	0.77	0.10	-0.02
Uptime	126	6.54	0.79	0.17	0.21
Performance	126	6.46	0.88	0.11	0.10
HPSS Overall	134	6.46	0.84	0.07	-0.04
User Interface	127	5.98	1.24	0.03	-0.04

Satisfaction - Servers:

Question	No. of Responses	Average	Std. Dev.	Change from 2002	Change from 2001
Escher	13	5.23	1.30	-0.15	0.15
Newton	15	5.20	1.37	-0.24	-0.27

Satisfaction - Networking

Question;	No. of Responses	Average	Std. Dev.	Change from 2002	Change from 2001
LAN	114	6.54	0.67	NA	NA
WAN	100	6.12	1.02	NA	NA

Summary of Hardware Comments

Comments on NERSC's IBM SP

[Read all 51 responses]

16	Good machine
15	Queue issues
12	Scaling comments
8	Provide more interactive and debugging resources
5	Allocation issues
4	Provide more serial resources
3	User environment issues
2	Other (down times and need faster processors)

Comments on NERSC's PDSF Cluster

[Read all 17 responses]

10	Disk, I/O and file issues
6	Batch issues
4	Good system
4	Provide more interactive and debugging resources
2	Down time issues
2	Other (slow processors, utilitarian code)

Comments on NERSC's HPSS Storage System

[Read all 29 responses]

14	Good system
7	Hard to use / user interface issues
5	Performance improvements needed
3	Authentication is difficult
2	Don't like the down times
2	Network / file transfer problems
2	Other (Grid, SRUs)

Comments about NERSC's math and vis servers

[Read all 5 responses]

3	Network connection too slow
2	Good service
1	Remote licenses

---

Software Resources

• Legend
• Satisfaction with Software (sorted by Average Score)
• Satisfaction with Software (sorted by Platform)
• Comments about Software:   28 responses

Legend:

Satisfaction Average Score Mostly Satisfied 5.5 - 6.4 Somewhat Satisfied 4.5 - 5.4

Significance of Change significant increase significant decrease not significant

Satisfaction with Software

Sorted by Satisfaction Rating

Question	No. of Responses	Average	Std. Dev.	Change from 2002	Change from 2001
PDSF C/C++ Compilers	54	6.44	0.79	-0.02	NA
SP Fortran Compilers	152	6.34	0.97	-0.02	0.38
PDSF User Environment	55	6.33	0.77	-0.05	NA
SP Libraries	131	6.27	0.86	0.18	0.27
SP User Environment	169	6.24	0.90	0.12	0.17
SP C/C++ Compilers	103	6.22	1.10	0.11	0.50
PDSF Fortran Compilers	29	6.03	1.09	-0.42	NA
SP Applications	94	6.00	1.04	0.30	0.33
PDSF Libraries	43	6.00	1.07	-0.24	NA
SP General Tools and Utilities	111	5.98	1.04	0.18	0.26
PDSF General Tools and Utilities	44	5.93	1.07	-0.11	NA
PDSF Applications	39	5.87	1.06	-0.34	NA
PDSF Bug Resolution	34	5.85	1.21	-0.15	NA
SP Bug Resolution	81	5.64	1.15	0.05	0.19
SP Performance and Debugging Tools	117	5.57	1.31	0.08	0.88
PDSF Performance and Debugging Tools	42	5.31	1.39	0.06	NA
SP Viz Software	37	5.08	1.46	NA	NA
Escher Viz Software	8	4.75	1.39	NA	NA

Satisfaction with Software

Sorted by Platform

Question	No. of Responses	Average	Std. Dev.	Change from 2002	Change from 2001
SP Fortran Compilers	152	6.34	0.97	-0.02	0.38
SP Libraries	131	6.27	0.86	0.18	0.27
SP User Environment	169	6.24	0.90	0.12	0.17
SP C/C++ Compilers	103	6.22	1.10	0.11	0.50
SP Applications	94	6.00	1.04	0.30	0.33
SP General Tools and Utilities	111	5.98	1.04	0.18	0.26
SP Bug Resolution	81	5.64	1.15	0.05	0.19
SP Performance and Debugging Tools	117	5.57	1.31	0.08	0.88
SP Viz Software	37	5.08	1.46	NA	NA
PDSF C/C++ Compilers	54	6.44	0.79	-0.02	NA
PDSF User Environment	55	6.33	0.77	-0.05	NA
PDSF Fortran Compilers	29	6.03	1.09	-0.42	NA
PDSF Libraries	43	6.00	1.07	-0.24	NA
PDSF General Tools and Utilities	44	5.93	1.07	-0.11	NA
PDSF Applications	39	5.87	1.06	-0.34	NA
PDSF Bug Resolution	34	5.85	1.21	-0.15	NA
PDSF Performance and Debugging Tools	42	5.31	1.39	0.06	NA
Escher Viz Software	8	4.75	1.39	NA	NA

Comments about Software

[Read all 28 responses]

8	Debuggers and performance analysis tools
7	IBM compilers and libraries
4	Unix environment (shells, editors, GNU)
4	PDSF software
4	Satisfied
3	3rd party applications and libraries (viz, HDF)
2	Other

---

Training

• Legend
• Satisfaction with Training
• How Useful are these resources for training in HPC?
• What training methods would you like NERSC to offer?   161 responses
• Comments about training:   23 responses

Legend

Satisfaction Average Score Mostly Satisfied 5.5 - 6.4 Somewhat Satisfied 4.5 - 5.4	Usefulness Average Score Very Useful 2.50 - 3 Somewhat Useful 1.50 - 2.49 Not Useful 1.00 - 1.49
Significance of Change significant increase significant decrease not significant

Satisfaction with Training

Question	No. of Responses	Average	Std. Dev.	Change from 2002	Change from 2001
New User's Guide	137	6.26	0.86	0.05	0.32
NERSC Online Tutorials	121	6.07	0.99	0.10	0.10
NERSC Training Web Pages	98	5.83	1.06	-0.06	NA
NERSC Training Classes	24	4.88	1.15	-0.25	-0.67
Access Grid classes	27	4.67	1.41	NA	NA

How Useful are these resources for training in HPC?

Question	No. of Responses	Average	Std. Dev.	Change from 2002	Change from 2001
Online Tutorials	54	2.44	0.63	-0.08	-0.11
Classes	33	1.76	0.75	-0.24	-0.31
Access Grid classes	30	1.63	0.72	0.00	0.00

What training methods would you like NERSC to offer?   161 responses

138	(85%)	General online web documentation
83	(83%)	Online web tutorials on specific topics
26	(16%)	Live web broadcasts with teleconference audio
21	(13%)	Live in-person classes at LBNL
18	(11%)	Live in-person classes at your site
10	(16%)	Live Access Grid Classes

Comments about training

[Read all 23 responses]

This year most of the comments concerned the training format rather than suggestions for topics. Seven users said that web documents are sufficient; six pointed out that training via the Access Grid is not a good solution; and five recommended webcasts.

NERSC response: As of October 2003 NERSC has captured all training sessions using RealPlayer. These are available for replay on the web. On the day of the training classes users can participate in any of the following ways:

• Watch live media streaming using RealPlayer from their desktop.
• Join an audio conference to interact with NERSC staff and ask questions.
• Attend via the Access Grid.
• Watch the slides via PowerPoint of HTML slides

The training topics users requested most frequently are performance and how to get started.

15	Recommended a training format
8	Suggested topics
2	General comments

---

User Services

Legend:

Satisfaction Average Score Very Satisfied 6.5 - 7 Mostly Satisfied 5.5 - 6.4

Significance of Change significant increase not significant

Satisfaction with User Services

Question	No. of Responses	Average	Std. Dev.	Change from 2002	Change from 2001
Timely response	207	6.55	0.73	0.04	-0.01
Technical advice	200	6.54	0.69	0.08	0.08
Followup to initial questions	186	6.49	0.75	0.10	0.12
Time to solve problems	196	6.36	0.84	-0.04	NA
Response to special requests	126	6.35	1.06	-0.05	0.12
RightNowWeb interface	109	6.02	1.07	0.08	NA

Comments about Consulting and Account Support

[Read all 23 responses]

19	Good service
4	Mostly happy, mixed evaluation
2	Unhappy

Comments about NERSC

What does NERSC do well?

[Read all 119 responses]

69	Good hardware management, good uptime, access to HPC resources
62	User support, good staff
51	Generally happy, well run center
17	Job scheduling / batch throughput
11	Documentation
10	Software / user environment
6	Good network access
5	Allocations process

What should NERSC do differently?

[Read all 75 responses]

The area of greatest concern is job scheduling; 14 users expressed concerns with favoring large jobs at the expense of smaller ones; six wanted more resources devoted to interactive computing and debugging. Next in concern is the need for more hardware: more compute power overall, different architectures, mid-range computing support, vector architectures. Seven users pointed out the need for better documentation and six wanted more training.

24	Seaborg job scheduling / job policies
16	Provide more/new hardware; more computing resources
7	Better documentation
6	General center policies
6	More/better training
6	PDSF improvements
5	Seaborg software improvements
4	Other Seaborg improvements
3	Network improvements
3	No need for change
2	HPSS improvements
1	Shorter survey

How does NERSC compare to other centers you have used?

[Read all 65 responses]

Reasons given for preferring NERSC include good hardware, networking and software management, good user support, and better job throughput. The most common reason for finding dissatisfaction with NERSC is job scheduling.

41	NERSC is the best / overall NERSC is better / positive response
11	NERSC is the same as / mixed response
7	NERSC is less good / negative response
6	No comparison made

---

Page last modified: Thu, 22 Jul 2004 23:35:40 GMT Page URL: http://www.nersc.gov/news/survey/2003/summary.php Web contact: webmaster@nersc.gov Computing questions: consult@nersc.gov Privacy and Security Notice