2007/2008 User Survey Results

Response Summary

Many thanks to the 467 users who responded to this year's User Survey. The response rate has significantly increased from previous years:

• 70 percent of users who had used more than 1 million MPP hours when the survey opened responded
• 43 percent of users who had used between 10,000 and 1 million MPP hours responded
• The overall response rate for the 2,804 authorized users during the survey period was 16.3%.

The respondents represent all six DOE Science Offices and a variety of home institutions: see Respondent Demographics.

The survey responses provide feedback about every aspect of NERSC's operation, help us judge the quality of our services, give DOE information on how well NERSC is doing, and point us to areas we can improve. The survey results are listed below.

You can see the 2007/2008 User Survey text, in which users rated us on a 7-point satisfaction scale. Some areas were also rated on a 3-point importance scale or a 3-point usefulness scale.

Satisfaction Score Meaning Number of Times Selected 7 Very Satisfied 9,486 6 Mostly Satisfied 6,886 5 Somewhat Satisfied 1,682 4 Neutral 1,432 3 Somewhat Dissatisfied 485 2 Mostly Dissatisfied 130 1 Very Dissatisfied 81	Importance Score Meaning 3 Very Important 2 Somewhat Important 1 Not Important
	Usefulness Score Meaning 3 Very Useful 2 Somewhat Useful 1 Not at All Useful

The average satisfaction scores from this year's survey ranged from a high of 6.71 (very satisfied) to a low of 4.46 (neutral). Across 128 questions, users chose the Very Satisfied rating 9,486 times, and the Very Dissatisfied rating 81 times. The scores for all questions averaged 6.07, and the average score for overall satisfaction with NERSC was 6.3. See All Satisfaction Ratings.

For questions that spanned the 2007/2008 through 2003 surveys, the change in rating was tested for significance (using the t test at the 90% confidence level). Significant increases in satisfaction are shown in blue; significant decreases in satisfaction are shown in red.

Significance of Change significant increase (change from 2006) significant decrease (change from 2006) not significant

Areas with the highest user satisfaction include account support, the NERSC Global Filesystem, the HPSS mass storage system, consulting services, network performance within the NERSC center, and up times for the Jacquard, Seaborg and Bassi systems.

7=Very satisfied, 6=Mostly satisfied, 5=Somewhat satisfied, 4=Neutral, 3=Somewhat dissatisfied, 2=Mostly dissatisfied, 1=Very dissatisfied

Item	Num who rated this item as:							Total Responses	Average Score	Std. Dev.	Change from 2006
	1	2	3	4	5	6	7
SERVICES: Account support			2	1	6	82	265	356	6.71	0.57	0.07
NGF: Reliability				1	1	16	47	65	6.68	0.59	0.25
NGF: Uptime				1	1	17	47	66	6.67	0.59	0.32
HPSS: Reliability (data integrity)			1	3	4	29	111	148	6.66	0.70	-0.04
OVERALL: Consulting and Support Services			3	9	13	91	310	426	6.63	0.71	0.11
Network performance within NERSC (e.g. Seaborg to HPSS)				4	4	49	111	168	6.59	0.66	0.06
CONSULT: overall			4	8	7	102	241	362	6.57	0.74	0.10
CONSULT: Timely initial response to consulting questions	1	1	2	4	11	107	229	355	6.55	0.77	-0.02
Jacquard: Uptime (Availability)				4	6	34	82	126	6.54	0.73	-0.04
HPSS: Uptime (Availability)			1	3	6	45	96	151	6.54	0.73	-0.08
Seaborg: Uptime (Availability)				4	7	38	89	138	6.54	0.73	0.30
Bassi: Uptime (Availability)	1		1	5	6	48	122	183	6.54	0.84	0.13

Areas with the lowest user satisfaction include batch wait times for Bassi and Jacquard, Franklin availability and I/O, training classes and data analysis and visualization services.

7=Very satisfied, 6=Mostly satisfied, 5=Somewhat satisfied, 4=Neutral, 3=Somewhat dissatisfied, 2=Mostly dissatisfied, 1=Very dissatisfied

Item	Num who rated this item as:							Total Responses	Average Score	Std. Dev.	Change from 2006
	1	2	3	4	5	6	7
OVERALL: Data analysis and visualization facilities			6	68	28	67	62	231	5.48	1.24	0.11
Jacquard: Batch wait time	2	3	13	6	28	40	34	126	5.47	1.46	-0.40
TRAINING: NERSC classes: in-person			2	20	3	8	18	51	5.39	1.42	-0.55
Seaborg SW: Visualization software	1		1	10	4	12	9	37	5.38	1.42	-0.07
Bassi SW: Visualization software			2	16	4	16	11	49	5.37	1.27	0.00
Franklin SW: Visualization software	1		3	19	7	19	16	65	5.34	1.38
Live classes on the web		1		23	6	16	14	60	5.30	1.29	-0.46
Franklin: Disk configuration and I/O performance	8	11	20	36	34	73	51	233	5.15	1.63
Franklin: Uptime (Availability)	7	11	48	12	46	86	47	257	5.04	1.64
Bassi: Batch wait time	11	19	36	16	33	46	22	183	4.46	1.80	-1.39

The largest increases in satisfaction over last year's survey are for the now retired Seaborg IBM POWER3 system, for computer and network operations 24 by 7 support, and for the software available on our systems.

7=Very satisfied, 6=Mostly satisfied, 5=Somewhat satisfied, 4=Neutral, 3=Somewhat dissatisfied, 2=Mostly dissatisfied, 1=Very dissatisfied

Item	Num who rated this item as:							Total Responses	Average Score	Std. Dev.	Change from 2006
	1	2	3	4	5	6	7
Seaborg: Batch wait time	2	2	8	12	33	47	34	138	5.53	1.32	0.59
SERVICES: Computer and network operations support (24x7)			2	11	9	46	93	161	6.35	0.95	0.31
Seaborg: Uptime (Availability)				4	7	38	89	138	6.54	0.73	0.30
Seaborg: overall				5	12	55	71	143	6.34	0.78	0.25
OVERALL: Available Software			3	19	43	157	176	398	6.22	0.87	0.24

The largest decreases in satisfaction over last year's survey are shown below.

7=Very satisfied, 6=Mostly satisfied, 5=Somewhat satisfied, 4=Neutral, 3=Somewhat dissatisfied, 2=Mostly dissatisfied, 1=Very dissatisfied

Item	Num who rated this item as:							Total Responses	Average Score	Std. Dev.	Change from 2006
	1	2	3	4	5	6	7
Bassi: Batch wait time	11	19	36	16	33	46	22	183	4.46	1.80	-1.39
Jacquard SW: Visualization software				11	4	13	10	38	5.58	1.18	-0.54
Jacquard: Batch wait time	2	3	13	6	28	40	34	126	5.47	1.46	-0.40
Bassi: Batch queue structure	2	2	9	26	25	66	46	176	5.57	1.32	-0.35
Bassi: overall	1	1	7	7	30	74	67	187	5.96	1.11	-0.30

Survey Results Lead to Changes at NERSC

Every year we institute changes based on the previous year survey. In 2007 and early 2008 NERSC took a number of actions in response to suggestions from the 2006 user survey.

1. 2006 user survey: On the 2006 survey four users had concerns about the MOTD not being updated fast enough after status changes or that it was too long.

   NERSC response: The computer and network operations support has streamlined their procedures for managing status changes during outages, thus giving users more current status information both in the MOTD and by email (for users registered to receive status informational emails). The operations staff also increased their knowledge of account support procedures in order to provide better off hours support. The satisfaction score for Computer and network operations (24x7) support had a significant increase of .3 points over last year's score.

2. 2006 user survey: On the 2006 survey a number of users requested longer wall times for the largest machines.

   NERSC response: In January the wall time for Franklin's regular queues was increased from 12 hours to 24, and then to 36 hours in May. The satisfaction score for Franklin queue structure on the 2007/2008 survey was 6.03 out of 7.

3. 2006 user survey: On the 2006 survey a number of users commented on poor reliability for PDSF disks, and the satisfaction score for PDSF disks had the lowest PDSF hardware rating (5.1).

   NERSC response: NERSC has retired over 90 percent of the old NFS disk vaults and has installed new fiber channel based storage with better failover capabilities. In 2007/2008 the satisfaction score for PDSF disks increased to 5.54.

4. 2006 user survey: On the 2006 survey a number of users requested more resources for interactive and debug jobs.

   NERSC response: NERSC now reserves nodes for interactive/debug jobs on weekends; previously this was done Monday through Friday. This change did not change the satisfaction ratings significantly.

5. 2006 user survey: On the 2006 survey users asked that we provide more cycles and Get Franklin online ASAP.

   NERSC response: Franklin was delivered in January and February 2007. Initially installed with Catamount on the compute nodes, NERSC and Cray decided to install, and successfully tested for production use, an early release of Compute Node Linux (CNL). Early users started using Franklin with CNL in July 2007, and all users had access in September. Franklin was accepted in late October, 2007. Since then NERSC and Cray have worked together to improve system stability, I/O performance and the user environment. In July 2008 NERSC and Cray began upgrading Franklin's compute nodes from dual core to quad core, doubling both the number of cores and the amount of memory. We are looking forward to future enhancements, such as integrating the compute nodes with the NERSC Global Filesystem.

Users are invited to provide overall comments about NERSC:

150 users answered the question What does NERSC do well?  

• 103 respondents stated that NERSC gives them access to powerful computing resources without which they could not do their science;
• 59 mentioned excellent support services and NERSC's responsive staff;
• 24 highlighted good software support or an easy to use user environment;
• 17 pointed to good queue management or job turnaround;
• 15 mentioned data services (HPSS, large disk space, purge policy, NGF, data analysis).

Some representative comments are:

 

The NERSC facility is fantastic. I'm very pleased with the hardware available, the people, the help, and the queues.

NERSC is generally excellent, and has both leadership computing power and good ease of use, increasing productivity. This is most of all because the staff are very responsive to user needs and are effective in making leadership class machines work well for user applications. Additionally, the queue structure is clear and usable, the networking is very good, and the storage resources are adequate for large jobs. The analytics and visualization programs and associated software support are very important.

Good computing. Good storage. We always need more.

What NERSC is best at is the combination of large-scale computing facilities with more flexible queuing policies than in other comparable facilities. Also the existence of "small-scale supercomputers" (Jacquard) is very useful to make tests.

NERSC is excellent. Franklin is a great resource - lots of cores. The waiting of queues for large core runs is very nice. [Obviously there is a wait time for 16384 core run for 36 hours :) ]

NERSC does customer service very well. I am always pleased whenever I deal with NERSC. I would also say that NERSC's infrastructure for users is very helpful.

108 users responded to What should NERSC do differently?.

The top three areas of concern were Franklin stability and performance, job scheduling and resource allocation policies, and the need for more or different hardware resources. NERSC will analyze these comments and implement changes where possible over the next year.

Some of the comments from this section are:

It would be great if NERSC could magically improve the stability of Franklin... Unfortunately, hardware failures increase with the size and complexity of the system.

Need to improve network and load management on the log in nodes for Franklin. At times it is very difficult to get any work done since the response time is so slow.

Providing for long serial queues (~12 hours) and enabling these for applications such as IDL would further improve the usefulness of Franklin in particular. We appreciate your efforts to do this and look forward to finding a solution with you soon.

Less emphasis on INCITE, special users. More emphasis on providing high throughput for production applications.

As computing clusters grow, it would be very interesting/helpful for NERSC to invest in robust queuing systems such as Google's MapReduce model. It seems that all of NERSC's clusters are based upon the premise that failures are abnormal and can be dealt with as a special case. As clusters and job sizes grow, single point failures can really mess up a massively parallel job (Franklin) or a large number of parallel jobs (bad nodes draining queues on PDSF). Companies like Google have succeeded with their computing clusters by starting with the premise that hardware failures will happen regularly and building queuing systems that can automatically heal, rather than relying upon the users to notice that jobs are failing, stop them, alert the help system, wait for a fix, and then resubmit jobs.

I would suggest doing more to discourage single node and small jobs

NERSC's seaborg was a great success because of its reliability and its large amount of per-node memory. That led to the situations that majority of scientific codes ran well on it. The future computer (NERSC6) shall have a configuration with large amount of per-node memory (similar to bassi or larger, but with larger amount CPUs than bassi has).

Bassi has become so busy as to be almost useless to me.

NERSC should tell more about their strategic plans. Hopefully in three years we will be operating differently than we are now (command line submission, manual data management etc.) Is NERSC going to actively help with this, or simply be a resource provider? Is NERSC going to help campaign to get better performance and resiliency tools (fault tolerance) actually put into production vs being left as academic demos?

More disk space to users. The whole point of having a LARGE cluster is to do LARGE simulations. That means LARGE amounts of data. We should get more storage space (individually).

104 users answered the question How does NERSC compare to other centers you have used?   61 users stated that NERSC was an excellent center or was better than other centers they have used. Reasons given for preferring NERSC include its consulting services and responsiveness, its security, and its queue management.

25 users said that NERSC was comparable to other centers or gave a mixed review and 11 said that NERSC was not as good as another center they had used. Some users expressed dissatisfaction with user support, with available disk space or with queue turnaround time.

 

Here are the survey results:

1. Respondent Demographics
2. Overall Satisfaction and Importance
3. All Satisfaction, Importance and Usefulness Ratings
4. Hardware Resources
5. Software
6. Visualization and Data Analysis
7. HPC Consulting
8. Services and Communications
9. Web Interfaces
10. Comments about NERSC

Respondents by DOE Office and User Role:

Office Respondents Percent ASCR 53 11.3% BER 55 11.8% BES 133 28.5% FES 64 13.7% HEP 58 12.4% NP 104 22.3%

User Role Number Percent Principal Investigators 71 15.2% PI Proxies 63 13.5% Project Managers 7 1.5% Users 326 69.8%

 

Respondents by Organization:

Organization Type Number Percent Universities 275 58.9% DOE Labs 145 31.0% Other Govt Labs 26 5.6% Industry 16 3.4% Private labs 5 1.1% Organization Number Percent Berkeley Lab 70 15.0% UC Berkeley 30 6.4% Oak Ridge 14 3.0% PNNL 12 2.6% NREL 11 2.4% Tech-X Corp 11 2.4% U. Washington 11 2.4% U. Wisconsin 11 2.4% UC Davis 11 2.4% NCAR 8 1.7% U. Tennessee 8 1.7% Argonne 6 1.3% Livermore 6 1.3% PPPL 6 1.3% U. Colorado 6 1.3% U. Maryland 6 1.3% SLAC 5 1.1% Colorado State 5 1.1% Ohio State 5 1.1% Stanford 5 1.1% Texas A&M 5 1.1% U. Chicago 5 1.1% U. Illinois 5 1.1% U. Oklahoma 5 1.1% Vanderbilt 5 1.1%

Organization Number Percent Brookhaven 4 0.9% Auburn Univ 4 0.9% MIT 4 0.9% Rice University 4 0.9% U. Michigan 4 0.9% UC Irvine 4 0.9% UCLA 4 0.9% APC Lab Astro France 3 0.6% Cal Tech 3 0.6% Georgia State 3 0.6% Harvard 6 2.3% Jefferson Lab 3 0.6% Los Alamos Lab 3 0.6% Louisiana State 3 0.6% Northwestern 3 0.6% Princeton 3 0.6% U. Texas 3 0.6% UC Santa Barbara 3 0.6% William & Mary 3 0.6% NASA GISS 3 0.6% Other Universities 95 20.3% Other Gov. Labs 15 3.2% Other DOE Labs 5 1.1% Other Industry 5 1.1% Private labs 5 1.1%

 

Which NERSC resources do you use?

Resource	Responses	Percent	Num who answered questions on this topic	Percent
NERSC Information Management (NIM) System	280	60.0%	363	77.7%
NERSC web site (www.nersc.gov)	278	59.5%	383	82.0%
Cray XT4 Franklin	268	57.4%	293	62.7%
IBM POWER5 Bassi	214	45.8%	225	48.2%
Linux Cluster Jacquard	170	36.4%	185	39.6%
HPSS Mass Storage System	164	35.1%	195	41.8
Consulting services	164	35.1%	362	77.5%
IBM POWER3 Seaborg (now retired)	147	31.5%	168	36.0%
Account support services	127	27.2%	356	76.2%
PDSF Cluster	75	16.1%	95	20.3
DaVinci	66	14.1%	115	24.6%
Off-hours 24x7 Computer and Network Operations support	47	10.1%	161	34.5%
NERSC Global Filesystem (NGF)	24	5.1%	73	15.6%
Visualization services	15	3.2%	71	15.2%
NERSC CVS server	11	2.4%	94	20.1%
Grid services	8	1.7%	42	9.0%

 

How long have you used NERSC?

Time	Number	Percent
less than 6 months	89	19.4%
6 months - 3 years	199	43.4%
more than 3 years	171	37.3%

 

What desktop systems do you use to connect to NERSC?

System	Responses
Unix Total	347
PC Total	232
Mac Total	207
Linux	316
OS X	206
Windows XP	181
Windows Vista	41
Sun Solaris	23
Windows 2000	10
IBM AIX	3
HP HPUX	2
MacOS	1
SGI IRIX	1

 

Web Browser Used to Take Survey:

Browser	Number	Percent
Firefox 2	202	43.3%
Safari	79	16.9%
MSIE 7	54	11.6%
Firefox 3	52	11.1%
Firefox 1	35	7.5%
Mozilla	25	5.4%
MSIE 6	17	3.6%
Opera	3	0.6%

 

Operating System Used to Take Survey:

OS	Number	Percent
Mac OS X	170	36.4%
Linux	138	29.6%
Windows XP	128	27.4%
Windows Vista	20	4.3%
Windows Server 2003	5	1.1%
Windows 2000	3	0.6%
SunOS	3	0.6%

Overall Satisfaction and Importance

• Legend
• Overall Satisfaction with NERSC
• How important to you is?
• General Comments about NERSC

 

Legend:

Satisfaction Average Score Very Satisfied 6.50 - 7.00 Mostly Satisfied - High 6.00 - 6.49 Mostly Satisfied - Low 5.50 - 5.99 Somewhat Satisfied 4.50 - 5.49	Importance Average Score Very Important 2.50 - 3.00 Somewhat Important 1.50 - 2.49
Significance of Change significant increase not significant

 

Overall Satisfaction with NERSC

7=Very satisfied, 6=Mostly satisfied, 5=Somewhat satisfied, 4=Neutral, 3=Somewhat dissatisfied, 2=Mostly dissatisfied, 1=Very dissatisfied

Item	Num who rated this item as:							Total Responses	Average Score	Std. Dev.	Change from 2006
	1	2	3	4	5	6	7
OVERALL: Consulting and Support Services			3	9	13	91	310	426	6.63	0.71	0.11
NERSC security		2	3	34	18	98	246	401	6.36	1.01	0.05
OVERALL: Satisfaction with NERSC	1	4	3	11	36	172	220	447	6.30	0.92	-0.01
OVERALL: Available Software			3	19	43	157	176	398	6.22	0.87	0.24
OVERALL: Software management and configuration	1	1	3	28	28	148	174	383	6.19	0.98	0.14
OVERALL: Network connectivity	2	6	8	25	33	154	196	424	6.13	1.13	-0.14
OVERALL: Available Computing Hardware	1	7	8	13	44	178	180	431	6.12	1.05	-0.06
OVERALL: Mass storage facilities	4	1	11	38	19	98	173	344	6.06	1.28	-0.10
OVERALL: Hardware management and configuration	3	2	10	32	39	164	146	396	5.97	1.14	-0.09
OVERALL: Data analysis and visualization facilities			6	68	28	67	62	231	5.48	1.24	0.11

 

How important to you is?

3=Very, 2=Somewhat, 1=Not important

Item	Num who rated this item as:			Total Responses	Average Score	Std. Dev.
	1	2	3
OVERALL: Available Computing Hardware		47	347	394	2.88	0.32
OVERALL: Satisfaction with NERSC	2	59	359	420	2.85	0.37
OVERALL: Network connectivity	2	90	295	387	2.76	0.44
OVERALL: Consulting and Support Services	6	89	304	399	2.75	0.47
OVERALL: Hardware management and configuration	14	132	221	367	2.56	0.57
OVERALL: Available Software	23	130	219	372	2.53	0.61
OVERALL: Software management and configuration	23	144	189	356	2.47	0.62
NERSC security	45	154	168	367	2.34	0.69
OVERALL: Mass storage facilities	53	129	162	344	2.32	0.73
OVERALL: Data analysis and visualization facilities	117	90	79	286	1.87	0.82

 

General Comments about NERSC:   87 responses

• 44 Comments about NERSC services / happy with NERSC
• 27 Franklin comments
• 11 Bassi comments
• 5 Network comments
• 4 Charging and Allocations comments
• 3 disk quota comments
• 3 PDSF comments
• 3 Security comments
• 3 Software comments
• 8 other comments

 

Comments about NERSC services / happy with NERSC:   44 responses

NERSC systems seem very well managed. Job turnaround time is excellent, much better than other systems that I've used, even for very large jobs. Web site information is also terrific. I have no problem finding information about software, queue policies, etc. for each machine. Keep up the good work!

NERSC continues to be user oriented, with quick responses to problems.

... From an overall satisfaction, "extremely satisfied" was not an option. I would have chosen it if it were. The staff of this facility put the High Performance in HPC.

NERSC is an invaluable resource to the completion of my doctoral dissertation research. The consulting and web documentation is amazing. Smooth sailing and reliable from start to finish. ...

The process for requesting a start-up account was very straight-forward, and I received a very prompt reply.

NERSC is a model for all supercomputer centers.

Being at a small college without many resources, access makes the difference between having a great research programme and barely getting by.

NERSC is one of the finest computing facilities that I have used in the US and abroad. The level of support is unmatched.

I think they are doing a super job. I can not think of any other place better than this one. ...

I am utilizing a fair amount of the new resources. I am deeply impressed with NERSC programs and how they cater to the user. First class!!!

In general, I have found NERSC to be very responsive to complaints.

... I have always received great support over the phone and by email.

Relative to other high-performance computing facilities that I use, I feel that NERSC is well managed and does a very good job with technical support and computer usage policies. ...

Overall I think that NERSC is a very well-run computing center (compared to others with which I have had experience). The people are very helpful and informative, the website is well-designed, with important information that is easy to find. The tutorials at NERSC or LBL are very useful. ... Overall, though, good job.

In general NERSC has the best user support and reliability of all the other computer resources that I have ever used, ...

NERSC continues to provide a superior level of technical expertise and user service on its consulting and hardware and software management staffs. ...

NERSC is an excellent center in terms of how it is run, and the services and resources that it provides to users. ...

Reliable, well-managed, essential in my research.

During the period where the HPSS system was having problems storing large numbers of relatively small files, I found the support staff to be very helpful and proactive

NERSC has very very _very_ professional administration and user services. I am very impressed with their quality of service and with the effort and dedication that their staff puts into making the clusters easy to use and making lots of useful software available. I worked a summer at LLNL and found LLNL's system administration pathetic by comparison.

You should add another tier in the available choices for response called "extremely satisfied" so I can check this instead of just the "Very satisfied" that exists right now. I am extremely satisfied with the professional way that NERSC is handling the user community and the immediate response to any problems that arise from time to time.

... We get a lot of help in the material science area from user service.

Excellent consulting service. Many thanks. And particular thanks to Zhengji!!!

Account support uniformly splendid; consulting also excellent with one exception.

Very good consulting and account support services. ...

I find the NERSC people quick to help and solve problems. Thanks,

good job. thanks.

NERSC is an unparalleled facility with excellent support and a commitment to making the machines work well for user applications that makes it very productive. The new machine Franklin is coming on line well for us. The visualization group is very helpful, and we appreciate NERSC staff efforts to help us with allowing visualization codes to run on the new platform.

Thanks very much for access to your wonderful facilities. It is very useful to my work

Better than ORNL!

Really great computing facilities and user support services!

NERSC's stability and level of readiness make it my preferred site for model development. ...

Overall I have been very pleased with the resources and service available at NERSC. I've been particularly pleased with how well NERSC has managed the roll-out of Franklin which, like all new hardware has had a few bumps, but the staff has been very good about making quick fixes and communicating the status to the users. I've also had very good telephone support for account issues. I also like the on-line queuing page for Franklin.

I am very grateful for everyone at NERSC for keeping this great facility of great scientific importance to be functional, reachable, and above all, easily usable. Best regards to you all. [Jacquard user]

NERSC is amazingly well run, and has a fantastic staff. Having worked with NCCS in the past year, I appreciate NERSC even more.

NERSC continues to be a reliable place to get my computational work done

NERSC remains a great resource for scientific computing, and our group relies on it very much for computational research. ...

good to have such a powerful facility available.

I am very happy with the opportunity to use the NERSC facilities. The access to large scale computing facility been extremely important to my work.

NERSC is an excellent means of advanced computing. I have been connected since the inception in one way or another, starting with John Killeen. I am very satisfied and very grateful for this professionally run facility.

Keep up the good work.

... Very good computer sources to do computationally expensive jobs.

NERSC is undoubtedly one of the most important resource available to thousands of researches in various disciplines of science and engineering. For state-of the art research in certain areas of pure and applied scientific research, NERSC is sine quo non as it alone can offer memory, disk space and CPU run time which are unmatched anywhere else. I my computational research in superheavy chemistry , we have run jobs which I could not have run elsewhere.
I realize that there are limitations at present on the parameters mentioned above, but I am sure these would be modified in future to meet the needs of the ever increasing community of users at NERSC. We are grateful to the DOE for offering this wonderful NERSC facility for research to the world-wide users.

... Overall, this has been a VERY useful HPC resource!!

 

Franklin comments:   27 responses

Franklin reliability

Hardware stability is a big issue. Please consider stable machines and options when upgrade new machines. ...

The transition from Seaborg to Franklin has been more difficult than I would have expected, mostly because of Franklin going down so often. This appears to have been fixed, so I would expect my answer of "somewhat satisfied" on Hardware management and configuration to change to "very satisfied" as Franklin achieves the reliability that Seaborg had. ...

It's not difficult to see why many people do not use Franklin -- the worst compute platform of my entire computing career (which started on a Burroughs B5500 in 1964 at the University of Denver). ...

Franklin is down too often...

General reliability of Franklin has been disappointing of late. ...

... However with the phasing out of the IBM Power 5 and its replacement by the Cray XT4, the overall reliability of NERSC systems has taken a definite downturn, with a noticeable negative impact on the productivity of our projects.

Although the queue time is short, Franklin seems unstable, compared to seaborg and bassi.

Franklin in particular has been unacceptably unreliable and hence a lot of CPU hours as well as human time were wasted running jobs that crashed at random. This also made diagnosing errors much more difficult. In addition, the many scheduled and unscheduled maintenance sections makes using this computer very frustrating. Franklin is far less reliable than any other computer cluster or supercomputer that I have used in the past. ...

... But in one area it has been a big disappointment this year: The poor stability and availability of Franklin.

Main problem is stability of the system and frequent unscheduled maintenances!!!

... However, the Franklin system has been a step backwards in terms of reliability. This has contributed to very long queues on Bassi, which is probably serving more types of computations than what was intended when purchased.

Too many outages!

Franklin is great when up. Unfortunately, most of the time it seems to be down. This has severely crippled my research activities this year. I cannot even transfer my data on more reliable systems like Bassi, simply because Franklin is either not up, or crashes during the process.
It's quite obvious that whatever problems Franklin has are of the severe sort, and I don't believe the daily few hour maintenance breaks resolve them. I'd rather see Franklin down for a week or longer period, if that prevented the random daily crashes.

Franklin is not a very stable system. ...

... however, Franklin is always going down and that disrupts the flow of our work. I think once the people in NERSC get Franklin fine tuned, NERSC would go back to the top notch reliability that all its users are familiar to.

... There have been some notable problems with new hardware such as the Franklin machine, but that is somewhat to be expected.

I think that Franklin is still stabilizing, ...

Difficulties using Franklin

The transition from Seaborg to Franklin has been tough! Bassi seems to be very congested as a result of Seaborg being taken down as many users not having ported their code to Franklin. It seems like NERSC could perhaps be a little more proactive in helping people to port to Franklin. In a best case scenario, consultants could offer to take code that compiles and runs on Bassi or Seaborg and port it to Franklin for users.

It takes an extraordinary effort to get my code working optimally on NERSC machines, if even possible, and as a result, I use these machines only as a last resort. I can't tell if this is because the system settings are too conservative or because Cray is not sufficiently interested in usability. Honestly, it took me one day to get my stuff running beautifully on BlueGene/P before the machine was even officially in-service, yet I have not run a single non-trivial job on Franklin except by using settings which render it no more powerful than a 32-node Infiniband cluster.
Possible solutions:
1. Buy a machine for big-memory applications, ie 8 GB per node USABLE memory.
2. Force Cray to create a manual for Franklin which allows a reasonably-intelligent code developer to figure out optimal memory settings without having to monopolize a consultant for multiple days and devote a week to tuning these via a guess-and-check approach.
3. Provide code developers of critical DOE software with time specifically for debugging purposes so they don't have to waste 25% of their allocation, which is supposed to be used for science, on this task.

... Licensing issues with compilers can be frustrating. ...

... And the computation time limit is too short (24hr).

... My only criticism of the franklin is the large reduction in available memory after bassi. For those of us who maintain large suites of codes, a lot of restructuring was required to fit codes onto 1.875 Gb of memory. I do not understand why this is not closer to 2.14 Gb. If people need profiling /debug and large MPI overheads can they be put on a subset of the compute nodes?. The 'result' is more important to me than how fast i got it.

Problems with Franklin performance

At times it is very difficult to work on Franklin. Pulling up an x window is so slow that I try to edit files using emacs without and x window. Even then it can be terribly slow to do anything. I press a key and 30 secs later I get a response from the terminal. I assume this is because so many people are logged into the head node. There needs to be some way of managing or upgrading the resources to make Franklin usable.

... Franklin file system is sometimes very slow. ...

... Franklin sometimes responds very slow especially when the command needs HD access.

My overall rating for NERSC is skewed by the little time that I actually run there, and primarily is a comment on my issues with Franklin [performance variability]. As such, my opinion should be given very little weight. I do not consider myself to be a NERSC user, per se.

Happy with Franklin

... The new machine Franklin is coming on line well for us. ...

In general, I have been very happy computing on Franklin. I was able to get a lot done and when there was any problem the support staff was very responsive. ... I finished a run on Franklin and have been fairly inactive the past several months.

... Otherwise, it's a nice, fast, easy-to-use machine, with sensible scheduling policy - it's easy to be productive if the machine stays up (now that we have a sensible amount of disk space). ...

I have been using NERSC for just three months and I am very satisfied, it is an important computational resource. The results obtained within DFT molecular dynamics simulations in the last month improved a lot my research activity. I am very satisfied of this centre, in particular the CRAY XT4 has been very useful because of the low time that you have to wait before getting a running job.

One nice thing is that I can choose between queues. For example, I can use the debug queue to run a test code in a couple of minutes. If I have to run a code who needs CPU time of hours, then I can use the regular queue.

 

Bassi comments:   11 responses

... The only major issue that arose was the decommissioning of Seaborg. However, once I transferred to Bassi, I never looked back. Bassi is a beautiful architecture. It can get a little crowded at times and I don't get the turnaround on jobs that I had on Seaborg, but it is worth it.

Having access to bassi especially strongly helps my research projects

... but bassi has been a great platform to run on.

... It is also my preferred site for all but my largest simulations, where NERSC's success creates its one weakness: it is so popular that it is hard to get through the job queues.

Waiting queue is too long today..

... Bassi is reliable but the queues are so long that it takes weeks for even a small job to run, therefore rendering it quite unusable.

I am waiting too long on queue

Bassi queue time is so long. ...

The management of INCITE accounts is detrimental to the entire community. There has to be a better way than allowing INCITE accounts to jump to the head of the queue. It means that until INCITE accounts use up their allocations, NERSC is unusable except for low priority computing by the average user.

The waiting time for Bassi is too long. Otherwise, I am satisfied with the service by NERSC team.

... Queue is sometimes too long. Interactive job should run immediately. It may also be a good idea to have interactive or debug queue for large number of processors since we often want to take the timing before the big job. ...

 

Network comments:   5 responses

My experience is biased as I work from France and that makes connection very slow.

... My comment above about network connectivity is less positive as I seem to recall rather slow transfer speeds to some of the NSF sites and FNAL. I was moving many output configurations to other centers. These files tended to be on the order of 6 GB, as best I recall. ...

We run nightly tests which require connectivity to nersc systems. In more than a year, we have had tests fail only once due to lost connectivity to nersc. Excellent job! ...

The data transfer time between the scratch disk on Jacquard and my "home" linux computer is still rather slow. I don't know if it's due primarily to the NERSC side, my side, or a combination of both. ... [University of North Carolina at Asheville user]

I have been only able to achieve upload and download bandwidth of 400KB/s from Rice University using scp or http. This is unacceptably slow. Is there a better way to move data? The problem is not at Rice. Somethings seems to be limiting bandwidth within ESNET.

 

Charging and Allocations comments:   4 responses

... One comment is that maybe they should charge half of what they are charging on Franklin, so that I can use them more frequently, and be more productive.

The main problem we had working with NERSC was the ease with which it was possible to blow through our entire allocation before we realized the we had even done so. This occurred for a number of reasons.
1) The accounting system is not very intuitive and is geared towards mistakes unless you actually read up on it carefully (I don't think many people actually do this). It seems to me that using a node for 1 hour under regular circumstances on the flagship machine should be counted as 1 node-hour and everything else should be adjusted downward from that (except possibly using a high priority queue). The fact that you need to consider an extra 6.5 multiplier when using Franklin is asking for trouble. We won't forget to do it in the future, but we found out about this the hard way.
2) The allocation management seems to be non-existent. There is no indication that you are running down to the end of your allocation or that you are asking for more resources than your allocation can provide. On our MPP2 machine at PNNL you can't get on the queue if you request a job size that overruns your existing allocation but there seems to be no such restriction at NERSC. MPP2 will also provide a warning that you are asking for more than 20% of an existing allocation, which provides another opportunity to check that you may be doing something stupid.

At some point it would be helpful to change the charging units to more up-to-date units such as Franklin CPU hours.

... My only problems is with the charge factors used for machine usage. A charge factor of 6.5 on Franklin means that I use a huge fraction of my allocation on just a few large runs. Although there is a large-scale run reimbursement program in effect, I am currently stuck unable to use NERSC's machines because the reimbursement has not yet been credited. ...

 

Disk quota comments:   3 responses

Hope more quota. [Franklin user]

Disk quota should be increased. [PDSF user]

I am dissatisfied with the disk policies on the NERSC machines. If memory serves, I had a very small (almost useless) allocation in my home directory, and the area where I could work without worrying about hitting quotas would get wiped out after so many days of non-use. [Jacquard user]

 

PDSF comments:   3 responses

I have not used PDSF extensively lately and am rather reluctant to start doing so because of the very large number of hardware (disk) problems. In principle I would want to rely heavily on PDSF for my work in ATLAS, but it needs to become FAR MORE reliable than it has been lately. For me to rely on PDSF, I need to know that PDSF will be available (except in very rare cases) when I need to use it during the days before an important talk, conference etc.

PDSF has seen a change of lead in 2007; the ratings above reflect ratings integrated over the year and thus include both "before" and "after" this change. The new lead is on track to improving my overall satisfaction with NERSC and the specific PDSF configuration.

Disks and network access are sometimes slow.

 

Security comments:   3 responses

The lack of having to use a SecureID or equivalent makes NERSC a much nicer place to use than the competing large computing facilities at ORNL, EMSL, and NCAR.

The 'strong password' requirements are annoying. I tried to change my password about a dozen times, and each time the system spat back something different wrong with it. While I can appreciate wanting a secure system, I think that requiring both lower and upper case, numbers, and a special character, plus a minimum length, plus I forget what else, increases the likelihood users will have to write down the damn thing in order to remember it - which isn't very secure!
The other supercomputing systems I use (NCAR, ORNL) have gone to OTPs which IMHO work well.

I answered neutral to security because I'm not sure how DOE would respond to a "dissatisfied". I am dissatisfied because on 4 occasions our database server has been erroneously blocked by NERSC security without informing us that they were blocking it. On 2 of these occasions it happened on a Friday afternoon before they left for the weekend. I'm dissatisfied not because NERSC isn't secure, but because the security is getting in the way of our work, without basic human cross checks like calling us to ask us about a certain traffic pattern.

 

Software comments:   3 responses

In the old days of newton.nersc.gov it was easy to use Mathematica and Matlab at NERSC. Now there are those problems with fonts that make these too important tools virtually unusable for me.

I want to use HDF5 1.8.0 since it has high level API for fortran. ...

please keep NAG-SMP

 

Other comments:   8 responses

... I was troubled with frequency with which DaVinci was down earlier this year.

Da Vinci is a very important part of my work. This machine works great! HPSS also works great!

Not a big deal, but it would be nice if the HSI command line interface was more like a *nix shell, with file name completion.

... Jacquard is somewhat ok. But queue time is long

1. The Wall time, sometimes, is not enough. Because, some jobs cannot be broken into smaller parts. Further more, asking for to many processors will result in a long queue time. Adding these two together prohibits of using NERSC computers for specific jobs.
2. I have suffered quite frequently from jobs crashing because of malfunction of one the nodes. I had not the time nor the will for keeping track of lost time. I found that quite disappointing.
[Jacquard user]

I am planning to use data analysis services in the future. This is an important feature. NERSC should definitely support data services.

Nearly all of our data analysis was done on DoD supercomputers - primarily because our graphics person was DoD.

My bad scores reflect my bad experience using Jacquard. I took quite some time to set up our software (hotspotter, a genetics program), because the storage was so limited that it was even difficult to compile the program with all libraries without exceeding the disk quota. Even when using the scratch space, not all data (SNP data of all human chromosomes and the corresponding hotspotter results) would fit simultaneously into the allotted quota.
After many hours of work, the system was finally running, and I noticed that on Jaquard I was allotted 4 (!) cores. I have an 8-core Macintosh sitting on my desk with many gigabytes of hard disk space. I figured that my own Mac is more of a supercomputer than what I was being offered at NERSC.
I did send an email about that issue, and admittedly it was explained to me that Jaquard and the queue I used is for low-priority jobs and other systems have more to offer. However, this was not sent to me as an email, instead I had to log in and view the request ticket status. I was not aware of having to log in, so I thought that my request was not being answered. By the time I figured out that I have to log in to view my response, I had already solved my computational tasks using computational resources not involving NERSC.
Suggestions: Old clusters and queues offering only 4 cores should be shut down, they are very discouraging to users. Maybe a small login message that other clusters have more to offer would have avoided this misunderstanding.
Another issue is that in genomics research, just as important as processing through-put is that it is possible to store data on a genome scale. About 50GB would be useful for me. Again, this is easily offered by my desktop computer, so one expects to have at least this amount available on a supercomputer.

All Satisfaction, Importance and Usefulness Ratings

• Legend
• All Satisfaction Topics - by Score
• All Satisfaction Topics - by Number of Responses
• All Importance Topics
• All Usefulness Topics

 

Legend

Satisfaction Average Score Very Satisfied 6.50 - 7.00 Mostly Satisfied - High 6.00 - 6.49 Mostly Satisfied - Low 5.50 - 5.99 Somewhat Satisfied 4.50 - 5.49 Neutral 4.50 - 4.49	Importance Average Score Very Important 2.50 - 3.00 Somewhat Important 1.50 - 2.49
Significance of Change significant increase significant decrease not significant	Usefulness Average Score Very Useful 2.50 - 3.00 Somewhat Useful 1.50 - 2.49

 

All Satisfaction Topics - by Score

7=Very satisfied, 6=Mostly satisfied, 5=Somewhat satisfied, 4=Neutral, 3=Somewhat dissatisfied, 2=Mostly dissatisfied, 1=Very dissatisfied

Item	Num who rated this item as:							Total Responses	Average Score	Std. Dev.	Change from 2006
	1	2	3	4	5	6	7
SERVICES: Account support			2	1	6	82	265	356	6.71	0.57	0.07
NGF: Reliability				1	1	16	47	65	6.68	0.59	0.25
NGF: Uptime				1	1	17	47	66	6.67	0.59	0.32
HPSS: Reliability (data integrity)			1	3	4	29	111	148	6.66	0.70	-0.04
OVERALL: Consulting and Support Services			3	9	13	91	310	426	6.63	0.71	0.11
Network performance within NERSC (e.g. Seaborg to HPSS)				4	4	49	111	168	6.59	0.66	0.06
CONSULT: overall			4	8	7	102	241	362	6.57	0.74	0.10
CONSULT: Timely initial response to consulting questions	1	1	2	4	11	107	229	355	6.55	0.77	-0.02
Jacquard: Uptime (Availability)				4	6	34	82	126	6.54	0.73	-0.04
HPSS: Uptime (Availability)			1	3	6	45	96	151	6.54	0.73	-0.08
Seaborg: Uptime (Availability)				4	7	38	89	138	6.54	0.73	0.30
Bassi: Uptime (Availability)	1		1	5	6	48	122	183	6.54	0.84	0.13
CONSULT: Quality of technical advice		1	1	11	16	103	212	344	6.49	0.79	0.03
CONSULT: Follow-up to initial consulting questions	1		3	13	10	99	212	338	6.48	0.86	-0.01
Seaborg SW: Software environment				3	4	36	57	100	6.47	0.72	0.06
HPSS: Overall satisfaction	1		2	5	7	45	103	163	6.46	0.92	0.08
DaVinci SW: Software environment				2	5	33	49	89	6.45	0.71	0.03
Bassi SW: Software environment			2	4	4	45	77	132	6.45	0.82	0.15
Bassi SW: Fortran compilers			3	3	6	37	75	124	6.44	0.89	-0.09
NGF: Overall				3	5	20	41	69	6.43	0.81	0.05
Seaborg SW: Fortran compilers				2	9	31	53	95	6.42	0.75	-0.03
WEB: Accuracy of information		1	3	10	22	121	199	356	6.40	0.83	-0.02
HPSS: Data transfer rates			2	4	16	39	88	149	6.39	0.88	0.10
WEB: NERSC web site overall (www.nersc.gov)			3	12	16	158	194	383	6.38	0.78	-0.01
DaVinci SW: Fortran compilers			1	3	6	20	42	72	6.38	0.91	0.15
NERSC security		2	3	34	18	98	246	401	6.36	1.01	0.05
CONSULT: Amount of time to resolve your issue	2	1	7	10	21	108	199	348	6.35	1.00	0.09
Jacquard SW: Software environment			2	3	5	41	54	105	6.35	0.85	0.08
SERVICES: Computer and network operations support (24x7)			2	11	9	46	93	161	6.35	0.95	0.31
Seaborg: overall				5	12	55	71	143	6.34	0.78	0.25
GRID: Access and Authentication			2	1	1	15	23	42	6.33	1.00	0.07
FranklinSW: Fortran compilers		1	6	5	12	59	103	186	6.32	1.00
HPSS: Data access time		1	2	4	13	52	77	149	6.31	0.92	0.23
GRID: Job Submission			1	2	2	12	20	37	6.30	1.00	-0.02
OVERALL: Satisfaction with NERSC	1	4	3	11	36	172	220	447	6.30	0.92	-0.01
Seaborg SW: Programming libraries				5	5	33	39	82	6.29	0.84	-0.03
DaVinci SW: Analytics software				3	4	17	24	48	6.29	0.87
On-line help desk	1			14	15	85	115	230	6.29	0.91	0.08
SERVICES: Response to special requests (e.g. disk quota increases, etc.)		1	2	14	9	51	97	174	6.29	1.02	0.04
Jacquard SW: Fortran compilers			2	4	8	29	48	91	6.29	0.96	0.18
NIM		1	6	6	33	148	169	363	6.28	0.86	-0.07
WEB: Timeliness of information		1	7	12	24	135	171	350	6.28	0.92	0.07
PDSF SW: C/C++ compilers			1	4	2	20	28	55	6.27	0.97	-0.15
DaVinci SW: C/C++ compilers	1		1	7	4	7	43	63	6.27	1.30	-0.35
Jacquard: overall	1		2	4	12	49	66	134	6.26	0.98	-0.01
Bassi SW: C/C++ compilers			2	7	3	27	44	83	6.25	1.03	-0.02
Bassi SW: Programming libraries			2	7	6	36	52	103	6.25	0.98	-0.01
PDSF SW: Software environment				5	1	26	25	57	6.25	0.87	0.33
NGF: File and Directory Operations				7	1	23	29	60	6.23	0.96	-0.18
TRAINING: New User's Guide			1	16	16	106	104	243	6.22	0.87	-0.12
OVERALL: Available Software			3	19	43	157	176	398	6.22	0.87	0.24
PDSF SW: Programming libraries				6	2	13	23	44	6.20	1.05	0.37
OVERALL: Software management and configuration	1	1	3	28	28	148	174	383	6.19	0.98	0.14
Franklin SW: Programming libraries		1	4	9	9	74	75	172	6.19	0.99
Jacquard SW: C/C++ compilers		1	2	6	4	23	39	75	6.17	1.16	-0.09
SERVICES: Allocations process	1	1	5	15	26	102	127	277	6.17	1.03	-0.12
Seaborg SW: Applications software	1			4	5	26	29	65	6.17	1.07	-0.01
Jacquard SW: Programming libraries			2	7	3	35	37	84	6.17	1.00	0.01
Franklin SW: Software environment	2	1	3	7	21	80	92	206	6.17	1.06
PDSF SW: Fortran compilers				5	1	7	15	28	6.14	1.15	0.03
Seaborg SW: C/C++ compilers			1	5	5	26	27	64	6.14	0.97	-0.26
TRAINING: Web tutorials		1	1	16	16	97	85	216	6.14	0.93	-0.01
PDSF: Uptime (availability)			2	6	8	25	36	77	6.13	1.06	0.30
OVERALL: Network connectivity	2	6	8	25	33	154	196	424	6.13	1.13	-0.14
OVERALL: Available Computing Hardware	1	7	8	13	44	178	180	431	6.12	1.05	-0.06
GRID: File Transfer	1		2	1	1	18	19	42	6.12	1.27	-0.06
SERVICES: E-mail lists			2	24	21	66	97	210	6.10	1.05	-0.08
DaVinci SW: Visualization software				6	9	17	27	59	6.10	1.01	0.10
PDSF: Overall satisfaction			2	7	8	27	36	80	6.10	1.06	0.29
GRID: Job Monitoring			2	2	4	12	17	37	6.08	1.14	-0.12
Jacquard SW: Applications software			3	6	6	27	32	74	6.07	1.10	-0.02
Franklin SW: C/C++ compilers		1	5	13	7	47	61	134	6.07	1.16
NGF: I/O Bandwidth			3	5	3	28	26	65	6.06	1.09	-0.11
Bassi: Disk configuration and I/O performance			2	18	15	61	67	163	6.06	1.03	-0.02
OVERALL: Mass storage facilities	4	1	11	38	19	98	173	344	6.06	1.28	-0.10
Remote network performance to/from NERSC (e.g. Seaborg to your home institution)		5	11	9	18	69	101	213	6.06	1.25	0.17
WEB: Ease of finding information	1	1	13	11	47	163	137	373	6.05	1.02	0.03
Bassi SW: Applications software	2		2	7	9	29	43	92	6.04	1.27	0.02
CONSULT: Software bug resolution			5	29	20	67	100	221	6.03	1.13	-0.11
Franklin: Batch queue structure	1	2	4	18	28	101	96	250	6.03	1.08
Franklin SW: Applications software		2	3	13	10	51	54	133	6.01	1.15
PDSF SW: Performance and debugging tools			1	4	6	11	17	39	6.00	1.12	0.52
PDSF SW: General tools and utilities		1		7	4	17	22	51	6.00	1.18	0.38
Seaborg: Batch queue structure	1	3	2	8	13	56	51	134	5.99	1.19	0.22
Seaborg: Disk configuration and I/O performance			2	15	15	40	50	122	5.99	1.09	0.07
Jacquard: Disk configuration and I/O performance		1	3	11	10	46	43	114	5.98	1.11	-0.08
Bassi SW: General tools and utilities			1	14	9	26	38	88	5.98	1.13	-0.06
OVERALL: Hardware management and configuration	3	2	10	32	39	164	146	396	5.97	1.14	-0.09
Franklin SW: General tools and utilities		1	5	16	6	66	53	147	5.97	1.12
Bassi: overall	1	1	7	7	30	74	67	187	5.96	1.11	-0.30
HPSS: User interface (hsi, pftp, ftp)	2	1	10	8	14	47	68	150	5.96	1.35	0.13
Seaborg SW: General tools and utilities			2	9	4	26	25	66	5.95	1.13	-0.14
PDSF SW: Applications software				5	6	15	14	40	5.95	1.01	0.09
Seaborg: Ability to run interactively		1	4	15	8	37	48	113	5.95	1.22	0.24
web_acts			1	12	7	22	29	71	5.93	1.15
Jacquard: Ability to run interactively			2	12	15	28	38	95	5.93	1.12	0.16
Jacquard: Batch queue structure	1		4	9	18	49	43	124	5.92	1.13	-0.03
SERVICES: NERSC CVS services				21	6	27	40	94	5.91	1.18	-0.09
Jacquard SW: General tools and utilities			1	11	5	40	22	79	5.90	1.01	-0.27
PDSF: Batch queue structure			1	12	7	22	26	68	5.88	1.15	-0.09
Franklin: Batch wait time	2	1	14	20	30	104	87	258	5.85	1.22
ANALYTICS: Visualization services			1	14	8	24	24	71	5.79	1.16	0.10
Bassi SW: Performance and debugging tools		1	3	10	10	26	25	75	5.76	1.24	-0.01
ANALYTICS: Data analytics software on computational systems	1		2	13	5	27	23	71	5.73	1.30
ANALYTICS: Visualization software on computational systems			1	15	15	24	26	81	5.73	1.14
WEB: Searching	1	2	7	23	27	67	55	182	5.71	1.24	-0.12
PDSF: Batch wait time			4	12	6	28	21	71	5.70	1.22	-0.24
Franklin: overall	1	8	14	14	47	102	76	262	5.70	1.29
Franklin SW: Performance and debugging tools	1	2	6	17	13	48	39	126	5.69	1.32
Seaborg SW: Performance and debugging tools		2	1	8	12	21	18	62	5.66	1.25	-0.03
Bassi: Ability to run interactively	4	2	6	18	23	44	50	147	5.63	1.46	0.08
Jacquard SW: Performance and debugging tools			2	14	9	28	16	69	5.61	1.14	0.02
Jacquard SW: Visualization software				11	4	13	10	38	5.58	1.18	-0.54
Franklin: Ability to run interactively	1	4	10	31	36	51	65	198	5.58	1.37
Bassi: Batch queue structure	2	2	9	26	25	66	46	176	5.57	1.32	-0.35
PDSF: Ability to run interactively		1	7	10	8	24	21	71	5.55	1.38	0.16
PDSF: Disk configuration and I/O performance	1	1	3	9	13	25	17	69	5.54	1.32	0.43
Seaborg: Batch wait time	2	2	8	12	33	47	34	138	5.53	1.32	0.59
OVERALL: Data analysis and visualization facilities			6	68	28	67	62	231	5.48	1.24	0.11
Jacquard: Batch wait time	2	3	13	6	28	40	34	126	5.47	1.46	-0.40
TRAINING: NERSC classes: in-person			2	20	3	8	18	51	5.39	1.42	-0.55
Seaborg SW: Visualization software	1		1	10	4	12	9	37	5.38	1.42	-0.07
Bassi SW: Visualization software			2	16	4	16	11	49	5.37	1.27	0.00
Franklin SW: Visualization software	1		3	19	7	19	16	65	5.34	1.38
TRAINING: Live classes on the web		1		23	6	16	14	60	5.30	1.29	-0.46
Franklin: Disk configuration and I/O performance	8	11	20	36	34	73	51	233	5.15	1.63
Franklin: Uptime (Availability)	7	11	48	12	46	86	47	257	5.04	1.64
Bassi: Batch wait time	11	19	36	16	33	46	22	183	4.46	1.80	-1.39

 

All Satisfaction Topics - by Number of Responses

7=Very satisfied, 6=Mostly satisfied, 5=Somewhat satisfied, 4=Neutral, 3=Somewhat dissatisfied, 2=Mostly dissatisfied, 1=Very dissatisfied

Item	Num who rated this item as:							Total Responses	Average Score	Std. Dev.	Change from 2006
	1	2	3	4	5	6	7
OVERALL: Satisfaction with NERSC	1	4	3	11	36	172	220	447	6.30	0.92	-0.01
OVERALL: Available Computing Hardware	1	7	8	13	44	178	180	431	6.12	1.05	-0.06
OVERALL: Consulting and Support Services			3	9	13	91	310	426	6.63	0.71	0.11
OVERALL: Network connectivity	2	6	8	25	33	154	196	424	6.13	1.13	-0.14
NERSC security		2	3	34	18	98	246	401	6.36	1.01	0.05
OVERALL: Available Software			3	19	43	157	176	398	6.22	0.87	0.24
OVERALL: Hardware management and configuration	3	2	10	32	39	164	146	396	5.97	1.14	-0.09
OVERALL: Software management and configuration	1	1	3	28	28	148	174	383	6.19	0.98	0.14
WEB: NERSC web site overall (www.nersc.gov)			3	12	16	158	194	383	6.38	0.78	-0.01
WEB: Ease of finding information	1	1	13	11	47	163	137	373	6.05	1.02	0.03
NIM		1	6	6	33	148	169	363	6.28	0.86	-0.07
CONSULT: overall			4	8	7	102	241	362	6.57	0.74	0.10
SERVICES: Account support			2	1	6	82	265	356	6.71	0.57	0.07
WEB: Accuracy of information		1	3	10	22	121	199	356	6.40	0.83	-0.02
CONSULT: Timely initial response to consulting questions	1	1	2	4	11	107	229	355	6.55	0.77	-0.02
WEB: Timeliness of information		1	7	12	24	135	171	350	6.28	0.92	0.07
CONSULT: Amount of time to resolve your issue	2	1	7	10	21	108	199	348	6.35	1.00	0.09
CONSULT: Quality of technical advice		1	1	11	16	103	212	344	6.49	0.79	0.03
OVERALL: Mass storage facilities	4	1	11	38	19	98	173	344	6.06	1.28	-0.10
CONSULT: Follow-up to initial consulting questions	1		3	13	10	99	212	338	6.48	0.86	-0.01
SERVICES: Allocations process	1	1	5	15	26	102	127	277	6.17	1.03	-0.12
Franklin: overall	1	8	14	14	47	102	76	262	5.70	1.29
Franklin: Batch wait time	2	1	14	20	30	104	87	258	5.85	1.22
Franklin: Uptime (Availability)	7	11	48	12	46	86	47	257	5.04	1.64
Franklin: Batch queue structure	1	2	4	18	28	101	96	250	6.03	1.08
TRAINING: New User's Guide			1	16	16	106	104	243	6.22	0.87	-0.12
Franklin: Disk configuration and I/O performance	8	11	20	36	34	73	51	233	5.15	1.63
OVERALL: Data analysis and visualization facilities			6	68	28	67	62	231	5.48	1.24	0.11
On-line help desk	1			14	15	85	115	230	6.29	0.91	0.08
CONSULT: Software bug resolution			5	29	20	67	100	221	6.03	1.13	-0.11
TRAINING: Web tutorials		1	1	16	16	97	85	216	6.14	0.93	-0.01
Remote network performance to/from NERSC (e.g. Seaborg to your home institution)		5	11	9	18	69	101	213	6.06	1.25	0.17
SERVICES: E-mail lists			2	24	21	66	97	210	6.10	1.05	-0.08
Franklin SW: Software environment	2	1	3	7	21	80	92	206	6.17	1.06
Franklin: Ability to run interactively	1	4	10	31	36	51	65	198	5.58	1.37
Bassi: overall	1	1	7	7	30	74	67	187	5.96	1.11	-0.30
Franklin SW: Fortran compilers		1	6	5	12	59	103	186	6.32	1.00
Bassi: Batch wait time	11	19	36	16	33	46	22	183	4.46	1.80	-1.39
Bassi: Uptime (Availability)	1		1	5	6	48	122	183	6.54	0.84	0.13
WEB: Searching	1	2	7	23	27	67	55	182	5.71	1.24	-0.12
Bassi: Batch queue structure	2	2	9	26	25	66	46	176	5.57	1.32	-0.35
SERVICES: Response to special requests (e.g. disk quota increases, etc.)		1	2	14	9	51	97	174	6.29	1.02	0.04
Franklin SW: Programming libraries		1	4	9	9	74	75	172	6.19	0.99
Network performance within NERSC (e.g. Seaborg to HPSS)				4	4	49	111	168	6.59	0.66	0.06
Bassi: Disk configuration and I/O performance			2	18	15	61	67	163	6.06	1.03	-0.02
HPSS: Overall satisfaction	1		2	5	7	45	103	163	6.46	0.92	0.08
SERVICES: Computer and network operations support (24x7)			2	11	9	46	93	161	6.35	0.95	0.31
HPSS: Uptime (Availability)			1	3	6	45	96	151	6.54	0.73	-0.08
HPSS: User interface (hsi, pftp, ftp)	2	1	10	8	14	47	68	150	5.96	1.35	0.13
HPSS: Data access time		1	2	4	13	52	77	149	6.31	0.92	0.23
HPSS: Data transfer rates			2	4	16	39	88	149	6.39	0.88	0.10
HPSS: Reliability (data integrity)			1	3	4	29	111	148	6.66	0.70	-0.04
Bassi: Ability to run interactively	4	2	6	18	23	44	50	147	5.63	1.46	0.08
Franklin SW: General tools and utilities		1	5	16	6	66	53	147	5.97	1.12
Seaborg: overall				5	12	55	71	143	6.34	0.78	0.25
Seaborg: Batch wait time	2	2	8	12	33	47	34	138	5.53	1.32	0.59
Seaborg: Uptime (Availability)				4	7	38	89	138	6.54	0.73	0.30
Franklin SW: C/C++ compilers		1	5	13	7	47	61	134	6.07	1.16
Jacquard: overall	1		2	4	12	49	66	134	6.26	0.98	-0.01
Seaborg: Batch queue structure	1	3	2	8	13	56	51	134	5.99	1.19	0.22
Franklin SW: Applications software		2	3	13	10	51	54	133	6.01	1.15
Bassi SW: Software environment			2	4	4	45	77	132	6.45	0.82	0.15
Franklin SW: Performance and debugging tools	1	2	6	17	13	48	39	126	5.69	1.32
Jacquard: Batch wait time	2	3	13	6	28	40	34	126	5.47	1.46	-0.40
Jacquard: Uptime (Availability)				4	6	34	82	126	6.54	0.73	-0.04
Bassi SW: Fortran compilers			3	3	6	37	75	124	6.44	0.89	-0.09
Jacquard: Batch queue structure	1		4	9	18	49	43	124	5.92	1.13	-0.03
Seaborg: Disk configuration and I/O performance			2	15	15	40	50	122	5.99	1.09	0.07
Jacquard: Disk configuration and I/O performance		1	3	11	10	46	43	114	5.98	1.11	-0.08
Seaborg: Ability to run interactively		1	4	15	8	37	48	113	5.95	1.22	0.24
Jacquard SW: Software environment			2	3	5	41	54	105	6.35	0.85	0.08
Bassi SW: Programming libraries			2	7	6	36	52	103	6.25	0.98	-0.01
Seaborg SW: Software environment				3	4	36	57	100	6.47	0.72	0.06
Jacquard: Ability to run interactively			2	12	15	28	38	95	5.93	1.12	0.16
Seaborg SW: Fortran compilers				2	9	31	53	95	6.42	0.75	-0.03
SERVICES: NERSC CVS services				21	6	27	40	94	5.91	1.18	-0.09
Bassi SW: Applications software	2		2	7	9	29	43	92	6.04	1.27	0.02
Jacquard SW: Fortran compilers			2	4	8	29	48	91	6.29	0.96	0.18
DaVinci SW: Software environment				2	5	33	49	89	6.45	0.71	0.03
Bassi SW: General tools and utilities			1	14	9	26	38	88	5.98	1.13	-0.06
Jacquard SW: Programming libraries			2	7	3	35	37	84	6.17	1.00	0.01
Bassi SW: C/C++ compilers			2	7	3	27	44	83	6.25	1.03	-0.02
Seaborg SW: Programming libraries				5	5	33	39	82	6.29	0.84	-0.03
ANALYTICS: Visualization software on computational systems			1	15	15	24	26	81	5.73	1.14
PDSF: Overall satisfaction			2	7	8	27	36	80	6.10	1.06	0.29
Jacquard SW: General tools and utilities			1	11	5	40	22	79	5.90	1.01	-0.27
PDSF: Uptime (availability)			2	6	8	25	36	77	6.13	1.06	0.30
Bassi SW: Performance and debugging tools		1	3	10	10	26	25	75	5.76	1.24	-0.01
Jacquard SW: C/C++ compilers		1	2	6	4	23	39	75	6.17	1.16	-0.09
Jacquard SW: Applications software			3	6	6	27	32	74	6.07	1.10	-0.02
DaVinci SW: Fortran compilers			1	3	6	20	42	72	6.38	0.91	0.15
ANALYTICS: Data analytics software on computational systems	1		2	13	5	27	23	71	5.73	1.30
ANALYTICS: Visualization services			1	14	8	24	24	71	5.79	1.16	0.10
PDSF: Ability to run interactively		1	7	10	8	24	21	71	5.55	1.38	0.16
PDSF: Batch wait time			4	12	6	28	21	71	5.70	1.22	-0.24
WEB: ACTS			1	12	7	22	29	71	5.93	1.15
Jacquard SW: Performance and debugging tools			2	14	9	28	16	69	5.61	1.14	0.02
NGF: Overall				3	5	20	41	69	6.43	0.81	0.05
PDSF: Disk configuration and I/O performance	1	1	3	9	13	25	17	69	5.54	1.32	0.43
PDSF: Batch queue structure			1	12	7	22	26	68	5.88	1.15	-0.09
NGF: Uptime				1	1	17	47	66	6.67	0.59	0.32
Seaborg SW: General tools and utilities			2	9	4	26	25	66	5.95	1.13	-0.14
Franklin SW: Visualization software	1		3	19	7	19	16	65	5.34	1.38
NGF: I/O Bandwidth			3	5	3	28	26	65	6.06	1.09	-0.11
NGF: Reliability				1	1	16	47	65	6.68	0.59	0.25
Seaborg SW: Applications software	1			4	5	26	29	65	6.17	1.07	-0.01
Seaborg SW: C/C++ compilers			1	5	5	26	27	64	6.14	0.97	-0.26
DaVinci SW: C/C++ compilers	1		1	7	4	7	43	63	6.27	1.30	-0.35
Seaborg SW: Performance and debugging tools		2	1	8	12	21	18	62	5.66	1.25	-0.03
NGF: File and Directory Operations				7	1	23	29	60	6.23	0.96	-0.18
TRAINING: Live classes on the web		1		23	6	16	14	60	5.30	1.29	-0.46
DaVinci SW: Visualization software				6	9	17	27	59	6.10	1.01	0.10
PDSF SW: Software environment				5	1	26	25	57	6.25	0.87	0.33
PDSF SW: C/C++ compilers			1	4	2	20	28	55	6.27	0.97	-0.15
PDSF SW: General tools and utilities		1		7	4	17	22	51	6.00	1.18	0.38
TRAINING: NERSC classes: in-person			2	20	3	8	18	51	5.39	1.42	-0.55
Bassi SW: Visualization software			2	16	4	16	11	49	5.37	1.27	0.00
DaVinci SW: Analytics software				3	4	17	24	48	6.29	0.87
PDSF SW: Programming libraries				6	2	13	23	44	6.20	1.05	0.37
GRID: Access and Authentication			2	1	1	15	23	42	6.33	1.00	0.07
GRID: File Transfer	1		2	1	1	18	19	42	6.12	1.27	-0.06
PDSF SW: Applications software				5	6	15	14	40	5.95	1.01	0.09
PDSF SW: Performance and debugging tools			1	4	6	11	17	39	6.00	1.12	0.52
Jacquard SW: Visualization software				11	4	13	10	38	5.58	1.18	-0.54
GRID: Job Monitoring			2	2	4	12	17	37	6.08	1.14	-0.12
GRID: Job Submission			1	2	2	12	20	37	6.30	1.00	-0.02
Seaborg SW: Visualization software	1		1	10	4	12	9	37	5.38	1.42	-0.07
PDSF SW: Fortran compilers				5	1	7	15	28	6.14	1.15	0.03

 

All Importance Topics

Importance Ratings: 3=Very important, 2=Somewhat important, 1=Not important
Satisfaction Ratings: 7=Very satisfied, 6=Mostly satisfied, 5=Somewhat satisfied, 4=Neutral, 3=Somewhat dissatisfied, 2=Mostly dissatisfied, 1=Very dissatisfied

Item	Num who rated this item as:			Total Responses for Importance	Average Importance Score	Std. Dev.	Total Responses for Satisfaction	Average Satisfaction Score	Std. Dev.	Change from 2006
	1	2	3
OVERALL: Available Computing Hardware		47	347	394	2.88	0.32	431	6.12	1.05	-0.06
OVERALL: Satisfaction with NERSC	2	59	359	420	2.85	0.37	447	6.30	0.92	-0.01
OVERALL: Network connectivity	2	90	295	387	2.76	0.44	424	6.13	1.13	-0.14
OVERALL: Consulting and Support Services	6	89	304	399	2.75	0.47	426	6.63	0.71	0.11
SERVICES: Account support	5	76	239	320	2.73	0.48	356	6.71	0.57	0.07
SERVICES: Allocations process	10	61	185	256	2.68	0.54	277	6.17	1.03	-0.12
OVERALL: Hardware management and configuration	14	132	221	367	2.56	0.57	396	5.97	1.14	-0.09
OVERALL: Available Software	23	130	219	372	2.53	0.61	398	6.22	0.87	0.24
SERVICES: Response to special requests (e.g. disk quota increases, etc.)	18	55	118	191	2.52	0.66	174	6.29	1.02	0.04
OVERALL: Software management and configuration	23	144	189	356	2.47	0.62	383	6.19	0.98	0.14
NERSC security	45	154	168	367	2.34	0.69	401	6.36	1.01	0.05
SERVICES: Computer and network operations support (24x7)	25	77	85	187	2.32	0.70	161	6.35	0.95	0.31
OVERALL: Mass storage facilities	53	129	162	344	2.32	0.73	344	6.06	1.28	-0.10
ANALYTICS: Visualization software on computational systems	35	35	41	111	2.05	0.83	81	5.73	1.14
SERVICES: E-mail lists	40	116	51	207	2.05	0.66	210	6.10	1.05	-0.08
ANALYTICS: Visualization services	35	31	38	104	2.03	0.84	71	5.79	1.16	0.10
ANALYTICS: Data analytics software on computational systems	37	30	38	105	2.01	0.85	71	5.73	1.30
SERVICES: NERSC CVS services	43	51	44	138	2.01	0.80	94	5.91	1.18	-0.09
OVERALL: Data analysis and visualization facilities	117	90	79	286	1.87	0.82	231	5.48	1.24	0.11

 

All Usefulness Topics

3=Very useful, 2=Somewhat useful, 1=Not useful

Item	Num who rated this item as:			Total Responses	Average Score	Std. Dev.
	1	2	3
TRAINING: New User's Guide	8	44	155	155	2.71	0.53
SERVICES: E-mail lists	5	99	230	155	2.67	0.50
TRAINING: Web tutorials	13	50	135	198	2.62	0.61
MOTD (Message of the Day)	27	108	173	308	2.47	0.65
SERVICES: Announcements web archive	36	134	130	300	2.31	0.68
Live classes on the web	26	44	28	98	2.02	0.75
TRAINING: NERSC classes: in-person	31	33	29	93	1.98	0.81
Phone calls from NERSC	86	66	75	227	1.95	0.84

Hardware Resources

• Legend
• Hardware Satisfaction - by Score
• Hardware Satisfaction - by Platform
• Hardware Comments

 

Legend:

Satisfaction Average Score Very Satisfied 6.50 - 7.00 Mostly Satisfied - High 6.00 - 6.49 Mostly Satisfied - Low 5.50 - 5.99 Somewhat Satisfied 4.50 - 5.49 Neutral 4.50 - 4.49

Significance of Change significant increase significant decrease not significant

 

Hardware Satisfaction - by Score

7=Very satisfied, 6=Mostly satisfied, 5=Somewhat satisfied, 4=Neutral, 3=Somewhat dissatisfied, 2=Mostly dissatisfied, 1=Very dissatisfied

Item	Num who rated this item as:							Total Responses	Average Score	Std. Dev.	Change from 2006
	1	2	3	4	5	6	7
NGF: Reliability				1	1	16	47	65	6.68	0.59	0.25
NGF: Uptime				1	1	17	47	66	6.67	0.59	0.32
HPSS: Reliability (data integrity)			1	3	4	29	111	148	6.66	0.70	-0.04
Network performance within NERSC (e.g. Seaborg to HPSS)				4	4	49	111	168	6.59	0.66	0.06
Jacquard: Uptime (Availability)				4	6	34	82	126	6.54	0.73	-0.04
HPSS: Uptime (Availability)			1	3	6	45	96	151	6.54	0.73	-0.08
Seaborg: Uptime (Availability)				4	7	38	89	138	6.54	0.73	0.30
Bassi: Uptime (Availability)	1		1	5	6	48	122	183	6.54	0.84	0.13
HPSS: Overall satisfaction	1		2	5	7	45	103	163	6.46	0.92	0.08
NGF: Overall				3	5	20	41	69	6.43	0.81	0.05
HPSS: Data transfer rates			2	4	16	39	88	149	6.39	0.88	0.10
Seaborg: overall				5	12	55	71	143	6.34	0.78	0.25
GRID: Access and Authentication			2	1	1	15	23	42	6.33	1.00	0.07
HPSS: Data access time		1	2	4	13	52	77	149	6.31	0.92	0.23
GRID: Job Submission			1	2	2	12	20	37	6.30	1.00	-0.02
Jacquard: overall	1		2	4	12	49	66	134	6.26	0.98	-0.01
NGF: File and Directory Operations				7	1	23	29	60	6.23	0.96	-0.18
PDSF: Uptime (availability)			2	6	8	25	36	77	6.13	1.06	0.30
GRID: File Transfer	1		2	1	1	18	19	42	6.12	1.27	-0.06
PDSF: Overall satisfaction			2	7	8	27	36	80	6.10	1.06	0.29
GRID: Job Monitoring			2	2	4	12	17	37	6.08	1.14	-0.12
NGF: I/O Bandwidth			3	5	3	28	26	65	6.06	1.09	-0.11
Bassi: Disk configuration and I/O performance			2	18	15	61	67	163	6.06	1.03	-0.02
Remote network performance to/from NERSC (e.g. Seaborg to your home institution)		5	11	9	18	69	101	213	6.06	1.25	0.17
Franklin: Batch queue structure	1	2	4	18	28	101	96	250	6.03	1.08
Seaborg: Batch queue structure	1	3	2	8	13	56	51	134	5.99	1.19	0.22
Seaborg: Disk configuration and I/O performance			2	15	15	40	50	122	5.99	1.09	0.07
Jacquard: Disk configuration and I/O performance		1	3	11	10	46	43	114	5.98	1.11	-0.08
Bassi: overall	1	1	7	7	30	74	67	187	5.96	1.11	-0.30
HPSS: User interface (hsi, pftp, ftp)	2	1	10	8	14	47	68	150	5.96	1.35	0.13
Seaborg: Ability to run interactively		1	4	15	8	37	48	113	5.95	1.22	0.24
Jacquard: Ability to run interactively			2	12	15	28	38	95	5.93	1.12	0.16
Jacquard: Batch queue structure	1		4	9	18	49	43	124	5.92	1.13	-0.03
PDSF: Batch queue structure			1	12	7	22	26	68	5.88	1.15	-0.09
Franklin: Batch wait time	2	1	14	20	30	104	87	258	5.85	1.22
PDSF: Batch wait time			4	12	6	28	21	71	5.70	1.22	-0.24
Franklin: overall	1	8	14	14	47	102	76	262	5.70	1.29
Bassi: Ability to run interactively	4	2	6	18	23	44	50	147	5.63	1.46	0.08
Franklin: Ability to run interactively	1	4	10	31	36	51	65	198	5.58	1.37
Bassi: Batch queue structure	2	2	9	26	25	66	46	176	5.57	1.32	-0.35
PDSF: Ability to run interactively		1	7	10	8	24	21	71	5.55	1.38	0.16
PDSF: Disk configuration and I/O performance	1	1	3	9	13	25	17	69	5.54	1.32	0.43
Seaborg: Batch wait time	2	2	8	12	33	47	34	138	5.53	1.32	0.59
Jacquard: Batch wait time	2	3	13	6	28	40	34	126	5.47	1.46	-0.40
Franklin: Disk configuration and I/O performance	8	11	20	36	34	73	51	233	5.15	1.63
Franklin: Uptime (Availability)	7	11	48	12	46	86	47	257	5.04	1.64
Bassi: Batch wait time	11	19	36	16	33	46	22	183	4.46	1.80	-1.39

 

Hardware Satisfaction - by Platform

7=Very satisfied, 6=Mostly satisfied, 5=Somewhat satisfied, 4=Neutral, 3=Somewhat dissatisfied, 2=Mostly dissatisfied, 1=Very dissatisfied

Item	Num who rated this item as:							Total Responses	Average Score	Std. Dev.	Change from 2006
	1	2	3	4	5	6	7
Bassi - IBM POWER5 p575
Bassi: Uptime (Availability)	1		1	5	6	48	122	183	6.54	0.84	0.13
Bassi: Disk configuration and I/O performance			2	18	15	61	67	163	6.06	1.03	-0.02
Bassi: overall	1	1	7	7	30	74	67	187	5.96	1.11	-0.30
Bassi: Ability to run interactively	4	2	6	18	23	44	50	147	5.63	1.46	0.08
Bassi: Batch queue structure	2	2	9	26	25	66	46	176	5.57	1.32	-0.35
Bassi: Batch wait time	11	19	36	16	33	46	22	183	4.46	1.80	-1.39
Franklin - Cray XT4
Franklin: Batch queue structure	1	2	4	18	28	101	96	250	6.03	1.08
Franklin: Batch wait time	2	1	14	20	30	104	87	258	5.85	1.22
Franklin: overall	1	8	14	14	47	102	76	262	5.70	1.29
Franklin: Ability to run interactively	1	4	10	31	36	51	65	198	5.58	1.37
Franklin: Disk configuration and I/O performance	8	11	20	36	34	73	51	233	5.15	1.63
Franklin: Uptime (Availability)	7	11	48	12	46	86	47	257	5.04	1.64
Grid Services
GRID: Access and Authentication			2	1	1	15	23	42	6.33	1.00	0.07
GRID: Job Submission			1	2	2	12	20	37	6.30	1.00	-0.02
GRID: File Transfer	1		2	1	1	18	19	42	6.12	1.27	-0.06
GRID: Job Monitoring			2	2	4	12	17	37	6.08	1.14	-0.12
HPSS - Mass Storage System
HPSS: Reliability (data integrity)			1	3	4	29	111	148	6.66	0.70	-0.04
HPSS: Uptime (Availability)			1	3	6	45	96	151	6.54	0.73	-0.08
HPSS: Overall satisfaction	1		2	5	7	45	103	163	6.46	0.92	0.08
HPSS: Data transfer rates			2	4	16	39	88	149	6.39	0.88	0.10
HPSS: Data access time		1	2	4	13	52	77	149	6.31	0.92	0.23
HPSS: User interface (hsi, pftp, ftp)	2	1	10	8	14	47	68	150	5.96	1.35	0.13
Jacquard - Opteron/Infiniband Linux Cluster
Jacquard: Uptime (Availability)				4	6	34	82	126	6.54	0.73	-0.04
Jacquard: overall	1		2	4	12	49	66	134	6.26	0.98	-0.01
Jacquard: Disk configuration and I/O performance		1	3	11	10	46	43	114	5.98	1.11	-0.08
Jacquard: Ability to run interactively			2	12	15	28	38	95	5.93	1.12	0.16
Jacquard: Batch queue structure	1		4	9	18	49	43	124	5.92	1.13	-0.03
Jacquard: Batch wait time	2	3	13	6	28	40	34	126	5.47	1.46	-0.40
NERSC Network
Network performance within NERSC (e.g. Seaborg to HPSS)				4	4	49	111	168	6.59	0.66	0.06
Remote network performance to/from NERSC (e.g. Seaborg to your home institution)		5	11	9	18	69	101	213	6.06	1.25	0.17
NGF - NERSC Global Filesystem
NGF: Reliability				1	1	16	47	65	6.68	0.59	0.25
NGF: Uptime				1	1	17	47	66	6.67	0.59	0.32
NGF: Overall				3	5	20	41	69	6.43	0.81	0.05
NGF: File and Directory Operations				7	1	23	29	60	6.23	0.96	-0.18
NGF: I/O Bandwidth			3	5	3	28	26	65	6.06	1.09	-0.11
PDSF - Linux Cluster (Parallel Distributed Systems Facility)
PDSF: Uptime (availability)			2	6	8	25	36	77	6.13	1.06	0.30
PDSF: Overall satisfaction			2	7	8	27	36	80	6.10	1.06	0.29
PDSF: Batch queue structure			1	12	7	22	26	68	5.88	1.15	-0.09
PDSF: Batch wait time			4	12	6	28	21	71	5.70	1.22	-0.24
PDSF: Ability to run interactively		1	7	10	8	24	21	71	5.55	1.38	0.16
PDSF: Disk configuration and I/O performance	1	1	3	9	13	25	17	69	5.54	1.32	0.43
Seaborg - IBM POWER3 (Decommissioned January 2008)
Seaborg: Uptime (Availability)				4	7	38	89	138	6.54	0.73	0.30
Seaborg: overall				5	12	55	71	143	6.34	0.78	0.25
Seaborg: Batch queue structure	1	3	2	8	13	56	51	134	5.99	1.19	0.22
Seaborg: Disk configuration and I/O performance			2	15	15	40	50	122	5.99	1.09	0.07
Seaborg: Ability to run interactively		1	4	15	8	37	48	113	5.95	1.22	0.24
Seaborg: Batch wait time	2	2	8	12	33	47	34	138	5.53	1.32	0.59

 

Hardware Comments:   48 responses

• 13 Franklin Stability comments
• 11 Franklin Performance comments
• 7 Happy with Franklin
• 5 Franklin User Environment comments
• 5 comments about HPSS
• 4 comments about scheduled downs
• 3 comments by Bassi Users
• 2 comments about the NERSC Global Filesystem / shared file storage
• 2 queue configuration and wait comments
• 2 comments on Network Performance
• 2 comments by PDSF Users
• 1 comment by DaVinci Users
• 2 other comments

 

Franklin stability comments:   13 responses

Franklin is not very stable. Now it is better, but any future upgrade might make it unstable again for a long time.

Aside from the Franklin unreliability January-April, the rest of the systems are excellent. The Franklin problem appears to have been solved, overall, but the Lustre file system access can vary by a factor of 10, it seems. If something could be done to reduce the periods of inaccessible /scratch on Franklin, it would be very appreciated.

Things I would like to see improved: ... 2)Franklin going down for "unscheduled maintenance" often. Although, I am happy to see it's up-time has been much improved recently.

The unscheduled downtimes for Franklin are seriously impacting our development. We need to be pushing our codes to 4000+ processors and Franklin is our only resource for this work. I also don't like being told I need to use my cycles in a timely fashion, while also being told that the machine is often down, and when it comes up the queues are deeply loaded.

Jacquard looks much more stable than Franklin

To date, Franklin, the Cray XT4, has not achieved the level of reliability I have come to expect from other NERSC systems. Although I am not certain of the hardware details, the problem seems to be a result of the inadequacy of the LUSTRE file system to deal with the volume of output from many simultaneously running massively parallel executables. This has resulted in slowdowns (on both interactive and batch nodes), unexpected and unrepeatable code failures, corruption of output, and far too many unscheduled system outages. This is especially disappointing given that Franklin is the only operating NERSC machine with the number of nodes necessary to accommodate many MPP users at once.

... Franklin is often nearly unusable. It's only tolerable when I remember that the alternative is ORNL's machine! ...

compared to the other NERSC systems (Seaborg, Bassi, and Jacquard), I am somewhat dissatisfied by the uptime of Franklin, but compared to the other Cray XT3/4 system I have access to (at ORNL), I am quite satisfied with the uptime of Franklin. ...

NERSC remains a great resource for scientific computing, and our group relies on it very much for computational research. However, the Franklin system has been a step backwards in terms of reliability. This has contributed to very long queues on Bassi, which is probably serving more types of computations than what was intended when purchased.

I put "mostly dissatisfied" for the I/O performance on Franklin due to regular failures that we experienced when writing large restart files when running our code. However, when it does work, the I/O speeds are very high.

Franklin is fantastic when it's working.

I have had some data integrity problems on Franklin's /scratch. I also had a problem where I worked for four hours editing code on Franklin, wrote the file many times to disk (in my home directory), and then Franklin went down. When it came up, the file was empty, zero bytes. Very frustrating, but nothing anyone could do for me afterwards.

franklin has a lot of promise to be the new NERSC workhorse (especially after the quadcore upgrade). Hopefully the filesystem will catch up to the rest of the machine soon :-)

 

Franklin performance comments:   11 responses

The Franklin login nodes are painfully slow at times. This is a real show stopper as simple ls can take 3-4 minutes at times.

Franklin's login nodes are not very responsive / slow most of the time.

The command to view the queue on Franklin takes a ridiculous amount of time to execute. Sometimes it's 10 seconds or so. Also the file system access time seems pretty slow on Franklin. I don't know if these two things are related.

Things I would like to see improved: 1)Franklin frequently has slow command line responsiveness. ...

the lustre file system on franklin is problematic. sometimes i have to wait to vi a small text file or do an ls. i have also seen problems where a running job cannot find a file and simply resubmitting the job with no changes works. the login nodes on franklin are often very slow.

(AM CHECKING TO SEE IF FRANKLIN COMMENT) Sometimes response is very slow. For example, the ls command takes few second. This is frustrating.

for the codes I use the Franklin charge factor is twice what it should be in relation to both Bassi and Jacquard.

Our model runs on Franklin and uses a standard NetCDF library for I/O and frequently times out when opening (or, more rarely, closing) a NetCDF file. This causes the model run to terminate. It is very frustrating trying to do production work on Franklin for this reason.

... I am somewhat unhappy with the sometimes large fluctuations in performance on Franklin, in particular in sections of the code that involve significant MPI communication. I suspect this is related to network activity (MPI activity) on the system and/or by the physical location of the compute nodes relative to each other.

I run at NERSC infrequently, and only at the request of some projects with which I work. I found that performance variability on Franklin was quite high the few times that I have run there. I find that this is partly a characteristic of the Cray XT4 and how jobs are scheduled, and also occurs on XT systems elsewhere. My brief experience on Franklin indicated to me that the problem was worse there than elsewhere, possibly due to the nature of the NERSC workload. However, I have not been on the Franklin for a number of months, and do not know whether these issues have been resolved. If/when my projects require that I run there again, I will run my performance variability benchmarks again and report what I see.

Disappointed by the performance and cost of Franklin. It didn't offer a refuge from Seaborg for codes that like a high processor to node ratio. All of those users got pushed onto Bassi. Hopefully the quad processor upgrade will improve this, but only if the OS software too improves.

 

Happy with Franklin:   7 responses

franklin is excellent.

Over all I'm very happy with Franklin. It has allowed our research group to complete scientific projects that were not possible before. ...

... You've done a great job with franklin.

Franklin is fantastic when it's working.

I've been very happy with Franklin which, despite the usual new hardware start-up bumps, has been a great machine for my applications.

franklin has a lot of promise to be the new NERSC workhorse (especially after the quadcore upgrade). ...

I'm very happy with the number of processors and power available on Franklin, but have also found the MPP charge large enough that I've limited my runs more than I'd like in order not blow through my allocation.

 

Franklin User Environment comments:   5 responses

1) would like to write data from large franklin job to /project
2) would like more disk space
3) want old (>month) data automatically shuffled off to hpss

My problem with Franklin is that the nodes don't have local scratch disk, as they do on Jacquard.

Our code relies extensively on Python. Not having shared libraries on the compute nodes of Franklin is a big problem for us, and has been preventing us from porting our code to Franklin so far.

Lack of dynamic loading on Fanklin nodes has prevented our use (and that of others) -- ideally this should be part of the evaluation before machine acquisition.

As ever, we need better performance tools that *really* work when used on 1000s of processors, jobs where we aren't sure where the problems are, where we really need the answers, and when we are *already* in a bad mood. Totalview and ddt are close on the debugging side, but the performance tools are lacking. Make the provider produce a "screencast' showing the performance analysis of a 2000 processor job with lots of mpi. The current vendor tools still have huge problems with the data deluge. The supposedly performing academic tools aren't setup/installed/tested/trusted.

 

HPSS Comments:   5 responses

it would be nice to have a graphical interface to the storage filesystem

Would be nice if hsi had a "copy if file doesn't exist, don't overwrite if it does" flag. Maybe it does and I just can't figure it out.

Our use of NERSC is primarily focused on the storage system. I would like to see a few more tools implemented to assist in archiving.
Specifically I would like to have the md5sum and/or sha1 computed and stored/cached at NERSC and displayable through hsi commands like 'ls' or 'find'. Checking file sizes and timestamps is not sufficient for assuring data integrity, especially since many data files are identical in size, but not in content.
It would also be very helpful to have a rsync-like command implemented in hsi so that changes to a directory can be easily mirrored to NERSC.
Lastly, and somewhat less importantly, to save space I would like to see htar seamlessly implement bzip2 and/or gzip to the resulting tar files. The idx files should point to the correct spot in the compressed file archive. Many forms of data pack better into a single tar file that is compressed instead of compressing each file individually and then taring them up.

htar was a great addition to the hsi interface. ...

HPSS needs a better user interface, such as C, python, perl libraries. When a system can store petabytes of data, it isn't very practical to rely upon tools originally written for semi-interactive use. Users shouldn't have to write parsing wrappers to hsi output just to be able to do an "ls" from within their pipeline code.

 

Comments on scheduled downs:   4 responses

I'd prefer maintenance to occur OUTside of 'normal business hours' (e.g. M-F 8-5)

... I would prefer to have a single morning a week when all the systems are down. For example, scheduled maintenance on bassi, franklin and hpss happen at different times, and I often loose productive morning hours to having one of these systems down. If I knew that there was no point in logging in on Wednesday morning because bassi, then hpss, and franklin were going to be down, I would simply move my work time to later in the day. ...

On one hand, I am very satisfied with the performance and reliability of the hpss systems. On the other hand, what the heck happens during the Tuesday morning close downs...every Tuesday? It always hits at a bad time, and I always wonder if it really needs work that day. If so, that's fine. Just curious.

I think when maintenance is going on, they should keep at least a few login nodes available to retrieve files. One can block the submission or others, but it is possible that one can access files in scratch and home directories. In case, there is a need to block the file access on scratch and home directories, and they can always send a notice to us.

 

Comments by Bassi Users:   3 responses

overall, very pleased

In general very satisfactory uptime and performance.

Time limit for premium class jobs should be increased.

 

Comments about he NERSC Global Filesystem / shared file storage:   2 responses

... I have tried to do my data processing on davinci. To do this, I need to upload data from bassi or franklin to hpss, then download to davinci. This is more time consuming than simply running my data processing scripts on the supers. I don't know if you could have a shared disk between the three machines, but it might be worth discussing.

1) would like to write data from large franklin job to /project ... 3) want old (>month) data automatically shuffled off to hpss

 

Queue configuration and wait comments:   2 responses

Would like more availability for moderate-size jobs (100-500 processor cores). [Franklin/Jacquard user]

CCSM utilizes a small number of processors, but needs to run more or less continuously. We are grateful for the boost in priority that you have given us, but at busy times, your computers are still over loaded, and wait times become unacceptably long. ... [Franklin/Bassi user]

 

Comments on Network Performance:   2 responses

I mentioned in a previous section concern about rate of file transfer. The destinations were often NCSA, FNAL or TACC.

To elaborate on sftp/scp from my home institution to NERSC : It is slow , never more than 450 Kb/sec, which tends to be a factor of 10 slower that the majority of my connections.

 

Comments by PDSF Users:   2 responses

Unfortunately, the PDSF disk arrays appeared not to be very stable during 2007

The integration of new compute hardware in PDSF and its batch systems has been a slow process. Although part of this may (perhaps) be ascribed to the change of lead in 2007, the integration of new hardware needs to be a high priority for NERSC staff.
Sakrejda has been key in particular to maintaining PDSF as a viable system for science in the transition period from one lead to the next. This well exceeded her high dedication level during normal PDSF operations periods. Perhaps a way can be found to recognize this.

 

Comments by DaVinci Users:   1 response

... finally, we've been working Davinci pretty hard lately, and I am extremely thankful for access. If I had to do all this data processing on Franklin, .... well, it wouldn't get done.

 

Other comments:   2 responses

About "Access and authentication", it is not a good way that 3 incorrect inputs of password will fail. Please change the rule. For example, after one fail, reopen the access page only.

NERSC systems are managed much better than the systems I use at other (DoD) sites [Franklin / Jacquard / Bassi user]

Software

• Legend
• Software Satisfaction - by Score
• Software Satisfaction - by Platform
• Software Comments

 

Legend:

Satisfaction Average Score Mostly Satisfied - High 6.00 - 6.49 Mostly Satisfied - Low 5.50 - 5.99 Somewhat Satisfied 4.50 - 5.49

Significance of Change significant decrease not significant

 

Software Satisfaction - by Score

7=Very satisfied, 6=Mostly satisfied, 5=Somewhat satisfied, 4=Neutral, 3=Somewhat dissatisfied, 2=Mostly dissatisfied, 1=Very dissatisfied

Item	Num who rated this item as:							Total Responses	Average Score	Std. Dev.	Change from 2006
	1	2	3	4	5	6	7
Seaborg SW: Software environment				3	4	36	57	100	6.47	0.72	0.06
Bassi SW: Software environment			2	4	4	45	77	132	6.45	0.82	0.15
Bassi SW: Fortran compilers			3	3	6	37	75	124	6.44	0.89	-0.09
Seaborg SW: Fortran compilers				2	9	31	53	95	6.42	0.75	-0.03
Jacquard SW: Software environment			2	3	5	41	54	105	6.35	0.85	0.08
FranklinSW: Fortran compilers		1	6	5	12	59	103	186	6.32	1.00
Seaborg SW: Programming libraries				5	5	33	39	82	6.29	0.84	-0.03
Jacquard SW: Fortran compilers			2	4	8	29	48	91	6.29	0.96	0.18
PDSF SW: C/C++ compilers			1	4	2	20	28	55	6.27	0.97	-0.15
Bassi SW: C/C++ compilers			2	7	3	27	44	83	6.25	1.03	-0.02
Bassi SW: Programming libraries			2	7	6	36	52	103	6.25	0.98	-0.01
PDSF SW: Software environment				5	1	26	25	57	6.25	0.87	0.33
PDSF SW: Programming libraries				6	2	13	23	44	6.20	1.05	0.37
Franklin SW: Programming libraries		1	4	9	9	74	75	172	6.19	0.99
Jacquard SW: C/C++ compilers		1	2	6	4	23	39	75	6.17	1.16	-0.09
Seaborg SW: Applications software	1			4	5	26	29	65	6.17	1.07	-0.01
Jacquard SW: Programming libraries			2	7	3	35	37	84	6.17	1.00	0.01
Franklin SW: Software environment	2	1	3	7	21	80	92	206	6.17	1.06
PDSF SW: Fortran compilers				5	1	7	15	28	6.14	1.15	0.03
Seaborg SW: C/C++ compilers			1	5	5	26	27	64	6.14	0.97	-0.26
Jacquard SW: Applications software			3	6	6	27	32	74	6.07	1.10	-0.02
Franklin SW: C/C++ compilers		1	5	13	7	47	61	134	6.07	1.16
Bassi SW: Applications software	2		2	7	9	29	43	92	6.04	1.27	0.02
Franklin SW: Applications software		2	3	13	10	51	54	133	6.01	1.15
PDSF SW: Performance and debugging tools			1	4	6	11	17	39	6.00	1.12	0.52
PDSF SW: General tools and utilities		1		7	4	17	22	51	6.00	1.18	0.38
Bassi SW: General tools and utilities			1	14	9	26	38	88	5.98	1.13	-0.06
Franklin SW: General tools and utilities		1	5	16	6	66	53	147	5.97	1.12
Seaborg SW: General tools and utilities			2	9	4	26	25	66	5.95	1.13	-0.14
PDSF SW: Applications software				5	6	15	14	40	5.95	1.01	0.09
Jacquard SW: General tools and utilities			1	11	5	40	22	79	5.90	1.01	-0.27
Bassi SW: Performance and debugging tools		1	3	10	10	26	25	75	5.76	1.24	-0.01
Franklin SW: Performance and debugging tools	1	2	6	17	13	48	39	126	5.69	1.32
Seaborg SW: Performance and debugging tools		2	1	8	12	21	18	62	5.66	1.25	-0.03
Jacquard SW: Performance and debugging tools			2	14	9	28	16	69	5.61	1.14	0.02
Jacquard SW: Visualization software				11	4	13	10	38	5.58	1.18	-0.54
Seaborg SW: Visualization software	1		1	10	4	12	9	37	5.38	1.42	-0.07
Bassi SW: Visualization software			2	16	4	16	11	49	5.37	1.27	0.00
Franklin SW: Visualization software	1		3	19	7	19	16	65	5.34	1.38

 

Software Satisfaction - by Platform

7=Very satisfied, 6=Mostly satisfied, 5=Somewhat satisfied, 4=Neutral, 3=Somewhat dissatisfied, 2=Mostly dissatisfied, 1=Very dissatisfied

Item	Num who rated this item as:							Total Responses	Average Score	Std. Dev.	Change from 2006
	1	2	3	4	5	6	7
Franklin - Cray XT4
FranklinSW: Fortran compilers		1	6	5	12	59	103	186	6.32	1.00
Franklin SW: Programming libraries		1	4	9	9	74	75	172	6.19	0.99
Franklin SW: Software environment	2	1	3	7	21	80	92	206	6.17	1.06
Franklin SW: C/C++ compilers		1	5	13	7	47	61	134	6.07	1.16
Franklin SW: Applications software		2	3	13	10	51	54	133	6.01	1.15
Franklin SW: General tools and utilities		1	5	16	6	66	53	147	5.97	1.12
Franklin SW: Performance and debugging tools	1	2	6	17	13	48	39	126	5.69	1.32
Franklin SW: Visualization software	1		3	19	7	19	16	65	5.34	1.38
Bassi - IBM POWER5 p575
Bassi SW: Software environment			2	4	4	45	77	132	6.45	0.82	0.15
Bassi SW: Fortran compilers			3	3	6	37	75	124	6.44	0.89	-0.09
Bassi SW: C/C++ compilers			2	7	3	27	44	83	6.25	1.03	-0.02
Bassi SW: Programming libraries			2	7	6	36	52	103	6.25	0.98	-0.01
Bassi SW: Applications software	2		2	7	9	29	43	92	6.04	1.27	0.02
Bassi SW: General tools and utilities			1	14	9	26	38	88	5.98	1.13	-0.06
Bassi SW: Performance and debugging tools		1	3	10	10	26	25	75	5.76	1.24	-0.01
Bassi SW: Visualization software			2	16	4	16	11	49	5.37	1.27	0.00
Jacquard - Opteron/Infiniband Linux Cluster
Jacquard SW: Software environment			2	3	5	41	54	105	6.35	0.85	0.08
Jacquard SW: Fortran compilers			2	4	8	29	48	91	6.29	0.96	0.18
Jacquard SW: C/C++ compilers		1	2	6	4	23	39	75	6.17	1.16	-0.09
Jacquard SW: Programming libraries			2	7	3	35	37	84	6.17	1.00	0.01
Jacquard SW: Applications software			3	6	6	27	32	74	6.07	1.10	-0.02
Jacquard SW: General tools and utilities			1	11	5	40	22	79	5.90	1.01	-0.27
Jacquard SW: Performance and debugging tools			2	14	9	28	16	69	5.61	1.14	0.02
Jacquard SW: Visualization software				11	4	13	10	38	5.58	1.18	-0.54
PDSF - Linux Cluster (Parallel Distributed Systems Facility)
PDSF SW: C/C++ compilers			1	4	2	20	28	55	6.27	0.97	-0.15
PDSF SW: Software environment				5	1	26	25	57	6.25	0.87	0.33
PDSF SW: Programming libraries				6	2	13	23	44	6.20	1.05	0.37
PDSF SW: Fortran compilers				5	1	7	15	28	6.14	1.15	0.03
PDSF SW: Performance and debugging tools			1	4	6	11	17	39	6.00	1.12	0.52
PDSF SW: General tools and utilities		1		7	4	17	22	51	6.00	1.18	0.38
PDSF SW: Applications software				5	6	15	14	40	5.95	1.01	0.09
Seaborg - IBM POWER3 (Decommissioned January 2008)
Seaborg SW: Software environment				3	4	36	57	100	6.47	0.72	0.06
Seaborg SW: Fortran compilers				2	9	31	53	95	6.42	0.75	-0.03
Seaborg SW: Programming libraries				5	5	33	39	82	6.29	0.84	-0.03
Seaborg SW: Applications software	1			4	5	26	29	65	6.17	1.07	-0.01
Seaborg SW: C/C++ compilers			1	5	5	26	27	64	6.14	0.97	-0.26
Seaborg SW: General tools and utilities			2	9	4	26	25	66	5.95	1.13	-0.14
Seaborg SW: Performance and debugging tools		2	1	8	12	21	18	62	5.66	1.25	-0.03
Seaborg SW: Visualization software	1		1	10	4	12	9	37	5.38	1.42	-0.07

 

Comments about Software:   27 responses

• 9 compiler comments
• 8 (non compiler) comments by Franklin users
• 7 general / cross platform software comments
• 4 comments by Seaborg users
• 2 comments by Bassi Users
• 2 comments by Jacquard Users
• 1 comment by an HPSS user
• 1 comments by a PDSF user
• 1 comment by a DaVinci users

 

Compiler Comments:   9 responses

I think pgi compiler is weak in error message and debug-mode compilation. ...

1. The PGI compilers do not seem nearly as "robust" as the other compilers I have used on x86_64 (gcc, pathscale). I have had several cases where code that works fine with these other compilers causes the PGI compilers to segfault. The OpenMP support is also buggy in my experience. ...

It's amazing how few PathScale compiler licenses are available on Franklin...absolutely astounding. A consultant tells me there's one more purchased, and waiting to be installed. One more...making 3! Yikes. How many users on that machine?

The pathscale compilers frankly aren't F95 compliant. I don't see why the Intel compilers can't be installed on Franklin and Jacquard.

The lack of Intel compilers on franklin is a problem because the Portland compilers cannot give a proper traceback with accompanying line numbers. Also, pgf90 tends to allow mistakes in coding to pass through compilation while ifort catches many of them, assuring more robust code.

My main software frustration at NERSC has been Fortran-compiler incompatibility. One code I use was extended to use parts of Fortran 2003 that PathScale and Portland Group compilers did not support (but Intel did).

A C++ compiler that is stable, standard-compliant, and fast to run would be a big help. specmark executable performance is not an issue really. The two or three percent faster that a binary executes does not matter if it takes all day to compile our applications, and then the compilation ends up breaking and requiring me to spend another two days working around an "internal compiler error". We don't do our compute kernels in C++ anyways, we do those in Fortran.

The ability to switch from the PG environment to the gfortran environment is helpful, because we have encountered several issues with the PG fortran compiler. A complete selection of math libraries for the gfortran environment would make it more useful, however.

 

(non compiler) Comments by Franklin Users:   8 responses

I think NERSC should add a Gaussian license to Franklin. My jobs on bassi wait for one month now.

I could never get CrayPat to work very well.

Some of the issues I have with software on franklin are related to the fact that "vendors" of some programming libraries don't support CRAY architecture --- in particular CFITSIO library had to be patched. Support for visualization on franklin seems to be pretty nonexistent --- I haven't asked for an xmgrace module but it would be nice...

The general environment is excellent. We appreciate efforts to improve analysis code (eg IDL) usability on Franklin, and need to continue these to optimize usefulness of the machine.

Franklin should have a newer gnuplot and nano/pico

Our code relies extensively on Python. Not having shared libraries on the compute nodes of Franklin is a big problem for us, and has been preventing us from porting our code to Franklin so far.

... 2. The fact that franklin has such a minimal OS, and does not support dynamic library loading, means that some complex analysis codes will simply not run on this machine without extensive patching. It would be nice to know more details about whether there are any plans to ever support dynamic loading.
For example, some of our collaborators use a ton of java software to do massive data analysis (yes, I laughed too when I first heard). In order to run this type of thing on franklin currently, I may be stuck having to compile a version of gcj, and then build a wrapper similar to cc/CC/ftn that statically links in all the CNL libraries. On the other hand, if franklin is going to get dynamic loading soon, I will just wait and run native jni code once it is supported.

... I have a weird hang up which happens only on franklin, but cannot figure out why this happens.

 

General / Cross Platform Software Comments:   7 responses

I am wondering about how well supported HDF5 stuff will be with H. Anand having departed....

I was surprised there was no 'xv' tool there.

The only significant software problem I encounter at NERSC is that the Totalview debugger is unstable. Totalview is great when it works, but it has a tendency to hang or die for me.
It would also be nice if more code editing and maintenance utilities could be supported. While I recognize that limited personnel resources mean that not all utilities can be supported, such utilities are valuable to code development.

The information on the website was very useful in getting me started.

I continue to be satisfied in general with the software resources I use at NERSC and with the responsiveness of the software maintainers.

The software applications I need have mostly already been available on the computer, but when I have requested new software installation, the staff has been really quick about doing that.

PETSc is a critical library for our application, so we need to have that library kept up to date as compilers are upgraded or a new release is made. The module system for managing the user software environment is very useful, but it needs to be kept up to date with all of the software and programming libraries that are available. I have noticed several instances where modules were out of date or broken or did not exist for a new library version.

 

Comments by Seaborg Users:   4 responses

I understand the reason why nersc has to remove imsl from seaborg. But, I am not happy about this action.

The OS of Seaborg seems a bit clunky. For example, users can't use the up arrow to get most recent commands, and doesn't do automatic completion.

As earlier, the scope of our project is served well by the current setup at NERSC.

On seaborg, compiling C++ with optimization is very slow.

 

Comments by Bassi Users:   2 responses

It would be nice if Bassi could be made to function like a Linux system in regards to grep and other simple commands.

Simple, light debugging tools like pdbx on seaborg and bassi (and as opposed to totalview) are crucial for those of us who use the NERSC resources over slow-ish connection (e.g. from Europe).

 

Comments by Jacquard Users:   2 responses

Software Jacquard could be updated quarterly to be relatively consistent with what is made up to date in our Linux workstations.

I periodically use support softwares such as Mathematica, or Matlab on Jacquard. I have been having consistent trouble with the font configuration of Mathematica when using load module commands.

 

Comments by HSS Users:   1 response

The HPSS interface options are shockingly bad. e.g. Kamland had to resort to writing hsi wrappers to achieve reasonable performance. The SNfactory wrote a perl module to have a standard interface within perl scripts rather than spawning and parsing hsi and htar calls one by one. Even the interactive interface of hsi doesn't have basic command line editing. htar failures sometimes don't return error codes and leave 0 sized files in the destination locations. NERSC should provide C, C++, perl, and python libraries for HPSS access in addition to htar, hsi, etc. The HPSS hardware seems great, but the ability to access it is terrible.

 

Comments by PDSF Users:   1 response

While GPFS file system performance has been adequate, it is not evident that GPFS is the most cost effective solution and, therefore, that it is best for computing at PDSF. Any future filesystem choice should thus include the unique PDSF requirements in its decision metrics.

 

Comments by DaVinci Users:   1 response

I would like to have tecplot on davinci.

Visualization and Data Analysis

• Legend
• Overall Satisfaction with NERSC
• How important to you is?

 

Legend:

Satisfaction Average Score Mostly Satisfied - High 6.00 - 6.49 Mostly Satisfied - Low 5.50 - 5.99	Importance Average Score Somewhat Important 1.50 - 2.49
Significance of Change not significant

 

Satisfaction with Analytics and Visualization Services

7=Very satisfied, 6=Mostly satisfied, 5=Somewhat satisfied, 4=Neutral, 3=Somewhat dissatisfied, 2=Mostly dissatisfied, 1=Very dissatisfied

Item	Num who rated this item as:							Total Responses	Average Score	Std. Dev.	Change from 2006
	1	2	3	4	5	6	7
DaVinci SW: Software environment				2	5	33	49	89	6.45	0.71	0.03
DaVinci SW: Fortran compilers			1	3	6	20	42	72	6.38	0.91	0.15
DaVinci SW: Analytics software				3	4	17	24	48	6.29	0.87
DaVinci SW: C/C++ compilers	1		1	7	4	7	43	63	6.27	1.30	-0.35
DaVinci SW: Visualization software				6	9	17	27	59	6.10	1.01	0.10
ANALYTICS: Visualization services			1	14	8	24	24	71	5.79	1.16	0.10
ANALYTICS: Data analytics software on computational systems	1		2	13	5	27	23	71	5.73	1.30
ANALYTICS: Visualization software on computational systems			1	15	15	24	26	81	5.73	1.14

How important to you is:

3=Very,2=Somewhat,1=Not important

Item	Num who rated this item as:			Total Responses	Average Score	Std. Dev.
	1	2	3
ANALYTICS: Visualization software on computational systems	35	35	41	111	2.05	0.83
ANALYTICS: Visualization services	35	31	38	104	2.03	0.84
ANALYTICS: Data analytics software on computational systems	37	30	38	105	2.01	0.85

Where do you perform data analysis and visualization of data produced at NERSC?

Location	Responses	Percent
All at NERSC	28	7.2%
Most at NERSC	63	17.4%
Half at NERSC, half elsewhere	63	17.4%
Most elsewhere	98	27.1%
All elsewhere	102	28.2%
I don't need data analysis or visualization	32	8.8%

Are your data analysis and visualization needs being met? In what ways do you make use of NERSC data analysis and visualization resources? In what ways should NERSC add to or improve these resources?

• Yes, data analysis needs are being met: 19 comments
• Requests for additional services: 11 comments
• Don't use (yet) / need training:   7 comments
• Do vis locally: 6 comments
• Network speed is an inhibitor: 4 comments

 

Yes, data analysis needs are being met / software used / positive comments:   19 comments

Emphasis on resources required:

I love using davinci for interactive serial/threaded jobs that need tons of RAM. I have compiled our timestream visualization software there (http://kst.kde.org/), and I can load up many GB's of data and view it interactively.
Some thoughts:
1. As much as I love the performance of Itaniums, running an emulated 32bit version of IDL has its limitations. Not only is the RAM limited to 4GB, but we cannot compile dynamically loaded modules against this IDL without cross-compiling them on i386. Hopefully the "next davinci" will use more "common" processors.
2. Whatever you decide on for the next machine, large amounts of shared memory is REALLY useful.

I intend to use visualization (VORPALVIEW) and analysis tools (primarily MATLAB) at NERSC in the near future. Most important thing in this respect for me is to run MATLAB on very large memory nodes.

Davinci for some number crunching. Interactively on Franklin and Bassi. ...

Data analysis is an integral part of our simulations, and automating it and incorporating it into the workflow is an essential part of making the simulations successful. To do this, we use serial queues to launch analytics and visualization (eg IDL) after the run, which often take ~12 hours requiring a long queue, and then to copy analyzed data elsewhere. We're working with NERSC staff to enable this in the Franklin environment and appreciate their efforts to do so. An alternate approach may be launching jobs & sharing data across machines (e.g. a job on Franklin finishes and commands a job start on Jacquard or Davinci for analysis, with the appropriate use of /project to store data). The visualization group has been very helpful, and we are working with them on these issues as well as databases, new ways of tracking and clustering particles, and 3D parallel visualization (VisIT). Continued expansion and availability of visualization and analytics, in particular of long serial queues (~ 12 hours) is a high priority for us to be productive.

I use Davinci to pull large CFD data files from hpss and do expensive processing to extract complex derived data. Most of my actual visualization is done with home-spun software aware of our special data structures for large, parallel data, or is over extremely reduced subsets transferred to my local desktop and viewed with commercial packages such as tecplot. I rely on the parallel processing, large disks, high-speed transfer to and from Davinci, and the compiler system functioning. I do not use any other paid-for packages.

I've only done a moderate number of runs on NERSC computers so far, and have found it easier to analyze data on a separate system to have a consistent storage location for runs done at NERSC and other sites.

Data analysis needs are mostly being met. Data analysis done at NERSC is done using custom, low level code. Some data analysis code is parallel and run through the regular batch queues while some data analysis code is run through the serial queues.

I processes all my data output from Davinci. My master thesis is exclusively analyzed from Davinci.

Emphasis on software used:

Our main need is for a sufficient number of free idl licenses, and this need is being met (thank you!).

I have exclusively used IDL and my needs are currently being met by this software.

I use IDL extensively ....

I use IDL.

I mainly use IDL at NERSC.

Only use ROOT4STAR. Very satisfied.

I use ROOT at PDSF

To visualize data I primarily use VisIt developed at LLNL which is installed on Davinci.

Very happy with the Vis Groups responsiveness. We use visit a lot. ...

Some basic plotting utilities are good on the science systems; gnuplot, xmgrace, etc. I don't have extensive experience with davinci... yet... but I will be using visit very soon.

Deploying the FreeNX software is big step in the right direction for users sitting on the East Coast. The graphical interactive software is extremely painful to use over a normal X-Windows connection through ssh. The FreeNX connection is very fast but not quite as robust as X-Windows. ...

 

Requests for additional services / problems:   11 comments

Requests for additional software / software capabilities and problems:

... It would be nice to compile visit modules in-line with our big runs on Franklin.

Something like gqview to see previously made plots!

The double precision version of the NCAR Graphics libraries (version 4.3.1 -- the last double precision version that will ever be available) does not work correctly. I, too, was unable to build these libraries from source code on davinci (although I have successfully done this on a wide variety of other platforms). Hence, I have had to use the single precision version of these libraries. This requires many painful changes in my Fortran programs, but it *is* a work-around.

I made a comment about Mathemtica and Matlab, they were very useful at newton.nersc.gov, now I cannot use them because of the font problem, I never could figure out why, and whether it can be fixed.

I need GTK installed on *head* nodes to be able to visualize @ NERSC [Jacquard user]

Latest version of GNUPlot with gd library and pdf, png, jpeg and gif terminals. [Bassi/DaVinci user]

Requests for consulting help/ vis is hard

More support for high use codes to interface with advanced visualization software like NIMROD support for VisIt, etc.

In my field we have a hard time visualizing fields in 4 dimensional space-time.

... I may need NERSC assistance for future applications.

Requests for (more) resources:

... Davinci needs to have access to Franklin /scratch.

davinci can get overloaded, especially if someone is using it for parallel visualization. it would be nice to have more processors.

 

Don't use (yet) / need more information / training:   7 comments

... not sure what the current status of tutorials on moviemaking is but if info is out of date, please update

... In spite of the network speed, I personally lack the training and the time to do advanced visualization so I would need some good tutorials with a data centric focus. I'm willing to learn but don't have much time...

I had no clue that there is such a thing as analysis and visualization services until reading this survey.

I don't usually need extensive analytics or visualization, but that need is also a function of availability. I do primarily electronic structure calculations at NERSC; if there were new and innovative means of analyzing and visualizing the results (derivable from the density matrix, for example) that could be done relatively easily, that might be valuable. However, that depends more on external scientific development than NERSC administration. So, good job with what you've got!

I don't do much advanced data visualization -- most of my analysis is not computationally expensive. But I think visualization is neat stuff that should be supported!

I do not at present use NERSC facilities for visualization; the data analysis I perform there is done with my own software. I was pleased to hear from a colleague this year, however, that DaVinci is now capable of running the AVS network that is our primary means of visualizing our data, and expect to make use of it in the future.

I am not aware of these services. I will check on them

 

Do vis locally:   6 comments

I've been doing my visualization work locally. Maybe I would benefit by using DaVinci, I just never thought of it.

I basically use the NERSC computers for the heavy runs, then copy the data to my desktop for visualization and analysis, so I don't really use the NERSC computers for visualization.

I do all this on my local machine

I perform all my analysis locally on Franklin, but due to firewall limitations at my end I bring all necessary files back to my home computing facility for visualization, etc.

Over 90-95% of our visualization has been elsewhere -- mainly from convenience point of view.

My data analysis involves customized software that I run on my own computers. ...

 

Network speed is an inhibitor:   4 comments

Currently, our local internet connection is slow, but I will use more if network is improved.

The GUI analysis program I typically use on Davinci (VCDAT) suffers badly when there is communication latency between me and the application.

Data analysis mostly done using root/root4star macros and classes. Resulting histograms are usually copied to a local machine for visualization, since root is so $&*! slow when running remotely.

Unfortunately I cannot use any software which needs interactive use of the X window since the network is too slow! Is this our side's problem? Cause I'm in the east coast?

HPC Consulting

Legend:

Satisfaction Average Score Very Satisfied 6.50 - 7.00 Mostly Satisfied - High 6.00 - 6.49

Significance of Change not significant

Satisfaction with HPC Consulting

7=Very satisfied, 6=Mostly satisfied, 5=Somewhat satisfied, 4=Neutral, 3=Somewhat dissatisfied, 2=Mostly dissatisfied, 1=Very dissatisfied

Item	Num who rated this item as:							Total Responses	Average Score	Std. Dev.	Change from 2006
	1	2	3	4	5	6	7
CONSULT: overall			4	8	7	102	240	361	6.57	0.74	0.10
CONSULT: Timely initial response to consulting questions	1	1	2	4	11	107	228	354	6.55	0.77	-0.02
CONSULT: Quality of technical advice		1	1	11	16	103	211	343	6.48	0.79	0.03
CONSULT: Follow-up to initial consulting questions	1		3	13	10	99	211	337	6.48	0.86	-0.01
CONSULT: Amount of time to resolve your issue	2	1	7	10	21	108	198	347	6.35	1.00	0.09
CONSULT: Software bug resolution			5	29	20	67	100	221	6.03	1.13	-0.11

 

Comments about Consulting:   39 responses

• Good service: 26 responses
• Mixed evaluation: 3 responses
• Unhappy: 10 responses

 

Good service:   26 responses

amazing consulting and technical support. Every issue was resolved quickly.

Staff is cheerful and "customer-oriented".

NERSC consulting is AWESOME!!!

We got a lot of help from NERSC consulting, especially in the area of material science simulation. We like to express our thanks for that. Please keep up the good work.

I had two occasions to contact support and received very prompt and helpful replies.

The consulting at NERSC has been excellent. I recently had cause to use a different supercomputer at a different facility, and the tech support there has been atrocious. I often lament that the other facility is not anywhere near as good as NERSC consulting. Keep up the good work!

The NERSC staff has always been extremely helpful

Thank you for another year of excellent support.

I should praise all the staffs there for their previous support on my job.

We have superb consultants as well as technical staff who are really most helpful in resolving problems promptly . Congratulations to you the best job done. Thanks for your timely help and advice.

Whoo hoo Katie Antypas!

They are always very helpful, prompt, and accurate.

Extremely helpful and eager to help the users. They follow up on making sure that the problem is resolved.

very important and helpful.

I always happy with the speedy response from NERSC consultants and their willingness to help.

I am very grateful all the people in the consulting services. They are so helpful whenever I need them. Thank you all.

Great job!

I have found the NERSC consulting services to be very timely and effective.

The consulting group has been very responsive to every inquiry. Very friendly too.

First class!

Excellent! Many thanks in particular to Zhengji.

Consulting services at NERSC are first-rate.

I have been very pleased with the consulting and support I've experienced.

The consultants are always very helpful.

have had very good service from the NERSC consulting staff.

Very nice. It couldn't be better.

he consulting services are very good, especially in comparison to other computing sites.

 

Mixed evaluation:   3 responses

The consulting staff does excellent job addressing all issues. The transition to Franklin has been difficult, though, with the consultant's expertise clearly developing as time progresses. When one is doing an excellent job, it is difficult to improve. But, with additional experience and education on the Franklin system, I think that the consulting support will be even better.

On nearly all NERSC general problems, your consulting staff has been outstanding. However when our model fails, we have a difficult time convincing your consultants that the problem lies with franklin rather than with our model.

I am very happy overall with the responsiveness and expertise of the NERSC consulting staff. I rated myself "somewhat dissatisfied" with the amount of time to resolve my issue only because of a particular yet-to-be-resolved issue with parallel HDF5 on Franklin, though I appreciate the diligence of the consultants in pursuing the issue and suspect that it may ultimately turn out to be a low-level I/O or hardware problem.

 

Unhappy / Would like additional services:   10 responses

Because there are multiple machines at NERSC, I suspect the consultants have a harder job than at other facilities, but my experiences with other facilities are noticeable different. At other facilities, the consultants know the machines like they built them (true in some cases) and ALWAYS have the right answer on the first try. At NERSC, getting the right answer is an iterative process which is not always convergent. There are times it would be good if machine-specific support could be provided by the companies responsible for them. I've had to go directly to Intel, etc. for support in the past, but it has always turned out well because I was talking directly to the people who built the thing that was not working.

Weekend support! Weekend support! Weekend support!

No satisfactory answer was obtained related to the XT_SYMMETRIC_HEAP_SIZE and XT_LINUX_SHMEM_HEAP_SIZE necessary for NWChem, leading to a relatively ineffective use of NWChem. NERSC Consulting should work together with the NWChem developers and the Cray software developers to get a fundamental, full, and comprehensive understanding of the issues related to memory management. I tried to run 1 core/node, effectively giving me twice as much memory, but there is no way to fully utilize the memory (about 3-3.5 Gbyte), only a max of 2 Gbyte can be accomplished.

In general response times are good for an initial query, but often if the consultant doesn't know the answer right away resolving the problem takes a lot of time. For example, I requested code compilation help on Franklin (code compiles fine on Bassi). Exchanges by email with the consultant were not overly helpful. It was necessary to go in person to LBL to get the needed assistance.

As mentioned in my other comment, I was dissatisfied with the Jacquard low-priority batch queue. It was answered to my request ticket, that other clusters have more to offer. However, I was not being told what cluster I should specifically use and how many processors I will have available. Also, because the answer was not emailed but instead stored in the request ticket requiring to log in, I though for many days that my request is being ignored.
Suggestions:
1) Not only tell users what they do wrong, but also tell them specifically what to do instead.
2) Send a copy of the answered request via email. Many of us organize any electronic communication using their email inbox (which allows to for example to store letters in folders) and not some request tracking system that requires to log in.

Quality is a bit hit and miss. Often we notice things broken or incorrect on the system, and we are given a workaround instead of actual repairs/corrections being made.

A response of 'You may want to change this environment variable' is fine. But if you don't know what value to set it to its not as helpful as it could be.

Sometimes consultants did not follow up promptly. There have also been issues that were not resolved.

Sometimes, responses were sent very quickly without really understanding the problem. It almost seemed as if some staff were too worried about their statistics for issues resolved / time.

Services and Communications

• Legend
• Satisfaction with NERSC Services
• How Important are NERSC Services to You?
• How useful are these methods for keeping you informed?
• Are you well informed of changes?
• Comments about Services and Communications

 

Legend:

Satisfaction Average Score Very Satisfied 6.50 - 7.00 Mostly Satisfied - High 6.00 - 6.49 Mostly Satisfied - Low 5.50 - 5.99 Somewhat Satisfied 4.50 - 5.49	Importance Average Score Very Important 2.50 - 3.00 Somewhat Important 1.50 - 2.49
Significance of Change significant increase not significant	Usefulness Average Score Very Useful 2.50 - 3.00 Somewhat Useful 1.50 - 2.49

 

Satisfaction with NERSC Services

7=Very satisfied, 6=Mostly satisfied, 5=Somewhat satisfied, 4=Neutral, 3=Somewhat dissatisfied, 2=Mostly dissatisfied, 1=Very dissatisfied

Item	Num who rated this item as:							Total Responses	Average Score	Std. Dev.	Change from 2006
	1	2	3	4	5	6	7
SERVICES: Account support			2	1	6	82	265	356	6.71	0.57	0.07
SERVICES: Computer and network operations support (24x7)			2	11	9	46	93	161	6.35	0.95	0.31
SERVICES: Response to special requests (e.g. disk quota increases, etc.)		1	2	14	9	51	97	174	6.29	1.02	0.04
TRAINING: New User's Guide			1	16	16	106	104	243	6.22	0.87	-0.12
SERVICES: Allocations process	1	1	5	15	26	102	127	277	6.17	1.03	-0.12
TRAINING: Web tutorials		1	1	16	16	97	85	216	6.14	0.93	-0.01
SERVICES: E-mail lists			2	24	21	66	97	210	6.10	1.05	-0.08
SERVICES: NERSC CVS services				21	6	27	40	94	5.91	1.18	-0.09
TRAINING: NERSC classes: in-person			2	20	3	8	18	51	5.39	1.42	-0.55
Live classes on the web		1		23	6	16	14	60	5.30	1.29	-0.46

 

How Important are NERSC Services to You?

3=Very important, 2=Somewhat important, 1=Not important

Item	Num who rated this item as:			Total Responses	Average Score	Std. Dev.
	1	2	3
SERVICES: Account support	5	76	239	320	2.73	0.48
SERVICES: Allocations process	10	61	185	256	2.68	0.54
SERVICES: Response to special requests (e.g. disk quota increases, etc.)	18	55	118	191	2.52	0.66
SERVICES: Computer and network operations support (24x7)	25	77	85	187	2.32	0.70
SERVICES: E-mail lists	40	116	51	207	2.05	0.66
SERVICES: NERSC CVS services	43	51	44	138	2.01	0.80

 

How useful are these methods for keeping you informed?

3=Very useful, 2=Somewhat useful, 1=Not useful

Item	Num who rated this item as:			Total Responses	Average Score	Std. Dev.
	1	2	3
TRAINING: New User's Guide	8	44	155	207	2.71	0.53
SERVICES: E-mail lists	5	99	230	334	2.67	0.50
TRAINING: Web tutorials	13	50	135	198	2.62	0.61
MOTD (Message of the Day)	27	108	173	308	2.47	0.65
SERVICES: Announcements web archive	36	134	130	300	2.31	0.68
Live classes on the web	26	44	28	98	2.02	0.75
TRAINING: NERSC classes: in-person	31	33	29	93	1.98	0.81
Phone calls from NERSC	86	66	75	227	1.95	0.84

 

Are you well informed of changes?

Do you feel you are adequately informed about NERSC changes?

Answer	Responses	Percent
Yes	362	97.3
No	10	2.7%

Are you aware of major changes at least one month in advance?

Answer	Responses	Percent
Yes	308	88.5%
No	40	11.5%

Are you aware of software changes at least seven days in advance?

Answer	Responses	Percent
Yes	312	92.3%
No	26	7.7%

Are you aware of planned outages 24 hours in advance?

Answer	Responses	Percent
Yes	332	93.5%
No	23	6.5%

 

Comments about Services and Communications:   15 responses

 

Issues with system outages / MOTD / Communication of down times and system changes:   5 responses

I wish that they could send unscheduled maintenance message to us on time too.

Please send an email when machine goes down and up in an unscheduled manner.

When a system goes out, please post the outage on the MOTD as soon as possible. Sometimes the outage is not posted until hours after it started.

If I 'ssh franklin' and its down - a warning message would be good - not me typing in my password to be denied over and over.

While I am generally aware of software changes, small changes to compiler versions are not generally advertised, even though they can sometimes be very important.

 

Training Comments:   5 responses

Sometimes the web tutorials are a little behind the actual configurations. It would also be nice to have some more detailed walkthrough examples, like "how to run your first program" or "how to build a program using X" where X is some popular library like ScaLAPACK.

I even use some of the tutorials, e.g., MPI, in my computational physics class. They are very valuable.

For training/information, I mostly refer to the PDSF faq

I was not aware of the tutorials and classes! I will have to start using them!

Looks like some tools I should make use of.

 

Off hours 24x7 support:   2 responses

The 24/7 support is only somewhat satisfied specifically because of my experiences with PDSF -- better weekend support for PDSF is needed.

Availability of account support from earlier in the day would be helpful for east-coast-based users.

Satisfied:   2 responses

This aspect of NERSC is fantastic.

Overall excellent.

Web Interfaces

Legend:

Satisfaction Average Score Mostly Satisfied - High 6.00 - 6.49 Mostly Satisfied - Low 5.50 - 5.99

Significance of Change not significant

Satisfaction with Web Interfaces

7=Very satisfied, 6=Mostly satisfied, 5=Somewhat satisfied, 4=Neutral, 3=Somewhat dissatisfied, 2=Mostly dissatisfied, 1=Very dissatisfied

Item	Num who rated this item as:							Total Responses	Average Score	Std. Dev.	Change from 2006
	1	2	3	4	5	6	7
WEB: Accuracy of information		1	3	10	22	121	199	356	6.40	0.83	-0.02
WEB: NERSC web site overall (www.nersc.gov)			3	12	16	158	194	383	6.38	0.78	-0.01
On-line help desk	1			14	15	85	115	230	6.29	0.91	0.08
NIM		1	6	6	33	148	169	363	6.28	0.86	-0.07
WEB: Timeliness of information		1	7	12	24	135	171	350	6.28	0.92	0.07
WEB: Ease of finding information	1	1	13	11	47	163	137	373	6.05	1.02	0.03
WEB: ACTS			1	12	7	22	29	71	5.93	1.15
WEB: Searching	1	2	7	23	27	67	55	182	5.71	1.24	-0.12

 

Comments about web interfaces:   24 responses

Comments about NIM:   9 comments

About "NIM web interface", please make any unclear words to link their explanations. Please avoid abbreviation of words.

It took me a little time to learn the NIM interface, but as I use it more I do find it very efficient.

... Nim is very difficult to use for complicated queries on account usage.

I know there is an incantation that will allow me to see how much I was charged for every job run in a specific timeframe, but I keep forgetting it. I would suggest making the NIM webpage show that option in clear English. Also, when I want to change a password, the option shouldn't be buried under a "Actions" button.

After much investigation, I still do not know my allocation of computer hours or how much of it I have used. This may be a problem with my liaison in these matters: Lawrence Buja at NCAR. Lawrence cannot answer these questions for me. (I am not sure why...) It is difficult to proceed with my work not knowing how much resources are left in my account. I cannot believe the online accounting information which shows that I have only used 5% after running CAM3 for 6900 years.

The NIM user interface appears to have been written in 1998 by a Berkeley undergrad as a homework assignment on how to use frames and drop-down menus. Other than that it's great.

NIM is a bit unordered I think and could be laid out better

NIM is horribly slow.

Accounting information needs better/more convenient descriptions. The current descriptions are vague and difficult to interpret, and the keys to the abbreviations are difficult to locate.

Suggestions for improvement /difficulties using:   9 comments

I have found the search facility to be almost useless on the web site. This is something that could be dramatically improved. For example, search on "ftp". With the results, do you know how to send an outbound ftp? ...

software info on website can be difficult to find and/or one must download large PDF files -- not necessarily optimum

Some programming documentation on Franklin is out of date.

The information, at least last time I checked, for using the parallel versions of Molpro on the different systems (e.g. Jacquard, Bassi, etc.) are out of date.

I would add a interactive moderated wiki with detailed instructions on using software and hardware on different platform. Wiki is the future all else is past.

... Any website takes some amount of learning to become comfortable with. At times I have found the locations of some information unintuitive at first, but once I know where it is I'm fine.

I think the NERSC website needs a major upgrade and a new look and feel.

I don't recall using search or ACTS.

PDSF diagnostics and metrics are improving, and this should continue. For example, the batch utilization numbers are presently useful. However, ganglia diagnostics are not.

Good website:   7 comments

I have recently used a different resources at a different supercomputing facility, and can say without doubt that the NERSC website is superior. Very helpful.

The web site is very good. Everything is there.

Excellent web documentation. Everything I know I learned from the we docs.

The software applications documentation has always been up-to-date, i.e. example scripts executed as they were supposed to. ...

The NERSC web site is one of the few that I've seen on computer systems that is easy to navigate, useful for both beginners and for more advanced users, and easy to locate information. No single part is necessarily excellent, but overall it is really impressive.

Great. It gets better all the time.

I liked the clean/simple nersc.gov CSS style so much that I used it for a basis of our group's module documentation website: http://crd.lbl.gov/~kisner/modules/cmb/ So I say, nice work!

Good online help desk:   1 comment

The online help desk has a very nice interface. I like it a lot.

Comments about NERSC

What does NERSC do well?

In their comments:

• 103 users mentioned computational resources or HPC resources for science; of which
  • 25 sited large number of cores, high end, scaling or parallel resources
  • 14 sited Franklin
  • 7 sited the PDSF
  • 7 sited architectural variety
  • 3 sited Bassi
  • 3 sited Jacquard
• 59 mentioned good consulting, staff support and communications;
• 24 mentioned good software support or an easy to use environment;
• 17 queue management or job turnaround;
• 15 data services (HPSS, large disk space, purge policy, NGF, data analysis);
• 14 stability and reliability;
• 9 good networking, access and security
• 7 good documentation and web services;
• 6 allocations and account management;

Their responses have been grouped for display as follows:

• Provides access to multiple HPC resources / is overall a good center
• Hardware and services are good
• Does well at scaling and job management / provides diverse machines
• Provides reliable computing services
• Enables science
• Provides good machines and cycles
• Good support services and staff
• Good software / easy to use environment
• Good data storage environment
• Other comments

What should NERSC do differently?

 

24:	Franklin issues
18:	Change job scheduling / resource allocation policies
16:	Provide more computing resources / new hardware / different acquisition strategy
14:	Allocations and Charging issues
13:	Improve queue turnaround times
13:	More or Better Services
8:	Data Storage and Disk Space Issues
8:	No suggestions / Satisfied
7:	Software issues
3:	Network issues

How does NERSC compare to other centers you have used?

 

61:	NERSC is the best / overall NERSC is better / positive response
25:	NERSC is the same as / mixed response
11:	NERSC is less good / negative response
7:	No comparison made

 

What does NERSC do well?   150 responses

  Provides access to multiple HPC resources / is overall a good center

Hardware management
Software management
User/account support
User communication
NERSC provides excellent computing services that my DOE-BES project needs.

I use NERSC because it is reliable, the cluster and software is well maintained and the consulting services are incredible. This all contributes to me getting my work done effectively.

NERSC is generally excellent, and has both leadership computing power and good ease of use, increasing productivity. This is most of all because the staff are very responsive to user needs and are effective in making leadership class machines work well for user applications. Additionally, the queue structure is clear and usable, the networking is very good, and the storage resources are adequate for large jobs. The analytics and visualization programs and associated software support are very important.

I use NERSC because of the following reasons:
1). Our groups codes compile and run well on these systems.
2). The queues are really reasonable, and I like the opportunity to put jobs at 'low' priority.
3). I like the fact that scratch is large enough to run jobs and it isn't cleaned so that I don't have to mess with backing up all my work continually. One of the Teragrid machines (Lonestar) is really annoying to work on because of this (the fact that their backup system is unreliable doesn't make it any better).

Excellent computational resources. State-of-the art supercomputers with well-maintained software.

NERSC is an important part of the scientific computing resources for the US. Consequently, our hpc performance tools need to be applicable on NERSC platforms.
NERSC in general does things well.

I have been very pleased with the resources and staff at NERSC. I use the computers at NERSC because I am at a new university with few computer facilities and very little support staff. I've found that it's been very easy to use the resources at NERSC and that the queues for most job types are very short. Also, I've found that the staff quickly addressed any questions or software needs I had. Also, I've found the online queue viewer very useful.

I am very satisfied with NERSC in every respect.

The overall support for scientific computing is excellent. This is very important as NERSC machines are ones of those in the country that can be used for solving my scientific problems.

NERSC does a good job with the supercomputers, and I use Bassi because it is fast, works well with the NCAR atmospheric models, and has shorter queue times than the NCAR computers.

NERSC provides continuously active and very effective computational research environment. Thank you.

NERSC is a wonderful supercomputing center. I really like it.

NERSC is doing good job overall

NERSC is a model for the high performance computing centers

NERSC is doing an excellent job; and overall I am very satisfied with all resources it offers.

NERSC has in the past been the best run community computation facility on the planet. Basically it still is, but the INCITE awards kill my ability to productively use NERSC. I can't compute at NERSC until the INCITE awards run out of time.

Top notch computer center, but current situation with Franklin is hurting it badly and needs to be addressed quickly.

all covered in the survey

NERSC manages the resources for High performance computing very well.
NERSC online tutorials are often out-dated.
NERSC is important as the only major open HPC facility in the Country for scientific computing, large scale data analysis and for HPC algorithm development, experimentation and testing.

NERSC has succeeded to keep an open, and yet secure, computing environment at PDSF with access to large storage resources. This, together with the cost effectiveness of the solution, are the reasons that PDSF is of key importance to me.

  NERSC's hardware and services are good

NERSC has excellent, large-scale computing resources that we need for our application. The hardware and the "liveware" are both fantastic.

Capacity and capability computing.
Well-balanced systems (compute/communicate/IO).
Good range of systems.
Responsiveness to user community needs.

1. Availability of great computational resources. Franklin is a fantastic machine, if a bit fragile
2. The classes and tutorials are NERSC are very useful for newcomers to supercomputing.
3. The website is great, and blows away any of the other supercomputing websites that I have used (or tried to use).

NERSC seems to be doing everything well. I compute there because of the resources available, the friendliness of the staff and consultants and their willingness to help when a problem arises. NERSC is a national resource for computational science and a model of how a user facility should be run.

The NERSC facility is fantastic. I'm very pleased with the hardware available, the people, the help, and the queues.

Its very stable. Its faster, good compilers. Nice support structure -consulting, very secure.

NERSC provides sufficient hand holding to cope with our less experienced users while still providing sufficient computational resources to deal with our more demanding calculations.

NERSC provides an excellent resource for DOE researchers. They provide access to cutting edge supercomputers, virtually any software that a researcher desires and has strong user support services.

NERSC does very well at providing production cycles very efficiently. NERSC mainframes are well supported and easy to use from the field. The support services are superior. NERSC has the right mix of mainframes for my computing needs.

Most importantly, NERSC has top-of-the-line computing facilities that, in general, work. Additionally, NERSC has good staff who make an effort to work with the users in a professional manner.

NERSC provides state-of-the-art parallel platforms. The production runs would not be possible without using NERSC resources. The user support is excellent.

I can perform computational work there I can do in very few other places. The user support is usually very good. Software and compilers are adequate for my needs

NERSC provides advanced computing with excellent interaction with users. I have been using NERSC since its inception, starting with John Killeen. I would be lost without it.
The fortran packages are the only problem I have had. My codes were originally built on CRAY, and the transition has not always been ideal.

It's a well managed system and it has very good support.

Overall, my experience with NERSC has been great. The consultants are helpful and respond quickly to questions. The processing speed on Jacquard is adequate to my needs.

The clusters are really good. Well documented usage and info. easily found on their website.

Have received requested time with acceptable levels of prior justification.
Powerful machines.
Good support.

franklin is a very nice machine and is consistent in running my jobs.
consulting is very responsive.

NERSC is important as the only reliable computing resource we have. The consulting team is very good. NERSC environment is pleasant.

Systems are well maintained, user support is good and knowledgeable.

Excellent helpline, excellent access to machines, excellent hardware (Franklin has been a bit too much down)

Except for Franklin, all the machines work all the time. And when they don't, the consultants are always there for support.

I'm a relatively new supercomputer user, using one program (GYRO) developed by other individuals. I've only used Franklin in any serious quantity, and have been generally happy with its performance and availability. The live user support for passwords and technical help has been excellent, effective and very pleasant to interactive with.

The diversity of systems available to the user as well as the information available on - line (batch queue policies, software library information, etc.) are the main aspects of NERSC which we are most pleased with. Furthermore, actions such as the refund of time due to extended Franklin downtime shows how seriously NERSC handles its users.

The resources and technical consulting services are very good. NERSC provides a lot of the computing time required for our large production runs.

I use primarily Franklin, and when up, am very satisfied. When needed, the support has been very helpful and timely.

Despite some difficulties with keeping Franklin on-line consistently, computational resources at NERSC (both hardware and software) are superior and easy to use.

Simulations for STAR are done at NERSC, and all analyses are performed at PDSF. I am very pleased with the response times of trouble tickets.

HPSS storage makes NERSC (PDSF, specifically) a good bang-for-the-buck. PDSFs user support flexibility to adapt to users' special needs is good.

Support staff is very prompt and extremely helpful. The computational resources are excellent and well-maintained.

NERSC provides resources we do not have at our home institution. It offers excellent consulting and services.

  NERSC does well at scaling and job management / provides diverse machines

Flexibility--I can get my jobs done. Few other resources can accommodate the requirements of the jobs I run at NERSC.

What NERSC is best at is the combination of large-scale computing facilities with more flexible queuing policies than in other comparable facilities. Also the existence of "small-scale supercomputers" (Jacquard) is very useful to make tests.

NERSC is excellent. Franklin is a great resource - lots of cores. The waiting of queues for large core runs is very nice. (Obviously there is a wait time for 16384 core run for 36 hours :) )

1. NERSC offers a variety of computing resources (high memory, many nodes, shared memory visualization nodes etc.)- which is extremely useful
2. Tools required for Cosmic Microwave Background (CMB) research are properly installed and configured on some of the machines

Easy to access, different choice to queue. I need to use NERSC to fulfill our DOE project. 90% of my job is done in NERSC.

NERSC has a diverse collection of machines and gives out time readily to medium-scale users in a very fair manner.

The availability of large processor counts with relatively short wait times for scaling studies is the primary reason I use the NERSC systems. The friendliness and availability of the support staff in diagnosing problems make the systems a pleasure to use.

NERSC is the best place for me to do very large jobs due to scalability and processor count on Franklin. NERSC machines are easier to log into than some others.

NERSC is unique in the number and competence of staff it provides to keep its systems running smoothly and to solve problems for its users. Additionally, by providing systems with large numbers of processors without strongly incentivizing jobs that use a majority of these processors, it makes possible routine debugging and production runs, with good throughput, of jobs of intermediate sizes (hundreds to a few thousand processors) that are too large to be run effectively on local clusters but too small to qualify for INCITE-like programs that demand scaling to tens of thousands of processors or more.

Provides large clusters necessary for large computing projects.

Large number of processors for parallel jobs are very useful.

A place like NERSC is the only way one can run SMP jobs.

I mainly use Franklin for solid state electronic structure calculations. The scaling is very good in comparison to other systems and the software is very stable. The large number of nodes enables simulations of system sizes which were not previously possible.

The reason I compute at NERSC is the feasibility of the large-scale computation. As regards this point, I am very satisfied with the current situation of the computing facilities at NERSC.

The availability of cpu time without long queue waits on Franklin is the main factor for us. The fact that the Franklin environment meets all our needs allows us to take advantage of that.

fast queue

great queuing, lots of nodes

The availability of large scale parallel computing with enough resources to have the job done.

it provides large scale computing resources, with a fair batch queue structure.

Parallel computation is essential for doing the type of calculation we are doing. NERSC supplies exactly that.

we work on performance tools and application studies. NERSC provides large-scale systems on which our tools must run and cycles for code studies.

Testing the parallel codes and can test large number of nodes in one go. This help us to know how much scale up we do in our simulation codes. Also NERSC help us to make sure that our code has cross-platform compatibility.

  NERSC provides reliable computing services

NERSC provides high performance computing with good access and reliability.

NERSC is an absolute life saver. NERSC provides a reliable, well maintained, frequently upgraded platform so that I can do my work instead of spending my scant research time on system management and maintenance.

Very good service received so far. I am pleased with the strong stability, high efficiency and easy-to-use of NERSC. These are very important for users who need not to spend a lot of time in those jobs.

I use NERSC because it offers high-end computing resources that perform reliably and are supported well. Our research group requires large computing resources and NERSC provides a large fraction of what we need without any problems.

reliable operation
accessibility

Provides a resource for storage and computing facility while being reliable and easy to use.

  NERSC enables science

NERSC provides computing resources to accomplish by BES science goals.

NERSC is an essential research for successful completion of my DOE (HEP and NP) projects.

Big computer to run molecular dynamics simulations from first principles on large systems (100-300) atoms for simulation time of the order of some ps.

NERSC provides for all my high performance computational needs. It is crucial to my own research activity in nuclear theory.

I am very much satisfied with the fast machines which are necessary to carry out huge catalyst related calculations, especially transition-state search.

I compute at NERSC because our resources are insufficient for the amount of computational chemistry that I want to / need to do.

NERSC offers me the opportunity to try projects that I could not do elsewhere. It has allowed me to remain competitive as an atmospheric scientist and pursue grant proposals that I wouldn't were it not for the NERSC compute capabilities.

I can not get access to the state-of the art world class facility like NERSC anywhere in the world, and this supercomputer facility is sine quo non for my research in chemistry of superheavy elements . I perform one of the most gargantuan ( if not the most gargantuan) relativistic and non-relativistic all-electron Hartree-Fock and Dirac-Fock all- virtual spinor space coupled-cluster calculations for molecules of superheavy elements, which need almost unlimited cpu hrs and huge disk space. This is the raison-d'etre for my use of NERSC facility.

Serves best the broader user community of DOE researchers - the low-end to moderate-end user.

NERSC is very important to our group. We are able to run CCSM only on a small number of clusters.

  Provides good machines and cycles

NERSC provides access to fast, powerful computers for calculations.

I'm pleased with computational capability of Franklin.

keep up the good job and buy more machines. bassi can be extended for example;

Franklin is a great machine.

I have been very happy using Franklin. About the only thing that would make me more happy is having more capacity. NERSC is so important because it has allowed us to get a lot of calculations done in a reasonable period of time.

jacquard is an excellent cluster and is very well supported

I use NERSC as one of several places to do high-performance computing. The hardware is very good--the performance of our codes on NERSC systems is excellent.

I am very satisfied with the NERSC facilities, in particular for the jobs requiring significant memory for computations.

Computing power of the PDSF cluster is very helpful to the amount of calculations required by our research

The new machines are fast and generally stable. The interactive queues are relatively fast.

The computing source is very good.

The calculation is very fast, which makes my work more efficient.

NERSC delivers me computing power like no other computing center I have access to. Without this resource the work our group is doing could not be done in this extent.

NERSC is an important resource for my project, providing a large chunk of our computational needs. However, the XT3/XT4 does not seem ready to serve in a production environment. The MPI especially requires too much tinkering and tuning to be part of a simple usable system.

pdsf is all I use. Very satisfied, except old hardware keep failing.

Franklin is a great machine! We could get incredible results thanks to its power.

Normally I can get results quickly.

Computers are superb.
Allocation times are generous (although we always need more)
NERSC is very important for me because my research is purely computational, therefore having access to excellent supercomputers is definitely a plus.

  Good support services and staff

NERSC does customer service very well. I am always pleased whenever I deal with NERSC. I would also say that NERSC's infrastructure for users is very helpful.

NERSC is doing extremely well on account management. I did not appreciate enough until I started to use other computing centers.

I think the consultants are very good. I like that I don't need a cryptocard to login.

NERSC is very user oriented, has capabilities that fit my needs. The allocations process is not a large burden, and well worth the effort. NERSC is a model for a computational facility.

The allocation process is easier at NERSC for mid-sized projects (several hundred thousand CPU hours).

FABULOUS tech support that's available 24/7 and always very professional, and well-maintained software and configuration. This is more important to me than even having the latest machines. I liked Seaborg because it was always well-maintained, even though it was rather slow.

Great services.

Nersc is an example for all other national labs. They are so friendly and so helpful. They should get a raise.
The reason that I like Nersc is that the environment is friendly. Nobody will push you around and I do not need spend too much time on unnecessary things. It is very productive. It is important to me since I can work on my research that I could never be able to do. And the research has a huge impact. I am very productive, yes very.
Keep on doing your super jobs, and do not forget allocating more cpu time for me!

NERSC is top notch. Consultants are all great. Franklin has had a rough start, but is improving slowly.

Accounts, keeping your users informed, support is very responsive when available. Even getting phone calls out of the blue regarding patterns of my HPSS usage -- I was very impressed with that. Keeping franklin running sounds like quite a job to do, so I am pretty understanding when Lustre eats one of my files.

The process for requesting a start-up account was very straight-forward, and I received a prompt reply.
It was easy to get started on Franklin, thanks primarily to the excellent website.

NERSC provides me quick and kind helps to solve the issues
The batch system is very nice (faster and stable)

I am very happy with NERSC. Tech support is great; the people are very helpful.

The support group do good jobs, I can have instant help when needed.

The support is excellent. All inquiries are addressed in a timely manner.

NERSC is very responsive to our needs. The consultants are knowledgeable and helpful. The systems are generally up and stable.

Generally, I'm pleased with NERSC; I can do good work here and the level of support is excellent (so that I'm able to concentrate mostly on physics issues rather than why-won't-this-computer-compile-or-run-my-code-type issues).

I think the response of the support team is great. Thanks!

Excellent consulting service.

NERSC has a very knowledgeable staff that is user-oriented and very friendly. I enjoy interacting with everybody at NERSC.

Your user "how to" web sites for the different machines is very good. Really, the best I have seen. However, NERSC is just another computer resource for me. I run at computer systems at ORNL where I am located as well.

user consultation is superb. They are very helpful.

NERSC has been important to me as a source of information in new technologies with talks and a careful support.

  Good software / easy to use environment

NERSC provides large scale computers with high reliability, and provides expert software assistance.

NERSC offers a lot of very important software and have a great support team. Without this, my research would become significantly more challenging.

capability to compile and error check fortran codes better than other computers.
It allows us to to simulations to big and time consuming to do on our home computers.

NERSC presents updated software. I use it because my needs of parallel computing, and I find in NERSC all the adequate software, as well as, a good consultant help in order to do it (mainly, when implementing new codes or using new libraries).

I compute at NERSC because the software environment is good and because I often need to run problems 100X bigger than what a desktop can do.
If we ever get reasonable DOE funding, maybe we would run jobs 1000 to 10^4 times larger

The availability of large computing resources (since Franklin's inception) have made NERSC an invaluable resource for my research. I am glad that NERSC has moved away from IBM platforms towards linux clusters. This makes software development much easier and requires less training for students. Without NERSC I could not complete my research

Bassi is very fast. NERSC provide the latest version of VASP.

I mainly use pdsf for dta analysis in STAR collaboration. The starroot software package is well maintained and current, avoiding duplication of work at my home institution (UC Davis).

  Good data storage environment

NERSC is important to be because of PDSF, HPSS and the resources STAR uses for simulation and data analysis.
In addition to providing much needed simulation resources, PDSF (together with HPSS) allow access to the reduced STAR data, and make NERSC/PDSF a viable alternative to BNL/RACF. Since many of our colleagues have a difficult time accessing BNL/RACF, such an alternative is very important for collaboration.

Easy access to a range of different hardware systems, including a common directory (/project) shared between the platforms (and between different users of a repository) so that data are always accessible even if one or another machine is down.
Reliable, not much down time.

I use PDSF for simulations and analysis. It is essential because of its speed and disk resources.

NERSC (PDSF) is one only two centres supporting STAR computing where collaborators can get an account ie Tier 0 or 1. RCF at BNL is very full of jobs and disk space is very limited due to being full so I find PDSF better for something. In particular the access to HPSS through the grid is a unique feature and very important for for distributing STAR data produced at Tier 2 centres (no accounts for collaborators in general).

I use NERSC when my local machine is busy; also because NERSC has HPSS storage that is very convenient since I produce large datasets; also I use NERSC to check my codes on a different hardware that helps to catch difficult bugs.

Two words: disk space.

Good computing. Good storage. We always need more.

NERSC does a very job on supercomputing.
I am most satisfied with the purge policy, so that I only purge my data whenever is needed. This may sound simple, but it worth a million if you are analyzing a big project like climate research. People never knows which data is needed and it takes lots of time to do the backup jobs to the hpss, as hpss don't have a "rsync" feature.

  Other comments

I just wish I can have more allocation so more challenging problem can be attacked.

This was a good survey. It was not too long.

I compute at NERSC because my boss, Grant Branstator, told me to. Everything works fine at NERSC except franklin and HPN-SSH.

The hardware is good, but the OS was down for too many times. I compute at NERSC because this is the only place I can compute.

 

What should NERSC do differently? (How can NERSC improve?)   108 responses

  Franklin Issues:   24 comments

improve stability:

It would be great if NERSC could magically improve the stability of Franklin... Unfortunately, hardware failures increase with the size and complexity of the system.

Other than Franklin's frequent down time, I have no complaints, and even then I am very satisfied.

I have no suggestions. I'm sure now that franklin has been more stable lately that things will be a little more smooth.

Keep Franklin more stable.

Uptime should be more, downtime should be less. [Franklin user]

Improve Franklin uptime.

Reducing the outages. [Franklin user]

... Franklin is good when it is up.

needs overall improvement:

Get Franklin working !

franklin

Stop buying from Cray. Between the Catamount boondoggle and all the issues I've had with Franklin, I'm left wondering: is Cray run by 2 smart oxen or 1000 chickens?

Consider getting rid of the Cray XT4 (franklin) and getting something that works without endless fiddling on your part. ...

I think NERSC is doing fine right now. Their responses to problems are timely. Franklin has some initial issues, but these should settle down over time.

I do not believe the XT3/XT4 problems are due to any NERSC issues. The only thing NERSC might do is raise the priority of user problems higher and more quickly with Cray.

improve I/O:

Making the response time on Franklin faster would be nice.

Need to improve network and load management on the log in nodes for Franklin. At times it is very difficult to get any work done since the response time is so slow.

NERSC's greatest liability at the moment is the poor and unreliable I/O performance of Franklin, its flagship machine. I don't know how this can be fixed, but I would recommend that future purchasing decisions take more account of user experiences on similar machines. In the case of Franklin, the similar Oak Ridge NCCS/LCF machine, Jaguar, had been plagued by similar issues for some time.

fix lustre on franklin.

Hammer on the Sun people to fix the Lustre filesystem. It seems to me that issues with Lustre are the main reason that Franklin has issues. Otherwise it is a fantastic machine. NERSC has been a really great place to compute.

improve stability and usability:

... 2) Make franklin more reliable and user friendly

Franklin could be more stable and with less headaches. ...

improve stability and performance:

Continue to improve Franklin uptime and command line responsiveness.

I wish that Franklin was a little more stable, but I'm sure this will improve as the bugs get worked out. I wish the queue was faster on Franklin.

improve job management:

Needs a better resource management system. Limited run time is extremely frustrating. As I am running simulations that take weeks, having to resubmit jobs every day leaves a lot of lost compute time (waiting in the queue). Additionally, I have jobs killed because they ran overtime, but this was completely due to some slow down on the processor level, as I've had these jobs complete successfully previous (i.e. 200,000 steps can be completed in 24 hours, but for some reason, at 24 hours, I only get through 150,000 steps). [Franklin user]

  Change job scheduling / resource allocation policies:   18 comments

more support for mid range jobs:

... Don't push users to use large number of processors when they are not needed. Science first (NERSC should not be an experimental site for computer science). Computational capacity is more important than capability.

Do not force out "small" users --- there are many of us who are small only because DOE Office of Science has not given adequate funding to our programs. We hope this will change in the post-Dubya era...

Although this might be a minor point, I would like NERSC to set the class charge factors finely by the execution queues for optimizing the user's usage of the computing facilities. [In particular this user would like the addition of a regular_medium class on Franklin, with boosting and discounting over regular_small.]

I wish there were more of a market for SMP users, because it is a very convenient form of parallelism for some problems that are difficult to make totally distributed. However, getting through a queue that favors high total processor counts is hard. This doesn't mean that it is easy to find necessary resources (including large total number of hours) elsewhere, which seems to be the implicit assumption in setting queue preferences.

longer wall times:

1) Add a (super?) low priority, long duration (4 day) batch queue on franklin. ...

Providing for long serial queues (~12 hours) and enabling these for applications such as IDL would further improve the usefulness of Franklin in particular. We appreciate your efforts to do this and look forward to finding a solution with you soon.

Increase the time limit for premium class jobs. ... [Bassi user]

less support for INCITE and premium jobs:

Change the way INCITE awards work in the batch queue. [Bassi user]

Less emphasis on INCITE, special users. More emphasis on providing high throughput for production applications. [Jacquard / Bassi user]

Keep premium users from taking over bassi

more debug support:

Do not make it too costly to do development or testing. (where a user would only grab a node or two)

More support for debugging at scale. ...

training on queue use:

an improved queue system on bassi with proper instruction and approximate delay times to run the submitted jobs would be useful

The large number of different cluster and different queue options is confusing. Which one should I use?

more robust queuing system:

As computing clusters grow, it would be very interesting/helpful for NERSC to invest in robust queuing systems such as Google's MapReduce model. It seems that all of NERSC's clusters are based upon the premise that failures are abnormal and can be dealt with as a special case. As clusters and job sizes grow, single point failures can really mess up a massively parallel job (Franklin) or a large number of parallel jobs (bad nodes draining queues on PDSF). Companies like Google have succeeded with their computing clusters by starting with the premise that hardware failures will happen regularly and building queuing systems that can automatically heal, rather than relying upon the users to notice that jobs are failing, stop them, alert the help system, wait for a fix, and then resubmit jobs.

It should detect the jobs that fail because of hardware failure and reimburse the user of the time spent for running the job. I had so many jobs stopping because of insufficient wall time. I think that's a pity. However, I don't know what to suggest. [Jacquard user]

support for "massively serial" job streams:

I don't understand the rationale for limiting the total number of simultaneous jobs on jacquard to only 4. For my case, I required a lot of CPU time, but my jobs were implicitly parallel; I needed to run hundreds of independent single-processor jobs at once, not one hundred-processor job. The easiest thing would have been to simply submit 100 jobs and let the PBS queue figure out the most efficient way to distribute them. Instead, I had to first write wrapper scripts to launch my jobs through MPI, and then I had to manually divide them into submissions, arbitrarily selecting the number of CPUs based on estimates of the cluster loading. Basically, I had to wait for some large number of CPUs to be free at once before my jobs could start, even though I only needed one CPU at a time. It seems that allowing people to submit a large number of single-processor jobs simplifies life for the users *and* allows PBS to manage the overall cluster load more efficiently.
I understand the requirement that each job always be assigned to its own node (so that basically 2 processors is the minimum)... that simplifies everything and avoids RAM allocation problems. But it seems it would be better for everyone to limit the total number of requested processors, not the total number of jobs, so that people who don't need parallelization can still use the cluster efficiently.
In any case, this is not a major point, just more of an annoyance for which I don't understand the justification.

less support for small jobs:

I would suggest doing more to discourage single node and small jobs

  Provide more computing resources / new hardware / different acquisition strategy:   16 comments

provide more memory:

NERSC's seaborg was a great success because of its reliability and its large amount of per-node memory. That led to the situations that majority of scientific codes ran well on it. The future computer (NERSC6) shall have a configuration with large amount of per-node memory (similar to bassi or larger, but with larger amount CPUs than bassi has).

... more memory per processor core.

My applications requires a lot of per core memory. I hope that at least one of NERSC future machines will address this need in terms of computer architecture.

... Another very useful improvement would be to have more memory per core.

It is possible to provide a subset of Franklin machines with greater than 2 Gb of Ram. Maybe 16 with 8 Gb (per proc) 64 with 4 Gb (per proc) Certain problems are not efficiently addressed by 1.875 Gb of Ram

provide more cycles:

More machines to accommodate the large number of users.

As usual, bigger computers, more cycles, ...

Have bigger machine, and allocate large time.

You could always have more ... power ...

more careful machine transitions / longer runs during early user access:

Don't upgrade the machine and software too frequently. To be more conservative when purchasing new machines (stability should be an important factor for new machine, not just the speed). ...

... Longer transition period between old machine shutdown and stable period of new machines.

The Franklin supercomputer was purchased for a specific type of computing problem. As we know, not all computing strategies are appropriate for all problems. While Franklin may be a beneficial machine to some, there was an issue with Seaborg going away and a suitable replacement using a similar computing strategy not being in place. I'm both worried and excited about the upgrades to Franklin. Some researchers may feel that those upgrades are taking money away from a potential replacement for Seaborg - something with 8 CPUs (or more) per node. This may be the particular perception of those who view Franklin as a failure.
Two suggestions here then:
1. manage expectations of new hardware, and
2. when taking one kind of machine off-line, replace it with another of the same kind.

This is hard to answer. Most of our runs for the past 3-years have been at DoD sites. DoD has instituted a special month or two at the beginning of the life of a new acquisition of dedicated use of the whole machine to selected users [one writes proposals to be considered]. This works for DoD since they have over 6 different sites and typically 1-2 new supercomputer is being bought each FY.
For our work this has been of tremendous importance since our codes parallelize to all available cores without saturation [with help from Jonathan Carter :)].
While NERSC has early users -- this is primarily for short runs. For example, in Fall 07 we were chosen to run on the newly acquired 9000 core SGI Altix at ARFL (Wright-PAtt AFB). We were given all 9000 cores on shifts of 24 hours : we had the whole machine for 24 hours, and the other groups had the machine for 24 hours. This lasted for 6 weeks. Yes - the machine had its hick-ups, but we are still analyzing the TBs of data still!!
Obviously NERSC can only do this if they procure more machines than on a 3-5 year cycle.... -- so this is more of a comment and a wish than that practical for NERSC.

provide different architectures:

NERSC cannot do much differently. Computing procurements are driven by a very broad science base, which means certain areas (in my case chemistry) end up with an architecture that is not as well tuned for their applications.

Acquire a very large basic Linux cluster in order to offer more computing capacity to those users that don't particularly need to run very large single jobs.

I think that NERSC should evaluate whether its users computational needs might be better served by different computational configurations, particularly by a large collection of clusters with 256 - 1024 CPUs that would not require expensive interconnects but would be large enough to support the majority of the needs of users. On a per-CPU basis I know of several 'departmental clusters' that are ~half the cost of Franklin, for instance, and we could therefore have twice the total number of CPUs available for production work. Now that we have Franklin it can handle the relatively rare jobs that require many CPUs, and the next expansion of NERSC could more cost-effectively serve the majority of the work that does not need a huge number of CPUs.

  Allocations and Charging issues:   14 comments

improve allocation management:

See comments about managing allocations. I know of at least one other person that had a similar experience to ours so I know we are not the only ones.
The main problem we had working with NERSC was the ease with which it was possible to blow through our entire allocation before we realized the we had even done so. This occurred for a number of reasons.
...
2) The allocation management seems to be non-existent. There is no indication that you are running down to the end of your allocation or that you are asking for more resources than your allocation can provide. On our MPP2 machine at PNNL you can't get on the queue if you request a job size that overruns your existing allocation but there seems to be no such restriction at NERSC. MPP2 will also provide a warning that you are asking for more than 20% of an existing allocation, which provides another opportunity to check that you may be doing something stupid.

I'm only aware of the INCITE process for requesting significant time, and I would like INCITE to review applications twice yearly. The time between having a ready code, applying, and then waiting for the results can be quite long if you're a postdoc.

... It is also annoying to loose time on a quarterly basis if it is not utilized according to a predetermined standard (this does not happen at the other centers I use, e.g. SDSC, Artic)

The system of having to use up a certain percentage of your account by certain dates can be very irritating. It doesn't necessarily match my schedule of when I need to do computation, and rushing jobs through to meet the quotas can be a waste of computation time in the long run. Perhaps you can make the system more flexible, especially for small accounts where it can't make much of a difference in your overall scheduling scheme.

The allocation process should be stabilized - it seems like when it occurs changes every two years.

issues with Machine Charge Factors:

Treat a CPU hour as the base unit for allocations and drop the machine factors. These factors are confusing and led several different PIs at my institution (me included) to request only 1/6 of the time we actually needed because of the 6 times adjustment for the franklin hours.

Lose the "dog hours" accounting system, and base the numbers on real cpu-hours. This scaling to an old dead machine is silly and confusing.

The machine charge factors drive me crazy, but I understand the heavy demand for the NERSC resources.

... 1) The accounting system is not very intuitive and is geared towards mistakes unless you actually read up on it carefully (I don't think many people actually do this). It seems to me that using a node for 1 hour under regular circumstances on the flagship machine should be counted as 1 node-hour and everything else should be adjusted downward from that (except possibly using a high priority queue). The fact that you need to consider an extra 6.5 multiplier when using Franklin is asking for trouble. We won't forget to do it in the future, but we found out about this the hard way. ...

... Less confusing charging policy for SU's.

need a larger allocation:

Maybe, allocate a lot more time for me. ...

This biggest issue is with the allocation of hours. With the advent of massively parallel systems like Franklin, 1 million cpu hours for a modest sized group get used up very quickly. As the project calls are only yearly, this makes things difficult. Many other systems I have been involved with in Europe take open calls for project time throughout the year or at least quarterly.

One issue I've run up against is that my research group can use more hours than we are allocated (and we are allocated a lot of hours). I would run more if I didn't fear eating up others' hours. I don't know what the best way to deal with this issue is -- more frequent renewal possibilities?

We are mainly limited by the amount of cpu time we have available at NERSC. A deeper discount for jobs over, say, 10000 cores would be helpful to us and may encourage code/algorithm improvements by the community of researchers.

I would like more time on Franklin (or equivalent machines) by orders of magnitude. Nuclear physics did not get enough time to allocate.

  Improve queue turnaround times (especially on Bassi):   13 comments

The queue times on Bassi are long. ...

Bassi has become so busy as to be almost useless to me. ...

Sometimes the whole machine is taken over by a few large and long run time jobs. The queue waiting becomes very long in order to run small quick jobs that only require a few nodes and less than one hour run time. In this case, maybe good to designate a few nodes for small and short jobs. [Bassi user]

The queuing systems on most of the NERSC systems means an inordinate wait for many of my jobs. Especially on Bassi, the limits on the number of queued jobs limits the work that can be done on these systems.

Either NERSC computers are over subscribed, or the queues could be restructured to be more effective. Wait times can often go longer than 4 days, which is too long IMHO. [Bassi user]

Good as it is..sometimes batch jobs waiting in the queue is long. [Bassi user]

I am waiting for so long time on queue [Bassi user]

... Decrease waiting time. [Bassi user]

the waiting queue is very long. [Bassi user]

The queuing system in Bassi needs a serious work. The turn out time is just too long.

waiting times are long. [Bassi / Jacquard user]

... We still need to wait in the queue for quite some time. [Franklin / Jacquard user]

... My main concern is not what NERSC can improve but whether it can keep the queues short on Franklin -- less a matter of improvement than of staying in a very good place.

  More or Better Services:   13 comments

more PDSF support / recognition:

PDSF needs better NERSC recognition and support. It is a facility with large scientific output. It appears understaffed.

... I guess because of lack of personnel or time some of the thing on the webpages are out of date. Not sure how one would address that. It would mean someone had to have an encyclopaedic knowledge to know what was wrong. [PDSF user]

This is less affecting me now than when I was part of Nearby Supernova Factory, but a huge problem we had was weekend support for PDSF.

Accept pdsf as an important part of NERSC.

more or better training:

Enhance the tutorials that are accessible online (html & pdf versions), with better up to date examples. Some of these are mostly rehashed copies of the vendor's manuals.

Offer more trainings for remote users. Make them more accessible..

more consulting/visualization help:

More help with porting code to new platforms and more help with preparing data for visualization. ...

Dedicated support would be nice but I believe that it would be difficult to implement

improved web services:

a wiki on the use of the facilities

NIM could improve in its look and feel

shorter survey:

... This survey could be shorter - it is too complicated and long.

better performance and resiliency tools:

NERSC should tell more about their strategic plans. Hopefully in three years we will be operating differently than we are now (command line submission, manual data management etc.) Is NERSC going to actively help with this, or simply be a resource provider? Is NERSC going to help campaign to get better performance and resiliency tools (fault tolerance) actually put into production vs being left as academic demos?

improved reporting of outages:

The report of outage is not always in time. ....

  Data Storage and Disk Space Issues:   8 comments

If it were possible, the file system for Franklin /scratch should be accessible from the visualization/analytics server Davinci, and /projects should be accessible from Franklin compute nodes.
A significant improvement would be to make file permissions on the various platforms should have the same defaults. On Bassi, files are created with a "group" that is equal to the "user". This does not make sense to me. Groups should be for groups. We have to waste a lot of cpu time running chgrp on many files.

More disk space to users. The whole point of having a LARGE cluster is to do LARGE simulations. That means LARGE amounts of data. We should get more storage space (individually).

Scratch disk can be larger, or the limit on personal use of it can be larger. It will be good if there's a rule to make people delete the scratch data once a job is completed (or in a short time), everybody can share a bigger space.

Not sure. You could always have more storage and more power, I guess.

I have in the past had one job that *could* have used a very large amount of disk. The administrators increased my quota upon request to a reasonable limit, but unfortunately that was less than required. Calculating direct slowed things down, but I could get the calculation done. If there was a fraction of disk set aside for testing requirements (or, a flexible quota system that permitted large but infrequent and transient boosts to users' disk allowance), that might allow users to make better judgments about job requirements, and whether requesting a permanent quota increase will be enough. If a job actually requires a terabyte of scratch, disk quota increases are probably not going to be the solution.

If it is possible, a fast mass storage resource shared between the different systems would help my work. /project is very useful, but the I/O bandwidth leaves room for improvement.

I find the archive storage system a bit difficult to use, but this is not terribly important to us (we can cope with it easily). ...

The only noticeable problem is the frequency of disks getting problems but I think that this has improved. ... [PDSF user]

  No suggestions / Satisfied:   8 responses

I think NERSC does a great job.

everything is fine.

NERSC seems to be improving all the time.

I am basically satisfied with NERSC.

I think it already did very well.

Keep the same good work please.

I'm very pleased with the NERSC resources I've used, so I have no suggestions at this time.

I think you are doing really good now.

  Software issues:   7 comments

Adopting the same OS and compilers in Jacquard and Franklin would be nice as it would allow us to test our software in Jacquard and transfer to Franklin without any further modification. At this time, we are not always sure that a program running on Jacquard will also run on Franklin. ...

I would like to see some additional software libraries installed on *head* nodes. GTK in particular. [Jacquard user]

Make Gaussian available on Franklin. I could use NWChem, and probably should learn how to, but it provides results that are slightly different than Gaussian does, and I need to clean up the results using Gaussian after running an optimization on NWChem.

... Also add a new [Gaussian] license to Franklin. Do you allow users to make some recommendations on what you should add?

Maintenance of software library versions and the modules system needs to improve.

Our major electronic structure workhorse is MOLPRO. The inability to run molpro in parallel across multiple nodes significantly detracts from the attractiveness of running jobs at NERSC.

Keeping up to date with the STAR libraries at RCF.

  Network Issues:   3 comments

Better network access outside of ESNet would greatly improve our efficiency. [PDSF STAR project manager]

... We could use faster data transfer. This can be accomplished via HPN-SSH, I think. But you would need to support it on your end. Most of the speed-up that is achieved by using these patches to standard SSH happens on the sending end (that's you, when I am moving data from NERSC to NCAR). I have had an open ticket on this issue for a long time.

As I have stated earlier, the only "difficulty" is in the time it takes for me to download atmospheric model data output files. The network connection between NERSC and my "home" computer is consistently slow. I don't know where the bottleneck is occurring. [University of North Carolina at Asheville user]

 

How does NERSC compare to other centers you have used?   104 responses

  NERSC is the best / overall NERSC is better / positive response:   61 responses

NERSC is the best supercomputer user facility I have worked with. It provides the best user services and has an enormous software repository.

The user support, ticketing, and follow-up is particularly competent and professional relative to other DOE centers.

There is no comparison really as other facilities I have used are not comparable at all to NERSC. NERSC stands by itself alone in its class.

One key thing that I've been enjoying at NERSC is that the queues are not overloaded like some places I compute. Either that, or the queues are designed to facilitate the style of problems that I perform. So, I see progress regularly and am not frustrated by the wait times. The other thing that I really like is that the /scratch space is large enough that my files are not regularly being scrubbed. I really dislike some of the DOD centers because it takes a long time to bring my restart files across to the scratch space, then I wait in the queue forever to get the CPUs. Once I have the CPUs my job looks for the files and they've been scrubbed. Then I'm starting over again. This has never happened at NERSC.
NCAR
DOD (AFRL, ERDC, NAVO, ARL)

NERSC is better than average for the computer centers I have worked with, and there is no one facility I feel is better than NERSC. In particular, the security strategy at NERSC is more effective at keeping the computing resources both secure and available to users with limited interruptions. This is unique and very effective.
PDSF and HPSS are a powerful combination, and the commitment and responsiveness of their administrative teams should be commended.

NERSC systems seem to have much faster turnaround time than supercomputing facilities at Oak Ridge National Laboratory (Oak Ridge, Tennessee)

NERSC is No. 1. I used ORNL once. It was unpleasant.

The application/activation process for accounts is much easier than at ORNL. The lack of requiring electronically generated passwords (e.g. SecureID) is a great plus for NERSC. Other facilities I use that require these are ORNL, NCAR, and some EMSL resources.

NERSC is much better than NCCS in user service, and machine availability.

The best of all

Compared to the cluster at pnnl, NERSC is much better. Shorter queues and easier access.

Better support and better capabilities. (Environmental Molecular Science Laboratory -PNNL).

I've only really used the local Stanford clusters, and NERSC is heads and shoulders above Stanford in every respect: account services, compiler environment, queues, tutorials on-line, etc.

I compare NERSC to Teragrid systems and also nano millennium (on campus I believe). I think NERSC is the best out of these and I can't think of anything that the others do that I would like to see NERSC do. Well, I guess I don't like the fact that I can't run 'low' priority jobs over 12 hours long on Franklin, but that's about it.

I have also used NCAR, and I have found NERSC to be better in every way. Good job!

Better than the Goddard Space Flight Center at NASA. The people GSFC are also very responsive to our requests, but seem to lack the technical ability to actually get the job done for us. We no longer compute there.

Best so far....

In addition to NERSC, I use the following facilities:
1) Ohio Supercomputing Center (OSC)
This is a smaller, local center. It provides us with a small fraction of computing resources compared to NERSC - both in terms of allocation size and machine size available. However, we are allowed to renew our allocation frequently if needed.
2)National Center for Supercomputing Applications (NCSA)
NCSA also does not offer us as many resources (allocation/machine size) as NERSC.
3)National Center for Computational Sciences (NCCS)
NCCS also offers a large Cray-XT4 system, however our group is relatively new to this facility and our allocation is fairly small.
NERSC is the facility at which we do the majority of our computation. We have had a long history with NERSC and a closer relationship, including being involved in pre-beta/beta stage testing on Franklin. NERSC provides us with a sizable allocation with which we can afford to scale our simulations up to thousands of processors and complete heroic-sized research projects. NERSC is the very good at communicating with its users - notifying of system changes and offering special allocation programs. The support staff is also very friendly and responsive.

nersc is number 1.

NERSC offers pretty much hassle-free interaction and a long-term stable environment for the user. I am comparing it in this regard to NCSA. NERSC is much better.

Best so far (only other experience is SDSC's DataStar, which is significantly older than Bassi).

Better support.

Compared to SDSC at UCSD, NERSC is much better in terms of the resources it offers and the availability of staff for consulting issues. Compared to LC at LLNL, NERSC is a much more open facility (clearly) and has more resources available to the common user.

NERSC is the prime example of how a user facility should be run.

It's the best service I've ever used.

Better than most of the other centers. I would rank it on the top of others.

much better than our university supercomputing center.

Well.

Easy to contact. Helpful and dedicated staff.

The only other computer center I use routinely is the RHIC computing facility, and I much prefer using PDSF at NERSC. Because I can count on finding disk space, and throughput on batch jobs is generally higher.

Great

I have only used NERSC and NCAR. I find NERSC has shorter queues, equivalent support, and a less cumbersome security system (NCAR uses cryptocards which I hate).

very competitive

Good consultant and excellent software, and user attention.

The NERSC is the best one.

It's the best that I ever used.

I had used ORNL computers. I like NERSC's flexibility with job size (number of processors) and NERSC technical support.

NESRC is very very good in what she does.

As I said earlier, NERSC really shines when compared to NCCS. NERSC has spoiled me with the quality of the services.

NERSC compares very favorable with other centers such as Oakridge NCCS, RPI CCNI. NCSA and OSC with comparable resources to NCCS and CCI and more available resources than NCSA and OSC. Software and documentation is always up-to-date and your staff works hard on bugfixes etc.

NERSC has more reliable hardware and software support than NCCS.

LLNL clusters for low-clearance interns had MISERABLE support and software configuration. NERSC is doing it right by comparison.

Although Franklin initial experience was a bit rough. My experiences at TACC using Ranger were far, far worse.

The main alternative computing center with which I have experience is NCCS, which I have not used in six months. In my opinion, NERSC is superior in all respects, particularly in that it seems to be driven by the needs of its scientific users rather than by a desire to do cutting-edge computer science regardless of the quality of the science that enables.

Better. Much better actually (than the DoD centers)

NERSC is doing extremely well, as usual, especially since you are dealing with about 10 times more projects and users than the other centers. Hiring more people would be a good thing if DOE would allow it...

NERSC compares excellently with all the other centers we have dealt with. These DoD sites are NAVO, AFRL, ERDC.

I have recently been running stuff at NCAR and at NCCS, and I like NERSC better that these other two centers. Mostly I like the people I talk to at NERSC, and I think the time spent in the queue is shorter.

nersc is better than others, especially better than the jpl nasa computing center I also used to use.

Other centers I have used are NCCS at ORNL and MCS at Argonne.
1. NERSC website is far superior to either of the others.
2. The BG/L at MCS is not useful for me because of the incredible small amount of memory per core. Franklin is a much better machine with respect to my needs.
3. Jaguar is equivalent to Franklin, but the queues are heavily biased to very large jobs. This is good, because getting large jobs to run on Franklin takes a while (though I haven't done it for at least 6 months, so perhaps the queue throughput has changed by now). But getting small jobs to run on Jaguar is more difficult, so Franklin is better there. With access to both, I find them to be good complements to each other. But for those who have access to only one or the other, this could be problematic.

In my former life, I have used DoD HPC centers and NERSC provides service that is on par with and, in some aspects, exceeds what I experienced with the DoD HPC centers.

I also use some computers that are part of the Teragrid Consortium. I also like the Teragrid facilities, but I've generally found the queue waits to be shorter at NERSC and that it's easier to get timely answers to questions at NERSC, in part because it's sometimes not clear where to address question on the Teragrid.

Compares very well(if not much, much better), RHIC Computing Facility.

NERSC versus CINECA (in Bologna) is faster in running jobs in queue. Better informations about resources and software.

Comparing to the SX-8 at RCNP, Osaka University, Japan, and the TSUBAME (linux cluster for the parallel computation) at Tokyo Inst. Tech. Japan, I am much satisfied with the NERSC facilities for running the code, because of the prompt upgrade of the software and the bug-fixing.

very well run and managed. Usually good turn around. good consultants

Much better than an LSU cluster I attempted to use long ago, and much better than smaller institution-specific grids with a much less sophisticated (i.e. manual) queuing system.

Excellent

Very easy to use compared to RCF.

NERSC is the best center that I use overall. I also do computing at NCCS, LLNL, LANL, and SDSC.

For STAR it seems to be have a higher effective availability than RCF when one considers how tight resources are at the latter.

  NERSC is the same as / mixed response / no comparison:   25 responses

The primary other facility we use is the NASA Advanced Supercomputing (NAS) center. Both NERSC and NAS are providing excellent services; nothing occurs to me when I am asked to compare them. Both are excellent. Thanks for the great service!

I've used NERSC, LLNL, and ANL. NERSC is tied with LLNL as the most user friendly and responsive center I've dealt with.

NERSC is the best computing center I have used in terms of user support. In past years, I would have also rated it the best in terms of hardware reliability and availability, but as noted earlier, the Franklin system is not up to the high NERSC standard.
[I am comparing with past experience at Oak Ridge, LANL, and Air Force centers.]

NERSC is probably the best, though services and overall performance at LANL and at Argonne/APS local cluster are also very good. I am very satisfied with all supercomputer centers I have worked with so far.

Our other HPC experience is mainly at ATLAS at LLNL (https://computing.llnl.gov/?set=resources&page=OCF_resources#atlas). NERSC's queue structure is generally much easier to use for us, resulting in higher productivity. ATLAS does allow long serial queues for analysis (eg IDL) which is very important as noted above.

I also work at NCAR and ORNL. NERSC has been very responsive to our needs, and has a commendable uptime record. However queue times at both NCAR and ORNL tend to be much shorter.

Comparison with ORNL Jaguar
Pro: More user-friendly queue policies, in particular for small-scale jobs
Con: Slower execution time

Compared to the computational centre in Juelich, NERSC is far superior in terms of machines available and information available on - line. At Juelich, however, there is no storage limit on archived data (only on the number of files) and the disk space available by default is much more (~3TBytes).

In general, NERSC is more stable, and therefore a better place for development than other centers I have used. But this success means that there is more competition with other users for resources (long queues).

NERSC is very well run although I can't complaint too much about the services I receive at ORNL either. However, your user "how to" web site is much better than anything I have seen at ORNL. Also, your help desk is more responsive than the one we have here. I just get better turn round on my jobs at ORNL and therefore I get most of my work done ORNL. NERSC is a secondary source of computer resources for me.

Franklin seems to be up more reliable than Jaguar at ORNL. If/when Franklin gets upgraded to quad core, I hope that that can be done with significantly less down-time than the quad core upgrade of Jaguar.
I found several "hands-on" workshops organized by/at the ALCF (BlueGene/P) at ANL very useful; in particular the fact that they have "reserved queues" during the workshops so that you get immediate turn-around, even for big jobs. Of course, I realize that the ALCF serves a much smaller community, and reserving Frankling impacts a lot more users than reserving BlueGene/P at ALCF. In addition, Argonne is half a day driving for me, so it is easier to go there for a day or 2 than going to NERSC; which is probably why I have never (or I should say, not yet) attended a workshop at NERSC.

LLNL
Texas A&M Supercomputing center.
NERSC is on par with the LLNL center.
The professionalism and amount of support available is better than at TAMU, as is the stability of the systems.

NERSC seems on par with the other centers I use, which include ORNL and NCAR.

It is as good or better. Recently, I've used the ATLAS cluster at LLNL.

Compared to Jaguar at Oak Ridge - They have passcode access, it is a hassle to type the pass code but at NERSC I end up getting my password unblocked frequently (eg after trying to log into Franklin while it is down). Their queues have been bad during their quad core upgrade.
Compared to Red Storm at Sandia - You're accessibility is infinitely better for a variety of reasons.

It's much better than http://www.arl.hpc.mil , though franklin is down a lot more. When up, franklin is a better computing environment.

NERSC compares very favorably to other centers.
National Center for Atmospheric Research
NOAA Earth System Research Laboratory
NOAA National Centers for Environmental Prediction
The only thing that is done at these centers is a tighter coupling of the file systems, and more standard use of "user", "group", "world" permissions on files. NERSC's consulting and staff are on par or exceed that of these other facilities.

Very well.
sdsc/san diego
arsc/alaska

Comparing to RCF at BNL:
NERSC/PDSF has an accessible support staff
NERSC/PDSF disk systems are not very stable compared to experience at RCF (and CERN)

Other centers I've used:
National Center for Supercomputing Applications (NCSA)
The Ohio Supercomputing Center (OSC)
NERSC has bigger facilities than these others I've used and has allocated my research group more time than the other two. OSC, probably because it's smaller, has a frequent (monthly) allocation application review.

Good but others have shorter waiting queues

I have computed at SDSC, NCSA, ORNL, ANL (ALCF), TACC, FNAL and PSC and some older centers. NERSC compares quite well, at least as far a my experience on Franklin is concerned. I did not make much use of the IBM machines. I have found that PSC was willing to assign a single person as the first line of support to our project. This worked out well there as he knew who to contact when we had any unusual problems and he kept up well with our code and issues.

I'm also running GYRO on Jaguar at NCCS, where I have honestly done most of my work. I think this is mainly due to the relative computing allocations I have available to me at each location, but I've also found that the MPP charge on Franklin is such that if I try to do the number of runs I'd like, I burn through my allocation time at NERSC quickly. The work I'm doing requires many moderately sized runs rather than a few "hero" runs, so I'd actually prefer to run more on Franklin and request smaller processor batches than I do on Jaguar to improve the efficiency of my runs.

I also used JAGUAR. Franklin performs the same.

I'm also working on SDSC. Both supercomputing centers are satisfactory.

  NERSC is less good / negative response:   11 responses

I have better experiences with both PNNL and Argonne but some of the reasons have nothing to do with NERSC being less competent. That Argonne is operating BlueGene machines and is has a group developing system software for this platform on-site is a huge advantage. PNNL is operating a single Linux-based machine which, in addition to being rather mature, was built specifically to run my code.

LLNL was excellent and perhaps provided the model for NERSC.
The German facility at Juelich was a place where I worked for many years. they have kept up with the advances in computers a bit better than NERSC. They have now installed the latest Blue Gene IBM machine.

Allocation management at PNNL seems to be a lot more straightforward than at NERSC.

I was surprised to see that Franklin is about 4 times slower than EMSL's MPP2 for computational chemistry runs.

The NIH "helix" cluster team offers the "swarm" command. It is a wrapper to "embarrassingly parallel" PBS jobs, so that the user does not have to known anything about processor configurations. All these options (cput, ppn, ncpus etc.) are "behind the scenes" automatically generated such that the cluster is being used in the most efficient way (no empty cores on a node for example.) This makes parallel processing unbelievably easy, one simply specifies the commands that have to be executed. I was so enthusiastic about the swarm command that I contacted the NIH team about where they obtained this program. I was being answered that they developed it in-house, but it is freely available. Maybe it could be interesting for NERSC to contact the NIH "helix" cluster team about this extremely helpful program.

Coming from Virginia Tech with System X, I was able to submit a job and have it run to completion, whether it took 2 hours or 2 weeks.

I have also used DOD HPCMP resources. They tend to update their hardware more often than NERSC.

teragrid has a better web portal

NERSC seems to have stricter rules, more bureaucracy, and less flexibility.

BNL-RCF/ACF commits staff to aspects of distributed data storage, such as dcache and xrootd. If PDSF is to remain a functional facility, in particular xrootd will need active NERSC support.

I have used the Minnesota Supercomputing Institute. In my opinion, their support and help staff was better and friendlier.

  No comparison made:   7 responses

SDSC
Artic
LLNL

I have not used any other major center like NERCS

NCCS/ORNL
ATLAS/LLNL
THUNDER/LLNL
BLUE GENE/P/ARGONNE
The other centers we use are more geared to our largest-scale production runs. We are part of an INCITE award at ARGONNE and ORNL. We are also part of an LLNL Grand Challenge award.

NAVO

Other than NERSC, I have experience with only the LCRC at ANL, though the extent of my use thus far doesn't really give me much to really compare the two.

I have almost only used NERSC for my computation needs.

Not relevant.