It would be nice to have some info on debugging on the web site (not sure this
is the case at the moment).
Need a good debugger on the Cray - totalview sucks (very flaky, gives wrong
answers, crashes frequently, etc) [...]
The totalview debugger is great when it works, but it is unreliable (at least
on the T3E and PVP).
better debuggers for Seaborg needed!!!!!!
NERSC response:
We try to maintain the best debuggers available.
Totalview is, despite its failings, the best debugger we have.
Software requests may be made via our online form.
NERSC consultants try to keep abreast of the latest software developments,
including new debuggers.
Feel free to contact us with suggestions for new debuggers or bug reports
for our current ones.
The lack of an interactive queue on seaborg makes debugging, via, for instance,
totalview, impossible.
NERSC response:
Indeed, this was a problem until recently.
We have expanded the resources directed at debug and interactive needs.
There are now resources dedicated to interactive/debug work during the day.
See
NERSC Queue Policies on the SP
for more information.
bug resolution usually requires help from consultants
I want to learn to use more performance analysis tools, but just haven't had
the chance. The debugging has
been fairly straightforward, at least for the simple things I ask it to do.
All of my dissatisfaction is due to the lack of proactive acquisition, user
training, and the availability of a suitable
partition for the in-depth performance analysis of large production codes on
seaborg. I would like to learn and
be able to get hardware performance counter information and MPI performance
information for PCM. I have
tried to do this at the NCAR site using the available IBM documentation. It has
been an extremely frustrating
and daunting task. I think that NERSC has the resources to provide this
capability and training......
NERSC response:
Hardware performance monitoring on the SP is somewhat underdeveloped.
Two API's exists for gathering HPM,
PAPI and
PMAPI.
Neither of these APIs provide HPM
data for parallel applications without code modification and recompiling.
Also, neither provides a mechanism for aggregating HPM data
from all of the tasks in the parallel application.
We will continue to search for ways to make instrumenting code for HPM easier.
A tar with the -z option (like GNU\'s) would be good. Some of the GNU tools
seem to run (extremely* slowly for
no apparent reason. [...] Using Xwindows from here is so slow as to be nearly
useless. I really need a tool that
will run anywhere to translate Cray binary files into more conventional
formats.
NERSC response:
The GNU version of tar is available in the GNU module.
NERSC is working with IBM's xlf compiler group to enable reading of Cray unformatted binary
data on the SP.
When this functionality is available it will be announced to users.
NERSC supports a lot of neat DOE software, and also serves the function of
identifying what part of that
software is ready to be supported big-time.
The support work you do is outstanding. Some improvements could be made in how
quickly new releases of
supported software are installed, e.g. petsc.
The default SP account configuration leaves all customisation *and usability
creation* to the user. Please, give
users default shells/configurations with command line histories etc. This is
the 21st century. NERSC also
appears to be continuing its unofficial shell war - no/little bash support -
and while there are many technical
reasons for not liking it, the shell is the primary interface through which
users use the machines. Please, at
least provide some example "comfortable" configurations on the web page.
Keeping load on interactive nodes
down is presumably a consideration, but this should not be at the expense of
usability. Sorry for the rant - but
by default the SP has an interface from the 1980s.
NERSC response:
Good suggestions!
Sorry if it seems like we are playing favorites with shells, but the issue
is really software support.
We have recently allowed users to choose tcsh (and soon bash!) as their
default login shell. While there is still no official software support
available, we will make our best effort to accommodate shells requested by users.
We try to make the default configurations for all shells reasonably comfortable.
Specific suggestions are welcome.
Realize also that we prefer to use modules to provide functionalities which
not everyone might be interested in. E.g., the GNU module provides
color ls under bash, something that not all bash users might want.
More general tools could be helpful. Compression utility on IBMSP would allow
large file transfer faster.
It would be very useful for me to have some simple visualization software on
the IBM SP (seaborg), for example,
gnuplot, because it is very helpful for me to be able to take a quick look at
the data, without having to transfer
large files out.
NERSC response:
A gnuplot module has been added.
I would love to see vim installed. It's the best programming editor around.
NERSC response:
The vim and gvim editors have been on
seaborg for some time.
Everyone should have these in their path.
NERSC has added a vim
module as well to increase its visibility. Likewise you can alias
vi to vim.
I need some support when there are Python problems.
NERSC response:
Feel free to contact NERSC consultants with Python questions.
update emacs to xemacs.
NERSC response:
Looking into this.
[...] Cray cc also has many annoying quirks (e.g. no support for 16-bit data
types) - it would be nice to have
other options gcc has just been ported to the T3E - put it on mcurie!!!
The Cray C++ compiler is not very good, and KCC is too slow. The IBM xlC
compiler seems to be better.
I wish some vendor had a compiling Common Lisp for either of the useful
machines. sigh... Other high-level
languages would be useful (Haskell, OCaml, etc), as well.
The only thing that really matters to me is ANSI C++ compiler with KAI
preprocessor. NERSC does reasonable
well in this regard. My group my have specific comments.
[...] linking my F90 code seems to take a very, very long time (up to 8 min.);
linking the same
code here takes ~8 seconds The Cray f90's compilers idiosyncracies are such
that having the GNU gcc/g77 compiler would be handy. [...]
NERSC response:
We have not been able to compile gcc under unicos, but will investigate the tip above about a recent port. We agree that having multiple sets of compilers is
useful. Requests for new languages can be made through the
online software request form. Python was made available by such a request.
Well... IBM is known for its great hardware, not software. 64-bit MPI has been
part of SGI software for several
years by now and it works very well. Same thing for the profiling tools. Why
can't IBM figure it out? NERSC
should be more daring at getting beta-level software that the users could try
out... as long as that software
doesn't crash the computer though.
I will like to do compiling with 64-bit addressing for my high resolution
simulation of climate. I hope it will be
available on IBM SP soon
NERSC response:
NERSC has added the mpi64 module which provides this beta software.
Sorry for the delay, our highly unique dual adapter switch topology conflicted
with this software.
This conflict was recently resolved and the module is now available.
We look forward to the non-beta release of this software.
more documentation online about fortran libraries ..
NERSC response:
NERSC maintains local copies of most IBM documentation Fortran, C,
SP related subjects from
usgibm.nersc.gov
. Our main site hpcf.nersc.gov provides a great deal of
information and where appropriate links to usgibm. Please make the most of
these resources and feel free to
request additions or changes.
I'm using all the time the BSP library and the ARPACK package. It would
be nice if they were installed at NERSC
NERSC response:
We are looking into this. It would help us to have more details a
written into a software request.
It would be nice to install LAPACK95 on the Cray PVP. Right now there is only
the ancient LAPACK77.
Because we like to have PORTABLE codes, we had to install and compile LAPACK95
ourselves in our user
directory. The consultants did not seem to understand why we did not want to
make calls to LAPACK77 directly
(which would destroy code PORTABILITY).
NCAR/NCL never adequately replaced DISSPLA, after several years of bad
experience