Share
FollowVisualization and Data Analysis
- Legend
- Overall Satisfaction with NERSC
- How important to you is?
Legend:
|
|
||||||||||
|
Satisfaction with Analytics and Visualization Services
7=Very satisfied, 6=Mostly satisfied, 5=Somewhat satisfied, 4=Neutral, 3=Somewhat dissatisfied, 2=Mostly dissatisfied, 1=Very dissatisfied
| Item | Num who rated this item as: | Total Responses | Average Score | Std. Dev. | Change from 2006 | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | |||||
| DaVinci SW: Software environment | 2 | 5 | 33 | 49 | 89 | 6.45 | 0.71 | 0.03 | |||
| DaVinci SW: Fortran compilers | 1 | 3 | 6 | 20 | 42 | 72 | 6.38 | 0.91 | 0.15 | ||
| DaVinci SW: Analytics software | 3 | 4 | 17 | 24 | 48 | 6.29 | 0.87 | ||||
| DaVinci SW: C/C++ compilers | 1 | 1 | 7 | 4 | 7 | 43 | 63 | 6.27 | 1.30 | -0.35 | |
| DaVinci SW: Visualization software | 6 | 9 | 17 | 27 | 59 | 6.10 | 1.01 | 0.10 | |||
| ANALYTICS: Visualization services | 1 | 14 | 8 | 24 | 24 | 71 | 5.79 | 1.16 | 0.10 | ||
| ANALYTICS: Data analytics software on computational systems | 1 | 2 | 13 | 5 | 27 | 23 | 71 | 5.73 | 1.30 | ||
| ANALYTICS: Visualization software on computational systems | 1 | 15 | 15 | 24 | 26 | 81 | 5.73 | 1.14 | |||
How important to you is:
3=Very,2=Somewhat,1=Not important
| Item | Num who rated this item as: | Total Responses | Average Score | Std. Dev. | ||
|---|---|---|---|---|---|---|
| 1 | 2 | 3 | ||||
| ANALYTICS: Visualization software on computational systems | 35 | 35 | 41 | 111 | 2.05 | 0.83 |
| ANALYTICS: Visualization services | 35 | 31 | 38 | 104 | 2.03 | 0.84 |
| ANALYTICS: Data analytics software on computational systems |
37 |
30 |
38 |
105 |
2.01 |
0.85 |
Where do you perform data analysis and visualization of data produced at NERSC?
| Location | Responses | Percent |
|---|---|---|
| All at NERSC | 28 | 7.2% |
| Most at NERSC | 63 | 17.4% |
| Half at NERSC, half elsewhere | 63 | 17.4% |
| Most elsewhere | 98 | 27.1% |
| All elsewhere | 102 | 28.2% |
| I don't need data analysis or visualization | 32 | 8.8% |
Are your data analysis and visualization needs being met? In what ways do you make use of NERSC data analysis and visualization resources? In what ways should NERSC add to or improve these resources?
- Yes, data analysis needs are being met: 19 comments
- Requests for additional services: 11 comments
- Don't use (yet) / need training: 7 comments
- Do vis locally: 6 comments
- Network speed is an inhibitor: 4 comments
Yes, data analysis needs are being met / software used / positive comments: 19 comments
Emphasis on resources required:
I love using davinci for interactive serial/threaded jobs that need tons of RAM. I have compiled our timestream visualization software there (http://kst.kde.org/), and I can load up many GB's of data and view it interactively.
Some thoughts:
1. As much as I love the performance of Itaniums, running an emulated 32bit version of IDL has its limitations. Not only is the RAM limited to 4GB, but we cannot compile dynamically loaded modules against this IDL without cross-compiling them on i386. Hopefully the "next davinci" will use more "common" processors.
2. Whatever you decide on for the next machine, large amounts of shared memory is REALLY useful.
I intend to use visualization (VORPALVIEW) and analysis tools (primarily MATLAB) at NERSC in the near future. Most important thing in this respect for me is to run MATLAB on very large memory nodes.
Davinci for some number crunching. Interactively on Franklin and Bassi. ...
Data analysis is an integral part of our simulations, and automating it and incorporating it into the workflow is an essential part of making the simulations successful. To do this, we use serial queues to launch analytics and visualization (eg IDL) after the run, which often take ~12 hours requiring a long queue, and then to copy analyzed data elsewhere. We're working with NERSC staff to enable this in the Franklin environment and appreciate their efforts to do so. An alternate approach may be launching jobs & sharing data across machines (e.g. a job on Franklin finishes and commands a job start on Jacquard or Davinci for analysis, with the appropriate use of /project to store data). The visualization group has been very helpful, and we are working with them on these issues as well as databases, new ways of tracking and clustering particles, and 3D parallel visualization (VisIT). Continued expansion and availability of visualization and analytics, in particular of long serial queues (~ 12 hours) is a high priority for us to be productive.
I use Davinci to pull large CFD data files from hpss and do expensive processing to extract complex derived data. Most of my actual visualization is done with home-spun software aware of our special data structures for large, parallel data, or is over extremely reduced subsets transferred to my local desktop and viewed with commercial packages such as tecplot. I rely on the parallel processing, large disks, high-speed transfer to and from Davinci, and the compiler system functioning. I do not use any other paid-for packages.
I've only done a moderate number of runs on NERSC computers so far, and have found it easier to analyze data on a separate system to have a consistent storage location for runs done at NERSC and other sites.
Data analysis needs are mostly being met. Data analysis done at NERSC is done using custom, low level code. Some data analysis code is parallel and run through the regular batch queues while some data analysis code is run through the serial queues.
I processes all my data output from Davinci. My master thesis is exclusively analyzed from Davinci.
Emphasis on software used:
Our main need is for a sufficient number of free idl licenses, and this need is being met (thank you!).
I have exclusively used IDL and my needs are currently being met by this software.
I use IDL extensively ....
I use IDL.
I mainly use IDL at NERSC.
Only use ROOT4STAR. Very satisfied.
I use ROOT at PDSF
To visualize data I primarily use VisIt developed at LLNL which is installed on Davinci.
Very happy with the Vis Groups responsiveness. We use visit a lot. ...
Some basic plotting utilities are good on the science systems; gnuplot, xmgrace, etc. I don't have extensive experience with davinci... yet... but I will be using visit very soon.
Deploying the FreeNX software is big step in the right direction for users sitting on the East Coast. The graphical interactive software is extremely painful to use over a normal X-Windows connection through ssh. The FreeNX connection is very fast but not quite as robust as X-Windows. ...
Requests for additional services / problems: 11 comments
Requests for additional software / software capabilities and problems:
... It would be nice to compile visit modules in-line with our big runs on Franklin.
Something like gqview to see previously made plots!
The double precision version of the NCAR Graphics libraries (version 4.3.1 -- the last double precision version that will ever be available) does not work correctly. I, too, was unable to build these libraries from source code on davinci (although I have successfully done this on a wide variety of other platforms). Hence, I have had to use the single precision version of these libraries. This requires many painful changes in my Fortran programs, but it *is* a work-around.
I made a comment about Mathemtica and Matlab, they were very useful at newton.nersc.gov, now I cannot use them because of the font problem, I never could figure out why, and whether it can be fixed.
I need GTK installed on *head* nodes to be able to visualize @ NERSC [Jacquard user]
Latest version of GNUPlot with gd library and pdf, png, jpeg and gif terminals. [Bassi/DaVinci user]
Requests for consulting help/ vis is hard
More support for high use codes to interface with advanced visualization software like NIMROD support for VisIt, etc.
In my field we have a hard time visualizing fields in 4 dimensional space-time.
... I may need NERSC assistance for future applications.
Requests for (more) resources:
... Davinci needs to have access to Franklin /scratch.
davinci can get overloaded, especially if someone is using it for parallel visualization. it would be nice to have more processors.
Don't use (yet) / need more information / training: 7 comments
... not sure what the current status of tutorials on moviemaking is but if info is out of date, please update
... In spite of the network speed, I personally lack the training and the time to do advanced visualization so I would need some good tutorials with a data centric focus. I'm willing to learn but don't have much time...
I had no clue that there is such a thing as analysis and visualization services until reading this survey.
I don't usually need extensive analytics or visualization, but that need is also a function of availability. I do primarily electronic structure calculations at NERSC; if there were new and innovative means of analyzing and visualizing the results (derivable from the density matrix, for example) that could be done relatively easily, that might be valuable. However, that depends more on external scientific development than NERSC administration. So, good job with what you've got!
I don't do much advanced data visualization -- most of my analysis is not computationally expensive. But I think visualization is neat stuff that should be supported!
I do not at present use NERSC facilities for visualization; the data analysis I perform there is done with my own software. I was pleased to hear from a colleague this year, however, that DaVinci is now capable of running the AVS network that is our primary means of visualizing our data, and expect to make use of it in the future.
I am not aware of these services. I will check on them
Do vis locally: 6 comments
I've been doing my visualization work locally. Maybe I would benefit by using DaVinci, I just never thought of it.
I basically use the NERSC computers for the heavy runs, then copy the data to my desktop for visualization and analysis, so I don't really use the NERSC computers for visualization.
I do all this on my local machine
I perform all my analysis locally on Franklin, but due to firewall limitations at my end I bring all necessary files back to my home computing facility for visualization, etc.
Over 90-95% of our visualization has been elsewhere -- mainly from convenience point of view.
My data analysis involves customized software that I run on my own computers. ...
Network speed is an inhibitor: 4 comments
Currently, our local internet connection is slow, but I will use more if network is improved.
The GUI analysis program I typically use on Davinci (VCDAT) suffers badly when there is communication latency between me and the application.
Data analysis mostly done using root/root4star macros and classes. Resulting histograms are usually copied to a local machine for visualization, since root is so $&*! slow when running remotely.
Unfortunately I cannot use any software which needs interactive use of the X window since the network is too slow! Is this our side's problem? Cause I'm in the east coast?
