Tools and Strategies for Data Understanding
NERSC is commited to support users, including instructing them how to carry on analysis using their data stored in NERSC systems. This page describes tools and strategies for data understanding using software installed at NERSC. Do you have other software options? Please contact us and let us know which solutions will help you better extract information from your datasets.
Imagery coming from high resolution, high-throughput sensors is a fundamental challenge for data-intensive science. Our focus has been on supporting software and making algorithms available that efficiently deal with imaged experiments. Advances in image-based analysis will save time between experiments, make efficient use of samples, and increase access to imaging instruments. Read More »
Data analysis (or analytics) and visualization are two steps in data understanding, often interleaved and symbiotic, so that many of the available tools characterized as one category, end-up having some functionalities of the other. Bellow find links to software tools grouped under Analytics or Visualization, but have in mind that their functions may be interchangeable. VisualizationAnalyticsVisit Matlab Python tools: Numpy, Scipy, iPython, matplotlib Paraview Mathematica Perl IDL Python TCL/TK… Read More »
NERSC recognizes that an increasing number of scientific data problems rely on orchestrating and managing large numbers of tasks through complex workflows. An increasingly important case is high throughput computing, where analyses need to perform a very large number of independent tasks, each of which uses only a few cores. Managing these workflows via traditional batch jobs is inefficient for both humans and the batch queuing system. We define a workflow system to mean a system that can… Read More »