Analytics & Visualization
Overview and Strategies
Data analysis and data mining tools are used to compare datasets or find features within a dataset. Visualization of data is one of the primary tools for data exploration, and may precede or inspire more formal data analyses. The technologies described above may be used individually or together to explore data. Data exploration discusses how analytics technologies can be integrated to provide a framework for discovery. In general, scientific data management and workflow management are enabling technologies. Scientific data management provides tools for efficient access to large amounts of data, as well as supporting data organization and security. Workflow management describes a systematic approach to data processing pipelines or the pre-processing and post-processing steps involved in running simulations. Workflow management tools can be used to automate repetitive processing tasks and make processing pipelines more robust.
Visualization
Visualization facilitates data exploration. It often supports simulation since it allows inspection of the output varying in time or with changes in parameter values, or for the locations of interesting regions in large data sets. Read More »
Data Exploration
Among the modalities of data exploration, visual exploration is often necessary to guarantee that the algorithms are performing what they are supposed using some dataset. This page illustrates some of the scientific problems explored in terms of visualization tools. Read More »
Data Analysis and Mining
Data analysis techniques include post-processing (e.g., data statistics) of experimental datasets and/or simulation output, as well as the use of mathematical methods (e.g., filtering data) and statistical tests. Data mining usually refers to the application of more advanced mathematical techniques such as classification, clustering, pattern recognition, etc. Read More »
Strategies for Choosing Analytics and Visualization Software and Hardware at NERSC
Data analysis (or analytics) and visualization are two steps in data understanding, often interleaved and symbiotic, so that many of the available tools characterized as one category, end-up having some functionalities of the other. Bellow find links to software tools grouped under Analytics or Visualization, but have in mind that their functions may be interchangeable. VisualizationAnalyticsVisit Matlab Python tools: Numpy, Scipy, iPython, matplotlib Paraview Mathematica Perl IDL Python TCL/TK… Read More »
Workflow Management
Workflow management refers to the process of connecting various software tools based on specific input and output parameters. The goal of workflow management is to automate a specific sets of tasks. Read More »
