New Features of the Edison XC30
While the Edison and Hopper systems have similar programming environments and software, there are some key architectural differences between the two systems. This page describes those differences.
Edison and Hopper both have a total of 24 cores on each compute node. Edison, like Hopper, has two sockets on each compute node, but instead of four "NUMA" memory domains, Edison has only two. Edison uses Intel processors, unlike Hopper which has processors from AMD. Edison's processors have Intel Hyper-Threading (HT) enabled, which means you can run with 48 logical cores per node. At run time you can decide to run with 24 cores per node (the default setting) or 48 logical cores per node.
24 cores per node (48 logical cores with Hyper-Threading). Dual-socket, 12-core processor node, Intel Xeon "Ivy Bridge" @ 2.4 GHz
|24 cores per node. Dual-socket, 12-core processor node, AMD Opteron @2.1 GHz|
|64 GB memory per node (DDR3 memory at 1866 MHz); 2.67GB/core.
||32 GB memory per node (DDR3 1333 MHz memory); 1.33 GB/core.|
|5192 nodes||6384 nodes|
Edison uses Cray Aries interconnect for inter-node communication, while Hopper uses the Cray Gemini network. Aries provides a higher bandwidth, lower latency interconnect than Gemini, and should exhibit reduced network congestion. Edison's Aries network is connected through a "Dragonfly" topology, compared to Hopper's torus network. See the Configuration section for more details.
External Login Nodes
Like Hopper, Edison has 12 login nodes that are "external" to the main compute portion of the system. Since the login nodes are external, you can login, access file systesms and submit jobs when the main compute portion of the sytem is down for maintenance. The login nodes on Edison have 512 GB of memory, compared to 128 GB on Hopper.
|12 quad-socket, quad-core nodes (16 cores per node w/ Hyper-Threading 32 logical cores), Intel Xeon "Sandy Bridge" @ 2.0 GHz||12 quad-socket, quad-core nodes (16 cores per node), AMD Opteron @2.0 GHz|
|512 GB memory per login node||128 GB memory per login node|
Scratch File Systems
Edison has 7.5 PB of scratch spread over three disk partitions. The aggregate I/O bandwidth is 165 GB/sec, while Hopper has 2 PB disk with 70 GB/s I/O bandwidth. All users have a scratch area defined by the environment variable $SCRATCH on one of two smaller scratch partitions. A third larger file system is dedicated to users who have a demonstrated need for large I/O bandwidth. Users can request access to /scratch3 via a web form. Edison uses Cray Sonexion lustre storage system. The default scratch quota is 10TB.
|File systems||Size (PB)||Aggregate Peak Performance (GB/s)||# IO Servers (OSSs)||OSTs||File System Software||Disk Array Vendor||User quota(TB)|