Niagara Supercomputer

March 5, 2018 in Systems

Niagara is a homogeneous cluster of initially 61,920 cores but expanded (in 2020) to 80,640 cores, owned by the University of Toronto and operated by SciNet. This system is intended to enable large parallel jobs of 1024 cores and more. It is the one of the most powerful supercomputers in Canada available for academic research. Compute allocations are handled through Compute Canada’s annual resource allocation competition. Niagara was designed to optimize throughput of a range of scientific codes running at scale, energy efficiency, and network and storage performance and capacity.

Niagara was officially launched on March 5, 2018. The expansion became available in March 2020.

Specifics of the cluster:

  • Niagara consists of 2,016 nodes.
  • Each node has 40 Intel Skylake cores at 2.4GHz or Cascaselake cores at 2.5GHz, for a total of 80,640 cores.
  • There is 202 GB (188 GiB) of RAM per node.
  • EDR Infiniband network in a so-called ‘Dragonfly+’ topology.
  • 6PB of scratch, 3PB of project space (parallel filesystem: IBM Spectrum Scale, formerly known as GPFS).
  • 256 TB burst buffer (Excelero + IBM Spectrum Scale).
  • No local disks.
  • Theoretical peak performance (“Rpeak”) of 6 PF.
  • Estimated peak performance (“Rmax”) of 4 PF.
  • About 900 kW power consumption.