Launch of the Niagara Supercomputer at SciNet

March 5, 2018 in for_educators, for_press, for_researchers, for_users, frontpage, in_the_news, news, Road_to_Niagara

The Niagara supercomputer was officially launched on March 5th, 2018. We were honoured by the presence and remarks of Reza Moridi (Ontario Minister of Research, Innovation and Science), Nizar Ladak (Compute Ontario President and CEO), Dr. Roseann O’Reilly Runte (CFI President and CEO), Prof. Vivek Goel (Vice-president of Research and Innovation at the University of Toronto), and Prof. W. Richard Peltier (Scientific Director of SciNet).

SciNet’s CTO Daniel Gruner gave an overview of the new system:

Niagara is located at University of Toronto and operated by the university’s high-performance computing centre SciNet, but the system is open to all Canadian university researchers.

Niagara is the fastest computer system in the country and is able to run a single job across all 60,000 cores thanks to a high-performance network which interconnects all the nodes. For more information on the configuration, see here.

A time-lapse of the building of Niagara is available (part of SciNet’s YouTube channel):

This system is jointly funded by the Canada Foundation for Innovation, the Government of Ontario, and the University of Toronto.

Road to Niagara 3: Hardware setup

March 5, 2018 in blog-technical, for_press, for_researchers, for_users, news, Road_to_Niagara, Uncategorized

This is the fourth of a series of posts on the transition to SciNet’s new supercomputer called “Niagara”, which will replace the General Purpose Cluster (GPC) and Tightly Coupled Cluster (TCS). The transition to Niagara will take place in the fall of 2017, and the system is planned to be available to users in early 2018.

The University of Toronto has awarded the contract for Niagara to Lenovo, and some of the details of the hardware specifications of the Niagara system have been released:

The system will have the following hardware components:

  • 1,500 nodes.
  • Each node will have 40 Intel Skylake cores (making a total of 60,000 cores) at 2.4 GHz.
  • Each node will have 200 GB (188 GiB)of DDR4 memory.
  • The interconnect between the nodes will be Mellanox EDR Infiniband in a Dragonfly+ topology.
  • A ~9PB usable shared parallel filesystem (GPFS) will be mounted on all nodes.
  • A 256TB Excelero burst buffer (NVMe fabric, up to 160 GB/s) will be available for fast I/O.
  • Peak theoretical speed: 4.61 PetaFLOPS

Niagara is estimated to be installed and operational towards in March 2018, and ready for users not too long after.

Even before official ready-date, there will a period in which select users can try out and port their codes to Niagara.

After the friendly-user period, all current users of the GPC (and former users of the TCS) will get access to Niagara.

The large core count, ample memory per core, and fast interconnect support Niagara’s intended purpose to enable large parallel compute jobs of 512 cores or more.

The software setup will also be tailored to large parallel computations. Nonetheless, there will still be a fair amount of backfill opportunity for smaller jobs.

The setup of Niagara is intended to be similar in spirit to the GPC, but different in form: scheduling per node, a home, scratch and possibly project directory defined in environment variables, a module system, and access to our team of analyst to help you get your codes running, and running well.

International HPC Summer School 2018 in Ostrava, by SciNet, XSEDE, PRACE and Riken

December 22, 2017 in for_educators, for_researchers, for_users, frontpage, in_the_news, news

A High-Performance Computing Summer Institute
July 8-13, Ostrava, Czech Republic
Expenses-paid program
Apply by February 13, 2018

Graduate students and postdoctoral scholars from institutions in Canada, Europe, Japan and the United States are invited to apply for the ninth International Summer School on HPC Challenges in Computational Sciences, to be held July 8 to 13, 2018, in Ostrava, in the Czech Republic, and hosted by the IT4Innovations National Supercomputing Centre.

Applications are due Feb 13, 2018. The summer school is organized by the Partnership for Advanced Computing in Europe (PRACE), the Extreme Science and Engineering Discovery Environment (XSEDE), the RIKEN Advanced Insti­tute for Computational Science (RIKEN AICS), and the SciNet HPC Consortium.

Leading computational scientists and HPC technologists from the U.S., Europe, Japan and Canada will offer instructions on a variety of topics and also provide advanced mentoring. Topics include:

  • HPC challenges by discipline
  • HPC programming proficiencies
  • Performance analysis & profiling
  • Algorithmic approaches & numerical libraries
  • Data-intensive computing
  • Scientific visualization
  • Canadian, EU, Japanese and U.S. HPC-infrastructures

The expense-paid program will benefit scholars from Canadian, European, Japanese and U.S. institutions who use advanced computing in their research. The ideal candidate will have many of the following qualities, however this list is not meant to be a “checklist” for applicants to meet all criteria:

  • Familiar with HPC, not necessarily an HPC expert, but rather a scholar who could benefit from including advanced computing tools and methods into their existing computational work
  • A graduate student with a strong research plan or a postdoctoral fellow in the early stages of their research efforts
  • Regular practice with parallel programming (i.e., student utilizes parallel programming generally on a monthly basis or more)
  • May have a science or engineering background, however, applicants from other disciplines are welcome provided their research activities include computational work.

Students from underrepresented groups in computing are highly encouraged to apply (i.e., women, racial/ethnic minorities, persons with disabilities, etc.). If you have any questions regarding your eligibility or how this program may benefit you or your research group, please do not hesitate to contact the individual associated with your region below.

Interested students should apply by February 13, 2018. Participation, meals and housing will be covered for the selected participants, also support for intercontinental travel will be given (contingent to funding).

Further information and application, see

High Performance Computing Analyst – Post Doctoral Fellow (IBM/SOSCIP)

November 16, 2017 in for_researchers, for_users, HPC Jobs, HPC Jobs Ontario, news

SOSCIP is currently looping for a new BlueGene/Q support post-doc.

Job Description
In partnership with 15 leading Ontario academic institutions, federal and provincial governments, IBM is supporting collaborative research projects that leverage high performance computing to drive commercial outcomes for social and economic development in Ontario. We have announced over 40 exciting research projects aligned to Health, Energy, Cities, Water, Mining, Cybersecurity, Digital Media, Advanced Manufacturing and Agile Computing. We are now hiring to support these projects.

Specialist – IBM Blue Gene/Q supercomputer based at University of Toronto.

This role is for a cross-team specialist who will support IBM Blue Gene/Q, the largest supercomputer in Canada. The selected candidate will be part of a team based at University of Toronto and provide parallel programming support to research teams. The research teams include faculty, postdoctoral fellows, graduate students and other industry partners. This role is pivotal to advising, building and optimizing applications to leverage massively parallel platforms and drive innovation to impact the lives of Canadians.

Job Responsibilities:

• Assist research teams in making effective use of massively parallel platforms including one of the largest supercomputers in Canada and a new research platform for exascale computing
• Develop, port, optimize and analyze scientific numerical codes using thousands of CPU cores on the Blue Gene Q using Fortran, Python, C, C++, etc
• Parallel-programming assistance on smaller-scale x86 and Power clusters, possibly including GPU’s
• Install, upgrade and lead day-to-day operations for scientific infrastructure including mathematical software packages, libraries, compilers, visualization software, schedulers etc
• promote, advise and teach mini-courses on scientific computing and parallel programming

Click here for more information

Expired: Job Opportunity at SciNet: HPC Analyst

October 20, 2017 in for_researchers, for_users, HPC Jobs, HPC Jobs Ontario

The SciNet HPC Consortium is looking to augment their team with an HPC Analyst.

Title of job: Scientific Applications Analyst

Location: SciNet HPC Consortium, University of Toronto, Toronto, Ontario, Canada

Summary of job:

The Scientific Applications Analyst provides senior IT services and training in parallel programming, data science applications, and scientific computing workflows for the SciNet High Performance Computing (HPC) consortium which serves researchers at the University of Toronto including faculty, postdoctoral fellows and graduate students in all disciplines and fields (e.g. science and engineering, medicine, finance, languages, etc.). The incumbent is involved in massively-parallel computing (single jobs that make use of up to 40,000 cores) and data analytics and machine learning on large data sets (100TB and up). S/he works with researchers and research teams to plan, develop, install and optimize the initial SciNet systems for various research programs and provides technical consultation to researchers on their system needs for research operations. S/he also takes part in delivering and developing SciNet’s training and education program. SciNet operates large High Performance Computing (HPC) systems, and acquires the largest HPC cluster in Canada by the end of 2017. SciNet provides HPC resources and support to researchers at the University of Toronto, the affiliated research hospitals, and other Canadian universities.

Salary: CAD$94,672 with an annual step progression to a maximum of CAD$121,069. Pay scale and job class assignment is subject to determination pursuant to the Job Evaluation/Pay Equity Maintenance Protocol.

Closing date: Oct 26, 2017, 11:59:00 PM EST

For more details, see the job site of the University of Toronto.

2017 Compute Ontario Summer School Central

June 14, 2017 in blog, blog-general, for_educators, for_press, for_researchers, for_users, frontpage, news, Uncategorized

The Compute Ontario Summer School on Scientific and High Performance Computing is an annual educational event for graduate/undergraduate students, postdocs and researchers who are engaged in a compute intensive research. Held geographically in the west, centre and east of the province of Ontario, the summer school provides attendees with the opportunity to learn and share knowledge and experience in high performance and technical computing on modern HPC platforms.

Each site will have a slightly different list of courses. The summer school will include both in-class lectures and hands-on labs (done on the participants’ laptops). Those who attend at least three full days cumulatively will receive an official certificate in HPC training (i.e., a total of 6 full morning and afternoon sessions).

Instructors for this school have been provided by SciNet, CAMH and SHARCNET. Break refreshments are provided courtesy of Compute Ontario.


Registration for the central installment in Toronto from July 24-28, 2017 is now open!

The registration is free and is aimed at Compute Canada users as well as students, post-docs and other researchers from academic institutions. You do not need to have a SciNet account. Please be advised that seats are limited and tend to fill up.

More information and registration can be found on the summer school website.


High Performance Computing Stream Data Science Stream Biomedical Stream
Mon, Jul 24
Morning: 09:00-12:00
Welcome and Introduction to HPC and SciNet
Afternoon: 13:30-16:30
Shared Memory Programming with OpenMP Introduction to the Linux Shell PLINK
Tue, Jul 25
Morning: 09:00-12:00
Shared Memory Programming with OpenMP Introduction to R Next Generation Sequencing
Afternoon: 13:30-16:30
Programming Clusters with Message Passing Interface Data Science with Python RNASeq
Wed, Jul 26
Morning: 09:00-12:00
Programming Clusters with Message Passing Interface Parallel R for Data Science Python for MRI analysis
Afternoon: 13:30-16:30
Programming Clusters with Message Passing Interface Python for High Performance Computing (Parallel Python) Image Analysis at Scale
Thu, Jul 27
Morning: 09:00-12:00
Programming GPUs with CUDA Visualization with Python Machine Learning for Neuroimaging
Afternoon: 13:30-16:30
Programming GPUs with CUDA Scientific Visualization Suites R for MRI analysis
Fri, Jul 28
Morning: 09:00-12:00
Programming GPUs with CUDA Debugging, Profiling and Bring-Your-Own-Code Lab Public Datasets for Neuroimaging
Afternoon: 13:30-16:30
Programming GPUs with CUDA Debugging, Profiling and Bring-Your-Own-Code Lab Unit Testing / Neuroinformatics Pipeline Development


This event will be held in the Medical Science Building at the University of Toronto, 1 King Circle, Toronto, Ontario, M5S 1A8, Canada.

The nearest subway station is “Queen’s Park”. Paid parking is available on the St. George Campus.


Lodging is not provided by the organization. If you require lodging, you will have to make arrangements yourself. It may be worthwhile checking out the University’s summer residence program at .


Meals are *not* provided by the organization, but refreshments will be provided during the morning and afternoon breaks, courtesy of Compute Ontario.


For the hands-on sessions, participants are to bring their own laptop with working wireless and with an ssh client with X-windows installed. The latter is needed to connect to one of SciNet or SHARCNET supercomputers, to which the participant will get access for the duration of the School.


Participants that complete at least three days worth of instruction (i.e., a total of 6 morning and afternoon sessions combined) are to receive a Compute Ontario Summer School Certificate on the last day of the School. Note that this certificate is separate from the SciNet certificates, but parts of the school may count towards a SciNet certificate as well.

Grand Opening of the ArcNet Space at MaRS

May 15, 2017 in blog-general, for_educators, for_press, for_researchers, for_users, frontpage, news, Uncategorized

On May 9, 2017, the Grand Opening of the ArcNet space took place (despite having moved a while ago). What is ArcNet? It is a space where expertise and support of Advanced Research Computing (the “ARC” in ArcNet) from three organizations come together. SciNet is the oldest of the three; It is the supercomputing consortium at the University of Toronto, which has been providing Canadian researchers with computational resources and expertise necessary to perform their research on scales not previously possible in Canada, from the biomedical sciences and aerospace engineering to astrophysics and climate science. SOSCIP is a research and development consortium that pairs academic and industry researchers with advanced computing tools to fuel Canadian innovation. The third organization, Compute Ontario, partners with the four academic computing consortia in Ontario aims to drive advanced computing to accelerate research and enhance competitiveness in the global marketplace resulting in a more prosperous Ontario.

The Grand Opening brought together many of our stakeholders, and was also attended by the Ontario Minister of Research, Innovation and Science, MPP Rezi Moridi, by the President of the University of Toronto Meric Gertler, by the Vice-President of Research at the University of Toronto Prof. Vivek Goel, and by the Scientific Directory of SciNet Prof. W. Richard Peltier. We were honoured that each was willing to say a few words about the opening of this space.

“This facility is a true partnership between the University of Toronto’s SciNet High Performance Computing Consortium, SOSCIP and Compute Ontario. Bringing them together in a state-of-the-art facility will strengthen their partnership and undoubtably create new opportunities to drive innovation through advanced computing in Ontario.” said Prof. Goel, one of the driving forces of the creation of the space.

The President of the University of Toronto reminded us that “With ARCNet, we have created an amazing hub of talent and technology that fosters collaboration between the public and private sector.”

Minister Rezi Moridi remarked that “Today we are here to celebrate the grand opening of the Advanced Research Computing facility. It is great to see that three organizations got together and set up this wonderful facility: SOSCIP,
Compute Ontario, and University of Toronto’s SciNet. I wish you all the best in serving our research community.”

The speeches were given in the new teaching and visualization room. This room holds up to 40 students and is already frequently used for courses and other events. It features a large visualization wall, i.e., a 13 x 7.5 feet ultrahigh resolution screen (8K, to be precise).

Two presentations were given by SciNet analysts to demonstrate the capabilities of this visualization wall; Ramses van Zon showed how to get insight into the complexity of the software installed on SciNet’s main cluster by using graph visualizations, while Marcelo Ponce showed visualizations of several aspects of interacting neutron stars, with data from numerical general relativity simulations.

You can see the capabilities of the visualization wall in the following video:

Sheridan Cyber Security Symposium 3.0

May 6, 2017 in for_educators, for_industry, for_researchers, for_users, news

The Sheridan Cyber Security Symposium 3.0, will be held on Monday, May 15, 2017 at the Hazel McCallion Campus of Sheridan College in Mississauga.

This conference is hosted by the Sheridon College Faculty of Applied Science and Technology in collaboration with Sheridan Applied Research and Innovation, and will explore cyber security within public health & safety, community, education and business.

For more information and registration see the symposium’s website.

Expired: HPCS 2017, Kingston June 5-9, 2017

April 25, 2017 in blog-general, for_researchers, for_users, news, Uncategorized

HPCS (the High Performance Computing Symposium) is Canada’s premiere Advanced Research Computing (ARC) conference, bringing together top researchers from across Canada and around the world, as well as major industry partners.

This year’s conference is being held in Kingston, Ontario – June 5th – 9th, and will include a range of keynote sessions and technical workshops designed to appeal to the research community and ARC professionals. Topics will include “traditional” HPC disciplines, as well as emerging areas such as cognitive computing – and there will have sessions exploring future technologies.

For more information see

Note that Ontario students can go for fee (see the flyer).

International HPC Summer School 2017 in Boulder, CO

January 25, 2017 in for_educators, for_researchers, for_users, frontpage, news, Uncategorized


Apply by 6 March, 2017
Expenses-paid program
Sponsored by PRACE, XSEDE, Riken, and Compute Canada

The eighth International Summer School on HPC Challenges in Computational Sciences will be held from June 25-July 30, 2017, Boulder, Colorado, USA. This is an advanced summer school on High Performance Computing which targets graduate students who already have some experience in HPC parallel programming (for instance, MPI, OpenMP, or CUDA/OpenCL), preferably on software used in successful research projects.

The organizers of this summer school are XSEDE, PRACE, Compute Canada, and RIKEN.

Leading American, Canadian, European and Japanese computational scientists and HPC technologists will offer instruction on a variety of topics. The program is still being finalized, but previous summer schools included the following:

  • Access to EU, Canadian, Japanese and U.S. HPC-infrastructures
  • HPC challenges by discipline (e.g., bioinformatics, computer science, chemistry, and physics)
  • HPC Programming Proficiencies
  • Performance analysis & profiling
  • Algorithmic approaches & numerical libraries
  • Data-intensive computing
  • Scientific visualization

Participation in the summer school is decided through an application process. Meals, housing, and travel will be covered for the selected participants. Applications from students in all science and engineering fields are welcome. Ten out of 80 student participants will be from Canada. Preference will be given to applicants with parallel programming experience, and a research plan that will benefit from using high performance computing systems.

Applications are due by March 6, 2017
For further information and to apply online, please click here.