SciNet is the supercomputer centre at the University of Toronto, hosting Niagara, one of the fastest supercomputers in Canada. SciNet provides Canadian researchers with computational resources and expertise necessary to perform their research on scales not previously possible in Canada.
We help power work from the biomedical sciences and aerospace engineering to astrophysics and climate science. Our free education and training program for students and users helps them obtain advanced computing skills and knowledge needed in their research and in the workplace. SciNet provides resources for the Digital Research Alliance of Canada, and is funded by Innovation, Science and Economic Development Canada (ISED), the Government of Ontario, and the University of Toronto. SciNet also provides and hosts resources for the Southern Ontario Smart Computing Platform (SOSCIP) funded by FedDev Ontario.
This year starts with the publication of three papers by SciNet analysts in the Journal of Computational Science Education that are a reflection of SciNet’s ten years of training and educating academic researchers in the practical and scalable use of high performance computing.
Bridging the Educational Gap between Emerging and Established Scientific Computing Disciplines
Marcelo Ponce, Erik Spence, Ramses van Zon, and Daniel Gruner, Journal of Computational Science Education Vol 10 (1) 4-11 (2019) https://doi.org/10.22369/issn.2153-4136/10/1/1
In this paper, we describe our experience in developing curriculum courses aimed at graduate students in emerging computational fields, including biology and medical science. We focus primarily on computational data analysis and statistical analysis, while at the same time teaching students best practices in coding and software development. Our approach combines a theoretical background and practical applications of concepts. The outcomes and feedback we have obtained so far have revealed several issues: students in these particular areas lack instruction like this although they would tremendously benefit from it; we have detected several weaknesses in the formation of students, in particular in the statistical foundations but also in analytical thinking skills. We present here the tools, techniques and methodology we employ while teaching and developing this type of courses. We also show several outcomes from this initiative, including potential pathways for fruitful multi- disciplinary collaboration.
Scientific Computing, High-Performance Computing and Data Science in Higher Education
Marcelo Ponce, Erik Spence, Ramses van Zon, and Daniel Gruner, Journal of Computational Science Education Vol 10 (1), 24-31 (2019) https://doi.org/10.22369/issn.2153-4136/10/1/5
We present an overview of current academic curricula for Scientific Computing, High-Performance Computing and Data Science. After a survey of current academic and non-academic programs across the globe, we focus on Canadian programs and specifically on the education program of the SciNet HPC Consortium, using its detailed enrollment and course statistics for the past six to seven years. Not only do these data display a steady and rapid increase in the demand for research-computing instruction, they also show a clear shift from traditional (high performance) computing to data- oriented methods. It is argued that this growing demand warrants specialized research computing degrees.
Trends in Scientific Computing Training Delivered by a High-Performance Computing Center
Ramses van Zon, Marcelo Ponce, Erik Spence, and Daniel Gruner, Journal of Computational Science Education Vol 10 (1), 53-60 (2019) https://doi.org/10.22369/issn.2153-4136/10/1/9
We analyze the changes in the training and educational efforts of the SciNet HPC Consortium, a Canadian academic High Performance Computing center, in the areas of Scientific Computing and High-Performance Computing, over the last six years. Initially, SciNet offered isolated training events on how to use HPC systems and write parallel code, but the training program now consists of a broad range of workshops and courses that users can take toward certificates in scientific computing, data science, or high-performance computing. Using data on enrollment, attendence, and certificate numbers from SciNet’s education website, used by almost 1800 users so far, we extract trends on the growth, demand, and breadth of SciNet’s training program. Among the results are a steady overall growth, a sharp and steady increase in the demand for data science training, and a wider participation of ‘non-traditional’ computing disciplines, which has motivated an increasingly broad spectrum of training offerings. Of interest is also that many of the training initiatives have evolved into courses that can be taken as part of the graduate curriculum at the University of Toronto.
This is the fourth of a series of posts on the transition to SciNet’s new supercomputer called “Niagara”, which will replace the General Purpose Cluster (GPC) and Tightly Coupled Cluster (TCS). The transition to Niagara will take place in the fall of 2017, and the system is planned to be available to users in early 2018.
The University of Toronto has awarded the contract for Niagara to Lenovo, and some of the details of the hardware specifications of the Niagara system have been released:
The system will have the following hardware components:
1,500 nodes.
Each node will have 40 Intel Skylake cores (making a total of 60,000 cores) at 2.4 GHz.
Each node will have 200 GB (188 GiB)of DDR4 memory.
The interconnect between the nodes will be Mellanox EDR Infiniband in a Dragonfly+ topology.
A ~9PB usable shared parallel filesystem (GPFS) will be mounted on all nodes.
A 256TB Excelero burst buffer (NVMe fabric, up to 160 GB/s) will be available for fast I/O.
Niagara is estimated to be installed and operational towards in March 2018, and ready for users not too long after.
Even before official ready-date, there will a period in which select users can try out and port their codes to Niagara.
After the friendly-user period, all current users of the GPC (and former users of the TCS) will get access to Niagara.
The large core count, ample memory per core, and fast interconnect support Niagara’s intended purpose to enable large parallel compute jobs of 512 cores or more.
The software setup will also be tailored to large parallel computations. Nonetheless, there will still be a fair amount of backfill opportunity for smaller jobs.
The setup of Niagara is intended to be similar in spirit to the GPC, but different in form: scheduling per node, a home, scratch and possibly project directory defined in environment variables, a module system, and access to our team of analyst to help you get your codes running, and running well.
Excited about the beginning of a new academic year?
We, at SciNet, certainly are!
SciNet has created several new courses for this coming fall semester and we are really excited about that!
Take a look at our education website to learn about all the courses and workshops that we will be offering.
In addition to the traditional courses on Scientific Computing, we have also added courses on Computational BioStatistics, Machine Learning and Neural Networks, and basic level introductory courses for students without any previous background on computing or programming!
Additionally, several members of our team have obtained Graduate restricted appointed positions at the Institute of Medical Sciences and the Physics Department!
The number of SciNet courses that are listed as U of T graduate courses continues to increase (no small feat for a non-teaching unit like SciNet). Our full-term graduate courses in 2017/2018 are
Finally, starting this September we want to officially launch our “Research Initiative Program”!
This is a collaborative program, aimed to partner with research groups across the University, in order to boost and empower research.
Of course, research support is something that we have been doing since the beginning of SciNet, by providing technical support and the infrastructure to researchers for tackling their computational needs.
This program will go beyond that, by allowing researchers to explicitly partner with SciNet’s scientists, in order to pursue short and long term research projects.
More information about this program, ongoing collaborations and areas of expertise can be found at the Research @ SciNet page.
Comments Off on New Courses and New Initiatives for this Coming Semester
The Compute Ontario Summer School on Scientific and High Performance Computing is an annual educational event for graduate/undergraduate students, postdocs and researchers who are engaged in a compute intensive research. Held geographically in the west, centre and east of the province of Ontario, the summer school provides attendees with the opportunity to learn and share knowledge and experience in high performance and technical computing on modern HPC platforms.
Each site will have a slightly different list of courses. The summer school will include both in-class lectures and hands-on labs (done on the participants’ laptops). Those who attend at least three full days cumulatively will receive an official certificate in HPC training (i.e., a total of 6 full morning and afternoon sessions).
Instructors for this school have been provided by SciNet, CAMH and SHARCNET. Break refreshments are provided courtesy of Compute Ontario.
REGISTRATION
Registration for the central installment in Toronto from July 24-28, 2017 is now open!
The registration is free and is aimed at Compute Canada users as well as students, post-docs and other researchers from academic institutions. You do not need to have a SciNet account. Please be advised that seats are limited and tend to fill up.
This event will be held in the Medical Science Building at the University of Toronto, 1 King Circle, Toronto, Ontario, M5S 1A8, Canada.
The nearest subway station is “Queen’s Park”. Paid parking is available on the St. George Campus.
LODGING
Lodging is not provided by the organization. If you require lodging, you will have to make arrangements yourself. It may be worthwhile checking out the University’s summer residence program at www.studentlife.utoronto.ca/hs/summer .
MEALS
Meals are *not* provided by the organization, but refreshments will be provided during the morning and afternoon breaks, courtesy of Compute Ontario.
COMPUTING FACILITIES
For the hands-on sessions, participants are to bring their own laptop with working wireless and with an ssh client with X-windows installed. The latter is needed to connect to one of SciNet or SHARCNET supercomputers, to which the participant will get access for the duration of the School.
CERTIFICATES
Participants that complete at least three days worth of instruction (i.e., a total of 6 morning and afternoon sessions combined) are to receive a Compute Ontario Summer School Certificate on the last day of the School. Note that this certificate is separate from the SciNet certificates, but parts of the school may count towards a SciNet certificate as well.
Comments Off on 2017 Compute Ontario Summer School Central
On May 9, 2017, the Grand Opening of the ArcNet space took place (despite having moved a while ago). What is ArcNet? It is a space where expertise and support of Advanced Research Computing (the “ARC” in ArcNet) from three organizations come together. SciNet is the oldest of the three; It is the supercomputing consortium at the University of Toronto, which has been providing Canadian researchers with computational resources and expertise necessary to perform their research on scales not previously possible in Canada, from the biomedical sciences and aerospace engineering to astrophysics and climate science. SOSCIP is a research and development consortium that pairs academic and industry researchers with advanced computing tools to fuel Canadian innovation. The third organization, Compute Ontario, partners with the four academic computing consortia in Ontario aims to drive advanced computing to accelerate research and enhance competitiveness in the global marketplace resulting in a more prosperous Ontario.
The Grand Opening brought together many of our stakeholders, and was also attended by the Ontario Minister of Research, Innovation and Science, MPP Rezi Moridi, by the President of the University of Toronto Meric Gertler, by the Vice-President of Research at the University of Toronto Prof. Vivek Goel, and by the Scientific Directory of SciNet Prof. W. Richard Peltier. We were honoured that each was willing to say a few words about the opening of this space.
“This facility is a true partnership between the University of Toronto’s SciNet High Performance Computing Consortium, SOSCIP and Compute Ontario. Bringing them together in a state-of-the-art facility will strengthen their partnership and undoubtably create new opportunities to drive innovation through advanced computing in Ontario.” said Prof. Goel, one of the driving forces of the creation of the space.
The President of the University of Toronto reminded us that “With ARCNet, we have created an amazing hub of talent and technology that fosters collaboration between the public and private sector.”
Minister Rezi Moridi remarked that “Today we are here to celebrate the grand opening of the Advanced Research Computing facility. It is great to see that three organizations got together and set up this wonderful facility: SOSCIP,
Compute Ontario, and University of Toronto’s SciNet. I wish you all the best in serving our research community.”
The speeches were given in the new teaching and visualization room. This room holds up to 40 students and is already frequently used for courses and other events. It features a large visualization wall, i.e., a 13 x 7.5 feet ultrahigh resolution screen (8K, to be precise).
Two presentations were given by SciNet analysts to demonstrate the capabilities of this visualization wall; Ramses van Zon showed how to get insight into the complexity of the software installed on SciNet’s main cluster by using graph visualizations, while Marcelo Ponce showed visualizations of several aspects of interacting neutron stars, with data from numerical general relativity simulations.
You can see the capabilities of the visualization wall in the following video:
Comments Off on Grand Opening of the ArcNet Space at MaRS
HPCS (the High Performance Computing Symposium) is Canada’s premiere Advanced Research Computing (ARC) conference, bringing together top researchers from across Canada and around the world, as well as major industry partners.
This year’s conference is being held in Kingston, Ontario – June 5th – 9th, and will include a range of keynote sessions and technical workshops designed to appeal to the research community and ARC professionals. Topics will include “traditional” HPC disciplines, as well as emerging areas such as cognitive computing – and there will have sessions exploring future technologies.
The eighth International Summer School on HPC Challenges in Computational Sciences will be held from June 25-July 30, 2017, Boulder, Colorado, USA. This is an advanced summer school on High Performance Computing which targets graduate students who already have some experience in HPC parallel programming (for instance, MPI, OpenMP, or CUDA/OpenCL), preferably on software used in successful research projects.
Leading American, Canadian, European and Japanese computational scientists and HPC technologists will offer instruction on a variety of topics. The program is still being finalized, but previous summer schools included the following:
Access to EU, Canadian, Japanese and U.S. HPC-infrastructures
HPC challenges by discipline (e.g., bioinformatics, computer science, chemistry, and physics)
HPC Programming Proficiencies
Performance analysis & profiling
Algorithmic approaches & numerical libraries
Data-intensive computing
Scientific visualization
Participation in the summer school is decided through an application process. Meals, housing, and travel will be covered for the selected participants. Applications from students in all science and engineering fields are welcome. Ten out of 80 student participants will be from Canada. Preference will be given to applicants with parallel programming experience, and a research plan that will benefit from using high performance computing systems.
Applications are due by March 6, 2017
For further information and to apply online, please click here.
Comments Off on International HPC Summer School 2017 in Boulder, CO
The High Performance Storage System (HPSS) is a tape-backed hierarchical storage system that provides a significant portion of the allocated storage space at SciNet. It is a repository for archiving data that is not being actively used. Data can be returned to the active filesystem on the compute clusters when it is needed.
SciNet’s HPSS currently has nearly 90 PB of capacity.
For more information, see the technical documentation on the SciNet wiki
Comments Off on High Performance Storage System (HPSS)
On February 11th, 2016 (10:30AM EST), scientists from Caltech, MIT and the LIGO Scientific Collaboration together with representatives of the National Science Foundation, announced in a live press conference, the first direct detection of gravitational waves.
The event detected, named GW150914, was produced by two colliding black holes, inspariling and merging together. This signal was detected by LIGO on September 14th, 2015.
Details of this discovery can be found in the following papers:
Many of the researchers running simulations and analysing data in several of our clusters are directly involved in the efforts for accurately modelling, simulating and detecting gravitational wave signals.
As a matter of fact, one of the simulations used for visualizing one of the movies screened during the announcement of the discovery of gravitational waves, was performed on SciNet’s General Purpose Cluster (GPC).
“The first detection of gravitational waves passing through Earth is a tremendous discovery. These waves were generated by never before observed astronomical objects, colliding black holes. I am very grateful for SciNet and Compute Canada to provide the computing resources needed to explore the properties of binary black holes, research that was instrumental in building the waveform-templates that LIGO used in its momentous discovery.”
— Prof. Harald Pfeiffer (Canadian Institute for Theoretical Astrophysics, University of Toronto)
These simulations were performed by solving Einstein’s equations of General Relativity using a code co-developed on SciNet, by the SXS/CITA collaboration.
Credit: SXS/CITA/SciNet
Another of the videos displayed on the press-conf, was also produced by solving Einstein equations using this same computer code co-developed on SciNet by the SXS/CITA collaboration.
