2024 Compute Ontario Summer School

May 16, 2024 in blog, for_researchers, for_users, frontpage, news, Training, Uncategorized

Empower your Research: Expand Your Knowledge

We are excited to announce that registration is now open for the highly anticipated 2024 Compute Ontario Summer School!

Jointly organized by the Centre for Advanced Computing, SciNet, SHARCNET, and in collaboration with the Alliance and RDM experts from across Ontario and Canada, this virtual event promises to be an enriching experience for all participants.

Taking place from June 3 to June 21, the Compute Ontario Summer School offers a comprehensive curriculum packed with 40 courses. Delivered by experts in the field, these sessions cover a wide range of topics including Advanced Research Computing (ARC), High Performance Computing (HPC), Research Data Management (RDM), and Research Software (RS). With presentations and workshops available at introductory to intermediate levels, there is something for everyone.

Highlights of the Summer School:

  • It’s free!
  • All courses delivered online
  • Pick-and-choose the course(s) you want to attend
  • Many courses include a hands-on component
  • Courses range in length from 1.5 hours to three days
  • Course levels range from beginner to intermediate
  • Topics covered include: AI, machine learning, bioinformatics, GPU programming, advanced research computing basics, high-performance computing tools, programming languages, visualization, research data management, and more.

Please note that space is limited, so we encourage you to register for your desired course(s) as soon as possible in order to avoid missing out.

To register and learn more about the Compute Ontario Summer School, please visit our dedicated registration page.

Niagara at Scale Pilot

March 5, 2021 in blog-general, for_press, for_researchers, for_users, frontpage, news

SciNet will be reserving the Niagara cluster for two days in March for the first-ever “Niagara at Scale”, from March 30th, 2021, at 12 noon EST, to April 1st, 2021, at 12 noon EST.

Purpose of the “Niagara at Scale” event

This event will enable pre-approved projects that require all or nearly all of the capacity of the Niagara supercomputer at once. Such heroic computations are Niagara’s mandate, as it is the “Large Parallel cluster within the national systems of the Compute Canada Federation, and the fastest machine of its kind in Canada according to the TOP500 List. But computations of this size — think massively parallel codes running on tens of thousands of cores — are hard or impossible to run within the regular batch scheduler.

How to apply

We already have some groups interested in participating, but we would like to extend our invitation to the whole Canadian high-performance computing community before committing to a particular date. Users that have massively parallel jobs or workflows that could take advantage of this opportunity, are encourage to contact us at support@scinet.utoronto.ca by Friday, March 12, 2021 (note: this is an extension of the original deadline of March 5).

In the email, please briefly describe your intended computation, as well as the size and duration of the jobs you would like to run at scale.  Successful proposals will need to show evidence that their codes can run efficiently on at least 20,000 cores on Niagara and include strong and/or weak scaling data and plots.

In addition, your codes must be able to checkpoint and restart, especially since jobs will be restricted to shorter wall time.

Information session on March 10, 2021

We will hold an online information session regarding this program on March 10, 2021 at our SciNet User Group Meeting at noon EST. Attend to learn what kind of computations this program is aimed at. We will also provide guidance on how to get your computation to such a large scale if it needs it but your code does not yet scale to that size. For more information and sign-up for the event, go to https://scinet.courses/569

Future “Niagara at Scale” Events

The current event is a pilot project. If this initiative proves successful, we are planning to hold several of these events per year.

SciNet Virtual Summer Training Program

June 12, 2020 in blog, for_researchers, for_users

In lieu of its annual in person Ontario Summer School, this year, SciNet, in collaboration with CAMH, will be offering weekly virtual summer training on High Performance Computing from June through to August. The program consists of the following 11 courses, each with 3 online vents of 90 minutes on successive days within one week: a lecture, a hands-on session, and a wrap-up session.

The first of these courses, the “Introduction to Supercomputing” has just finished, with over 70 participants. Still to come in the next weeks are

  • Intro to Linux Shell
  • Parallel Programming on Multicore Computers with OpenMP
  • Parallel Programming at Scale on Supercomputers with MPI
  • Neuroimaging Analysis at Scale
  • Python for MRI Analysis
  • Python and High Performance Computing
  • Brain Network Modeling
  • R and High Performance Computing
  • Whole-Genome Association Analysis with PLINK
  • Debugging and Performance

See the program site on the SciNet education website for further details.

SciNet’s Summer School: a decade-old tradition

October 1, 2019 in blog, blog-general, for_educators, for_press, frontpage, news

Most would associate summertime with a relaxing and leisurely season of the year. However, HPC centres like SciNet, as in many others around the world, perceive this differently and are actually quite busy during this period.

Among the many activities SciNet carries out during the summer “break” are workshops and short courses. These activities are scheduled in the summer to fit between the term-long courses that SciNet offers to graduate students at the University of Toronto.

In particular, one of SciNet’s oldest training activities is a one-week intensive school on high-performance and technical computing. This annual summer school is our flagship training event, and is aimed at graduate students, undergraduate students, postdocs, researchers and occasionally even faculty members, who are engaged in compute intensive research. SciNet’s first such summer school was given in 2009, at which time it was called a “Parallel Scientific Computing” workshop. This first version of the school was heavily focused on parallel programming and applications in astrophysics.

These days, SciNet’s summer school is part of the Compute Ontario Summer School on Scientific and High Performance Computing. Held geographically in the west, centre and east of the province of Ontario in Canada, the summer school provides attendees with the opportunity to learn and share knowledge and experience in high performance and technical computing on modern HPC platforms. The central edition is the continuation of the SciNet summer school.

Not only is the school organized in a wider context, its program has expanded as well. In the last three years, the Toronto edition has had three streams with a wide variety of topics, from shell programming to data science, machine learning and neural networks, biomedical computing, and, still, parallel programming.

The type of training offered at the summer school is very practical, with a lot of hands-on exercises and live coding. This practical approach is very typical for most of SciNet’s courses but takes its ultimate form during the summer school instruction.

In addition to the training that participants received, the school also offers the opportunity of participants to interact with other participants, as well as the instructors, exchange ideas or discuss about current problems they are trying to solve. In fact, since a couple of years, the program includes focused sessions such as “Bring your own code” and “Bio-Hacking”, where this sort of interactions are not only promoted but the main theme.

Our summer school has the add-on feature of being absolutely free of charge for participants! That’s something we believe is quite important for several reasons, but mostly because we believe that in this way we can reach more researchers from fields that are relatively new to doing computational research.

This type of event not only benefits the students and participants of the summer school, but also enables collaborations between departments and consortia, as part of the training was delivered in partnership with colleagues from SHARCNET and the Centre for Addiction and Mental Health.

click on picture to enlarge

SciNet’s first summer school in 2009 focussed on Parallel Scientific Computing and placed emphasis on scientific applications such as in astrophysics.

click on picture to enlarge

SciNet’s latest (and largest) summer school, held in June 2019. This summer school had three parallel streams: the traditional High-Performance Computing, one on Data Science and a stream on BioInformatics/Medical applications, which was added in 2017. Details of the courses covered in the school can be found in SciNet education website: SciNet.courses/438

Logistics and Organizational details of the Summer School

There is no simple recipe to make a successful summer school that attracts and retains motivated participants for five full days, but below are a few necessary ingredients.

Sessions and instructors… Coming up with a program of three streams with sessions on scientific computing, parallel programming and data science is a challenge, but finding the excellent instructors for them is an even greater challenge, especially in summer, when many people are away.
Nonetheless, the summer school has been able to grow from a single-stream offering of 100 lecture hours in 2014 to a three-stream program with nearly 300 lecture hours in 2019. Luckily, we are not limited just to SciNet staff for instructors, but get help from the people from SHARCNET and CAMH as well.

Rooms… Organizing a training section of one-week long from Monday to Friday starting at 9:30am and finishing 4:30pm, offers a lot of challenges. For starting, finding rooms (not only one, but actually three –as there are three parallel concurrent sessions), ideally on the same building and each of them able of hosting around a hundred people, with proper power outlets, AC capabilities, and comfortable enough is a task far from trivial. We manage to do this, again with the effort of our instructors and staff who start to look into booking rooms months in advance… again summertime is not that “quiet and relaxing time” people may think of at the university premises…

Taking attendance… We issue certificates for those participants that attend at least three days. This requires that we record the attendance of the participants for every session every day. In the initial summer schools, where there were one or two parallel sessions at most, and the total number of participants wasn’t too large, we used a paper signing list, where students self-reported their attendance. By the end of the week we would collect and count these lists and manually awarded certificates.

But with 3 parallel streams and more than two hundred participants, the task of manually sorting out attendance has become unfeasible. To tackle this issue, we developed a system using our own education website, where we ask the participants to take a “test” selecting from 10 randomly generated codes the one that is given in the session they are attending.
In this way, the participation of each student is recorded and tied to the specific session associated with the selected code. The same site handles registration and dispenses the students’ access to temporary accounts on computing resources they will use during the week, and contains the teaching materials.

Certificates… Having recorded the attendance from the participants, this is just the beginning of the process of issuing the certificates. After this, we have scripts that can identify the participants that would be awarded a certificate of participating according to the criteria stated before, and generate a PDF document stating that. Years ago, we use to run through the university campus on the last day to print hard-copies of these, but since last year we send the participants an electronic version of it. The number of certificates demonstrates the growth in attendance over the years: In 2014 we awarded 30 attendees with summer school certificates, in 2019, this number has grown to 159.

Financial support… One remarkable thing about the school is that we are able to continue offering this high-quality and relevant training free of cost to the participants. This is not a easy task to achieve, as there are several costs associated to the event. The cost of the instructors is absorbed by the partnering organization (SciNet, SHARCNET and CAMH), while logistic costs for the rooms and AV utilizations are covered by SciNet, while coffee breaks that are provided to the participants were sponsored by Compute Ontario.

Other centres have decided to charge their participants a modest registration fee for their summer school, which allows them to tackle two things: one is to alleviate the cost associated with the event itself; and secondly, to reduce the number of no-shows during the school. Fortunately our attendance numbers have been rising steadily every year, but our turn-out rate seems to be steady and predictable at 70%, making the no-show effect non-issue.

More summer activities…

SciNet also participates in the International HPC Summer School, sending a few instructors and 10 students to this competitive one-week program every year.

Last but not least, SciNet finished this year’s summer season co-organizing and hosting a “virtual” remotely hosted one week-long PetaScale Computing Institute at the end of August.

Although physically and intellectually exhausted, we finished one of the busiest summer seasons ever in SciNet’s training and education history, allowing us to keep pushing ourselves and re-charge of our energies for the beginning of the academic year.

Further details and information about SciNet’s education and teaching endeavours can be found in the following link:

SciNet’s publication about Niagara deployment

August 3, 2019 in blog, blog-general, blog-technical, for_press, for_researchers, for_users, frontpage, news, Road_to_Niagara

Have you ever wondered how a supercomputer is designed and brought to life?
Read SciNet’s latest paper on the deployment of Canada’s fastest supercomputer: Niagara.

Niagara is currently the fastest supercomputer accessible to academics in Canada.
In this paper we describe the transition process from our previous systems, the TCS and GPC, the procurement and deployment processes, as well as the unique features that make Niagara a one-of-a-kind machine in Canada.

Please cite this paper when using Niagara to run your computations, simulations or analysis:
“Deploying a Top-100 Supercomputer for Large Parallel Workloads: the Niagara Supercomputer”, Ponce et al, “Proceedings of PEARC’19: Practice and Experience in Advanced Research Computing on Rise of the Machines (Learning)”, 34 (2019).

Learn more about SciNet’s research and publications by visiting the following link.

2019 Compute Ontario Summer School Central

May 14, 2019 in blog, for_educators, for_press, for_researchers, for_users, frontpage, news

The Compute Ontario Summer School on Scientific and High Performance Computing is an annual educational event for graduate/undergraduate students, postdocs and researchers who are engaged in a compute intensive research. Held geographically in the west, centre and east of the province of Ontario, the summer school provides attendees with the opportunity to learn and share knowledge and experience in high performance and technical computing on modern HPC platforms.

Each site will have a slightly different list of courses. The summer school will include both in-class lectures and hands-on labs (done on the participants’ laptops). Those who attend at least three full days cumulatively will receive an official certificate in HPC training (i.e., a total of 6 full morning and afternoon sessions).

Instructors for this school have been provided by SciNet, CAMH and SHARCNET. Break refreshments are provided courtesy of Compute Ontario.

Registration for the central installment in Toronto from June 24-28, 2019 is now open!

The registration is free and is aimed at Compute Canada users as well as students, post-docs and other researchers from academic institutions. You do not need to have a SciNet or Compute Canada account (although you can use that). Please be advised that seats are limited and tend to fill up.

Computational Science Education Publications by SciNet

January 16, 2019 in blog, blog-general, for_educators, for_press, for_researchers, frontpage, news, Uncategorized

This year starts with the publication of three papers by SciNet analysts in the Journal of Computational Science Education that are a reflection of SciNet’s ten years of training and educating academic researchers in the practical and scalable use of high performance computing.

  1. Bridging the Educational Gap between Emerging and Established Scientific Computing Disciplines
    Marcelo Ponce, Erik Spence, Ramses van Zon, and Daniel Gruner, Journal of Computational Science Education Vol 10 (1) 4-11 (2019)

    In this paper, we describe our experience in developing curriculum courses aimed at graduate students in emerging computational fields, including biology and medical science. We focus primarily on computational data analysis and statistical analysis, while at the same time teaching students best practices in coding and software development. Our approach combines a theoretical background and practical applications of concepts. The outcomes and feedback we have obtained so far have revealed several issues: students in these particular areas lack instruction like this although they would tremendously benefit from it; we have detected several weaknesses in the formation of students, in particular in the statistical foundations but also in analytical thinking skills. We present here the tools, techniques and methodology we employ while teaching and developing this type of courses. We also show several outcomes from this initiative, including potential pathways for fruitful multi- disciplinary collaboration.

  2. Scientific Computing, High-Performance Computing and Data Science in Higher Education
    Marcelo Ponce, Erik Spence, Ramses van Zon, and Daniel Gruner, Journal of Computational Science Education Vol 10 (1), 24-31 (2019)

    We present an overview of current academic curricula for Scientific Computing, High-Performance Computing and Data Science. After a survey of current academic and non-academic programs across the globe, we focus on Canadian programs and specifically on the education program of the SciNet HPC Consortium, using its detailed enrollment and course statistics for the past six to seven years. Not only do these data display a steady and rapid increase in the demand for research-computing instruction, they also show a clear shift from traditional (high performance) computing to data- oriented methods. It is argued that this growing demand warrants specialized research computing degrees.

  3. Trends in Scientific Computing Training Delivered by a High-Performance Computing Center
    Ramses van Zon, Marcelo Ponce, Erik Spence, and Daniel Gruner, Journal of Computational Science Education Vol 10 (1), 53-60 (2019)

    We analyze the changes in the training and educational efforts of the SciNet HPC Consortium, a Canadian academic High Performance Computing center, in the areas of Scientific Computing and High-Performance Computing, over the last six years. Initially, SciNet offered isolated training events on how to use HPC systems and write parallel code, but the training program now consists of a broad range of workshops and courses that users can take toward certificates in scientific computing, data science, or high-performance computing. Using data on enrollment, attendence, and certificate numbers from SciNet’s education website, used by almost 1800 users so far, we extract trends on the growth, demand, and breadth of SciNet’s training program. Among the results are a steady overall growth, a sharp and steady increase in the demand for data science training, and a wider participation of ‘non-traditional’ computing disciplines, which has motivated an increasingly broad spectrum of training offerings. Of interest is also that many of the training initiatives have evolved into courses that can be taken as part of the graduate curriculum at the University of Toronto.

We also recently gave a webinar on these topics.

Other publications by SciNet people can be found at here.

2019 International Summer School on HPC Challenges in Computational Sciences, Japan, July 7-12

December 7, 2018 in blog, for_educators, for_press, for_researchers, for_users, frontpage, news

Graduate students and postdoctoral scholars from institutions in Canada, Europe, Japan and the United States are invited to apply for the tenth International Summer School on HPC Challenges in Computational Sciences, to be held from July 7 to 12, 2019, at the campus of the RIKEN Center for Computational Science (R-CCS) in Kobe, Japan.

Applications are due Feb 4, 2019. The summer school is organized by the Extreme Science and Engineering Discovery Environment (XSEDE), the Partnership for Advanced Computing in Europe (PRACE), the R-CCS, and the SciNet HPC Consortium.

Leading computational scientists and HPC technologists from the U.S., Japan, Europe and Canada will offer instructions on a variety of topics as well as dedicated mentoring. Topics include:

  • HPC challenges by discipline
  • HPC programming proficiency
  • Performance analysis & profiling
  • Scientific visualization
  • Big Data Analytics
  • Machine Learning
  • Canadian, EU, Japanese and U.S. HPC-infrastructures

The expense-paid program will benefit scholars from Canadian, European, Japanese and U.S. institutions who use advanced computing in their research. The ideal candidate will have many of the following qualities, however this list is not meant to be a “checklist” for applicants to meet all criteria:

  • Familiar with HPC, not necessarily an HPC expert, but rather a scholar who could benefit from including advanced computing tools and methods into their existing computational work
  • A graduate student with a strong research plan or a postdoctoral fellow in the early stages of their research careers
  • Regular practice with parallel programming (i.e., student utilizes parallel programming generally on a monthly basis or more)
  • May have a science or engineering background, however, applicants from other disciplines are welcome, provided their research activities include computational work.

Students from underrepresented groups in computing are highly encouraged to apply (i.e., women, racial/ethnic minorities, persons with disabilities, etc.). If you have any questions regarding your eligibility or how this program may benefit you or your research group, please do not hesitate to contact the individual associated with your region below.

Interested students should apply by February 4, 2019. School fees, meals and housing will be covered for all accepted applicants, as well as intercontinental flight costs.
Further information and application: http://ss19.ihpcss.org.


SciNet HPC Consortium
Ramses van Zon
SciNet, Univ. of Toronto, Canada
Email: rzon at scinet.utoronto.ca

Hermann Lederer
Max Planck Computing and Data Facility, Germany
Email: lederer at mpcdf.mpg.de

Simon Wong
ICHEC, Ireland
Email: simon.wong at ichec.ie

Toshiyuki Imamura
Email: Imamura.toshiyuki at riken.jp

Jay Alameda
NCSA, University of Illinois at Urbana-Champaign, United States
Email: alameda at illinois.edu

SciNet Receives HPCwire Award

December 6, 2018 in blog, for_press, for_researchers, for_users, frontpage, in_the_news, news, Road_to_Niagara, success_story

We are very proud that SciNet has received the 2018 HPCwire Editor’s Award for Best Use of HPC in Physical Sciences. The award was announced at the 2018 International Conference for High Performance Computing, Networking, Storage and Analysis (SC18), in Dallas, Texas.

SciNet used Lenovo and Mellanox technologies on the new Niagara cluster to create spatial resolution models of the Pacific Ocean, helping to validate ocean waves movement and to assist in global warming calculations. These calculations were performed by a team of scientists involving University of Toronto’s Prof. W. Richard Peltier, University of Michigan oceanographer Prof. Brian Arbic, and NASA JPL’s Dr. Dimitris Menemenlis. More on this calculation can be found here.

This calculation was part of the “early science” program of the Niagara supercomputer at the SciNet HPC Consortion. In this short period in March of 2018, a number of scientists were given the opportunity to perform “heroic” calculations. These large scale calculation were essential to test, to tune and to get Niagara ready for use as a Canada’s fastest national academic supercomputer.

SciNet concludes its biggest-ever training week in scientific, high performance, and data science computing

June 19, 2018 in blog-general, for_educators, for_press, for_researchers, for_users, frontpage, news, success_story

When summer arrives, it means the return of Ontario’s Summer School on Scientific and High Performance Computing. This annual set of educational events bring together graduate students, undergraduate students, postdocs and researchers and gives them an opportunity to learn and share knowledge and experience in technical computing, data science and biomedical computing on modern high-performance computing platforms.

The summer school has been given at three locations for the last seven years and is a collaborative effort of the three High-Performance Consortia in Ontario: SHARCNET, CAC, and SciNet. This year, the first was from May 28 to June 1 in London, while the second one was hosted by SciNet on the St. George campus of the University of Toronto from June 11 to 15, and the third will take place in Kingston from July 30 to August 3.

HPC Python session at the 2018 summer school

High Performance Computing with Python (“Parallel Python”) Session at the 2018 Ontario HPC Summer School in Toronto

The format of the school is that of a five-day intensive workshop with mixed lectures and hands-on sessions on a number of selected topics. The topics slightly vary by location, but may include shared memory programming, distributed memory programming and general purpose graphics processing unit programming, and Python, R, machine learning, neural networks, visualization, debugging, bioinformatics and bioimaging. To make all these topics fit in one week, the lectures are organized in different streams. In Toronto, there was a high performance computing stream, a data science stream, and a biomedical stream, given by instructors from SciNet, SHARCNET, and CAMH.

group photo HPC Summer School in Toronto

Group Photo of the 2018 Ontario HPC Summer School in Toronto

This year was SciNet’s biggest summer school yet. For the first time, there were more than two hundred participants, from widely varying academic background like Business Administration, Dentistry, Biology, Medicine, Economics, Health Sciences, Phyics, Chemistry, Engineering, and Astronomy. Those who attended at least three days received a certificate of attendance.

To get a sense of the tremendous growth of this event, consider that in 2012, one of the first years of the summer school, in Toronto, we had 35 participants in the summer school, issued 20 certificates, and a total of 400 contact-hours were delivered by a handful of instructors. In 2018, these numbers are 7 to 10 times larger: there were 211 participants, 135 certificates were issues, and 3800 contact-hours were delivered by 17 instructors. If that is not a clear enough indication of the demand for this kind of training, consider this: within one day of opening the registration, there were over 100 registrations, and just one week later, the 200 registration mark was crossed. There were people on the waiting list, although we made an effort to try and accommodate most people in the end.

The school is offered for free, but without support for travel, lodging or meals. It is therefore not surprising that most participants are from the Toronto area, but there was a sizeable number of attendants from outside Toronto (60), from outside of Ontario (15) and even from outside Canada (5).

The participants seem to really appreciate the event, as borne out by the preliminary results from the post-event survey. From the survey, we got suggestions for improvements for the future, such as (even) more hands-on and better explanation of prerequisites and software for the sessions. Nonetheless, 94% of the respondents was very satisfied with the summer school, and thought the overall value of the summer school was high. 92% would recommend the event to their peers. Interestingly, the answers to the question “What was the most important thing you learned or did during the summer school?” varied a lot, indicating, perhaps, that the broad range of offered topics allowed participants to pick what suits them best.

Organizing the school was a tremendous experience, made possible by 8 instructors and staff members from SciNet, 9 from CAMH, and 2 from SHARCNET. The University of Toronto provided the space for the session, while Compute Ontario sponsored the coffee-and-cookie breaks.

If you missed it this time, look for the announcement next spring of the 2019 Ontario Summer School on High Performance Computing.