SciNet News September 2018

September 14, 2018 in for_researchers, for_users, newsletter

SUMMARY

  • Niagara takes the number 53 spot in the June 2018 TOP500 list of supercomputers (https://www.top500.org/list/2018/06).
  • Scratch purging policy on Niagara is in effect.
  • Burst buffer available on demand.
  • Various SciNet courses and events to start next week, including a “Intro to Niagara/SciNet” session and a TechTalk on “Machine Learning Cosmic Structure Formation” on Sept 12.
  • SciNet’s Jupyterhub with access to files on Niagara is online.
  • my.SciNet website with access to your Niagara jobs records is online.
  • Courses website now accessible with your Compute Canada password.

SYSTEM NEWS

EVENTS COMING UP

Registration for SciNet courses is done by logging into https://courses.scinet.utoronto.ca with your Compute Canada username
and password.

Many of the events are at the teaching room or boardroom in the SciNet offices on the eleventh floor of the MaRS West Tower, suite 1140A (661 University Avenue, Toronto ON M5G 1M1). SciNet events are often recorded and broadcast (see the courses site for links).

  • INTRO TO SCINET AND NIAGARA
    Wednesday Sept 12, 2018, 10:00 am – 11:30 am
    SciNet Boardroom (suite 1140, 661 University Avenue, Toronto).

    This is a class of approximately 90 minutes to introduce SciNet and the new supercomputer Niagara and teach you how to use Niagara.

    Participation counts towards the SciNet HPC Certificate.

    For more information and (free) registration, go to https://courses.scinet.utoronto.ca/404

  • SCINET USER GROUP MEETING
    Wednesday Sept 12, 2018, 12:00 noon – 1:00 pm
    SciNet Boardroom (suite 1140, 661 University Avenue, Toronto).

    The SciNet Users Group (SNUG) meetings are every month on the second Wednesday (except during the summer), and involve pizza, user discussion, feedback, and a half-hour talk on topics or technologies of interest to the SciNet community.

    The TechTalk will be on

    MACHINE LEARNING COSMIC STRUCTURE FORMATION

    by George Stein (Dept. of Astronomy-UofT, CITA).

    Abstract: In modern astrophysics and cosmology, accurate simulations of the large scale structure of the universe are necessary. Usually, this is accomplished by so called N-body simulations, which calculate the full gravitational collapse of a region of the universe over its 14 billion year history. Instead of calculating this costly gravitational evolution, we trained a three-dimensional deep Convolutional Neural Network (CNN) to identify dark matter proto-haloes directly from the cosmological initial conditions, and showed that a CNN of this type can be a viable alternative in some cases. In this talk I will discuss current cosmological simulations and the invasion of machine learning techniques, with a focus on our work. For more information see https://arxiv.org/abs/1805.04537.

    For sign up and more information, see https://courses.scinet.utoronto.ca/410

  • INTRODUCTION TO COMPUTATIONAL BIOSTATISTICS WITH R (MSC1090)
    Tuesdays and Thursday, 11 am – 12 noon
    Twelve weeks starting Sept 12.

    In this course data analysis techniques utilizing the R statistical language, will be discussed and introduced, as well as, the basics of programming and scientific computing. The goal of this course is to prepare graduate students to perform scientific data analysis. Successful students will learn how to use statistical inference tools to gain insight into large and small data sets, as well as be exposed to cutting-edge techniques and best practises to store, manage and analyze (large) data. Topics include: R programming, version control, automation, modular programming and scientific visualization.

    Students willing to take the course as part of their graduate program have to enroll through Acorn/ROSI. This course is part of the IMS graduate program and to be taught at the UofT St. George campus (i.e., not in the SciNet classroom). Contact us if you wish to audit the course without credit.

    This course will be recorded, but not broadcast.

    For more information, see https://courses.scinet.utoronto.ca/399

  • INTRODUCTION TO THE LINUX SHELL
    Wednesday Sept 19, 1:00 pm – 4:00 pm
    Teaching Room 1140A (MaRS West Tower, 661 University Ave, Toronto)

    Working with many of the HPC systems in Ontario involves using the Linux/UNIX command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. In this half-day session, you can become initiated with this course. This hands on session will cover basic commands and scripting. It could be a great boon for your productivity!

    Participation counts towards the SciNet Scientific Computing Certificate.

    For sign up and more information, see https://courses.scinet.utoronto.ca/407

  • INTRODUCTION TO PROGRAMMING
    Tuesdays and Thursdays, 1:00 pm – 2:00 pm
    Four weeks starting Oct 2.
    Teaching Room 1140A (MaRS West Tower, 661 University Ave, Toronto)

    New to programming? Learn the basics of programming using python in eight one-hour sessions over the course of four weeks. Sessions will consist of a mix of lectures and hands-on exercises.

    Participation counts towards the SciNet Scientific Computing Certificate.

    For sign up and more information, see https://courses.scinet.utoronto.ca/401

  • INTRO TO SCINET AND NIAGARA
    Wednesday Oct 10, 2018, 10:00 am – 11:30 am
    SciNet Boardroom (suite 1140, 661 University Avenue, Toronto).

    This is a class of approximately 90 minutes to introduce SciNet and the new supercomputer Niagara and teach you how to use Niagara.

    Participation counts towards the SciNet HPC Certificate.

    For more information and (free) registration, go to https://courses.scinet.utoronto.ca/405

  • SCINET USER GROUP MEETING
    Wednesday Oct 10, 2018, 12:00 noon – 1:00 pm
    SciNet Boardroom (suite 1140, 661 University Avenue, Toronto).

    The SciNet Users Group (SNUG) meetings are every month on the second Wednesday (except during the summer), and involve pizza, user
    discussion, feedback, and a half-hour talk (TBA) on topics or technologies of interest to the SciNet community. We’ll likely
    discuss the upcoming Resource Allocation Competition.

    For sign up and more information, see https://courses.scinet.utoronto.ca/411

  • ADVANCED SHELL PROGRAMMING
    Wednesday Oct 17, 2018, 1:00 pm – 4:00 pm
    Teaching Room 1140A (MaRS West Tower, 661 University Ave, Toronto)

    Learn how to write bash script, use environment variables, how to control process, and much more. Requires some linux basic command line experience.

    Participation counts towards the SciNet Scientific Computing Certificate.

    For sign up and more information, see https://courses.scinet.utoronto.ca/409

  • NUMERICAL COMPUTING WITH PYTHON
    Tuesdays and Thursdays, 1:00 pm – 2:00 pm
    Four weeks, starting Nov 6, 2017 (skipping the week of Nov 12-16)
    Teaching Room 1140A (MaRS West Tower, 661 University Ave, Toronto)

    Learn about research computing even with little programming experience. Covers programming in python, best practices and
    visualization. Some experience with python is required. Four home work sets will be the basic of the evaluation.

    Participation counts towards the SciNet Scientific Computing Certificate.

    For sign up and more information, see https://courses.scinet.utoronto.ca/402

  • INTRO TO SCINET AND NIAGARA
    Wednesday Nov 14, 2018, 10:00 am – 11:30 am
    SciNet Boardroom (suite 1140, 661 University Avenue, Toronto)

    This is a class of approximately 90 minutes to introduce SciNet and the new supercomputer Niagara and teach you how to use Niagara.

    Participation counts towards the SciNet HPC Certificate.

    For more information and (free) registration, go to https://courses.scinet.utoronto.ca/406

  • INTRODUCTION TO THE LINUX SHELL
    Wednesday Nov 21, 1:00 pm – 4:00 pm
    Teaching Room 1140A (MaRS West Tower, 661 University Ave, Toronto)

    Working with many of the HPC systems in Ontario involves using the Linux/UNIX command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. In this half-day session, you can become initiated with this course. This hands on session will cover basic commands and scripting. It could be a great boon for your productivity!

    Participation counts towards the SciNet Scientific Computing Certificate.

    For sign up and more information, see https://courses.scinet.utoronto.ca/407

  • SCIENTIFIC COMPUTING FOR PHYSICISTS (PHY1610)
    Winter 2019
    Teaching Room 1140A (MaRS West Tower, 661 University Ave, Toronto)

    This course is aimed at reducing your struggle in getting started with computational projects, and make you a more efficient computational scientist. Topics include well-established best practices for developing software as it applies to scientific computations, common numerical techniques and packages, and aspects of high performance computing. While we will introduce the C++ language, in one language or another, students should already have some programming experience. Despite the title, this course is
    suitable for many physical scientists (chemists, astronomers, …).

    This course is part of the physics graduate program. Students willing to take the course as part of their graduate program have to enroll through Acorn/ROSI.

    For more information, see https://courses.scinet.utoronto.ca/398

  • QUANTITATIVE APPLICATIONS FOR DATA ANALYSIS
    Winter 2019
    University of Toronto Scarborough Campus

    In this course data analysis techniques utilizing Python and R statistical language, will be discussed and introduced, as well as the basics of programming and scientific computing. The goal of this course is to prepare graduate students to perform scientific data analysis. Successful students will learn how to use statistical inference tools to gain insight into large and small data sets, as well as be exposed to cutting-edge techniques and best practises to store, manage and analyze (large) data.

    Topics include: Python and R programming, version control, automation, modular programming and scientific visualization.

    Students willing to take the course as part of their graduate program have to enroll through Acorn/ROSI. This course is part of the EES graduate program and to be taught at the UTSc campus.

    For more information, see https://courses.scinet.utoronto.ca/403

As always, further details can be found below on the SciNet courses siteand the SciNet wiki.

SciNet concludes its biggest-ever training week in scientific, high performance, and data science computing

June 19, 2018 in blog-general, for_educators, for_press, for_researchers, for_users, frontpage, news, success_story

When summer arrives, it means the return of Ontario’s Summer School on Scientific and High Performance Computing. This annual set of educational events bring together graduate students, undergraduate students, postdocs and researchers and gives them an opportunity to learn and share knowledge and experience in technical computing, data science and biomedical computing on modern high-performance computing platforms.

The summer school has been given at three locations for the last seven years and is a collaborative effort of the three High-Performance Consortia in Ontario: SHARCNET, CAC, and SciNet. This year, the first was from May 28 to June 1 in London, while the second one was hosted by SciNet on the St. George campus of the University of Toronto from June 11 to 15, and the third will take place in Kingston from July 30 to August 3.

HPC Python session at the 2018 summer school

High Performance Computing with Python (“Parallel Python”) Session at the 2018 Ontario HPC Summer School in Toronto

The format of the school is that of a five-day intensive workshop with mixed lectures and hands-on sessions on a number of selected topics. The topics slightly vary by location, but may include shared memory programming, distributed memory programming and general purpose graphics processing unit programming, and Python, R, machine learning, neural networks, visualization, debugging, bioinformatics and bioimaging. To make all these topics fit in one week, the lectures are organized in different streams. In Toronto, there was a high performance computing stream, a data science stream, and a biomedical stream, given by instructors from SciNet, SHARCNET, and CAMH.

group photo HPC Summer School in Toronto

Group Photo of the 2018 Ontario HPC Summer School in Toronto

This year was SciNet’s biggest summer school yet. For the first time, there were more than two hundred participants, from widely varying academic background like Business Administration, Dentistry, Biology, Medicine, Economics, Health Sciences, Phyics, Chemistry, Engineering, and Astronomy. Those who attended at least three days received a certificate of attendance.

To get a sense of the tremendous growth of this event, consider that in 2012, one of the first years of the summer school, in Toronto, we had 35 participants in the summer school, issued 20 certificates, and a total of 400 contact-hours were delivered by a handful of instructors. In 2018, these numbers are 7 to 10 times larger: there were 211 participants, 135 certificates were issues, and 3800 contact-hours were delivered by 17 instructors. If that is not a clear enough indication of the demand for this kind of training, consider this: within one day of opening the registration, there were over 100 registrations, and just one week later, the 200 registration mark was crossed. There were people on the waiting list, although we made an effort to try and accommodate most people in the end.

The school is offered for free, but without support for travel, lodging or meals. It is therefore not surprising that most participants are from the Toronto area, but there was a sizeable number of attendants from outside Toronto (60), from outside of Ontario (15) and even from outside Canada (5).

The participants seem to really appreciate the event, as borne out by the preliminary results from the post-event survey. From the survey, we got suggestions for improvements for the future, such as (even) more hands-on and better explanation of prerequisites and software for the sessions. Nonetheless, 94% of the respondents was very satisfied with the summer school, and thought the overall value of the summer school was high. 92% would recommend the event to their peers. Interestingly, the answers to the question “What was the most important thing you learned or did during the summer school?” varied a lot, indicating, perhaps, that the broad range of offered topics allowed participants to pick what suits them best.

Organizing the school was a tremendous experience, made possible by 8 instructors and staff members from SciNet, 9 from CAMH, and 2 from SHARCNET. The University of Toronto provided the space for the session, while Compute Ontario sponsored the coffee-and-cookie breaks.

If you missed it this time, look for the announcement next spring of the 2019 Ontario Summer School on High Performance Computing.

SciNet News June 2018

June 4, 2018 in for_users, newsletter

SUMMARY

  • Scratch purging policy on Niagara starts July 16.
  • Niagara takes it place at the number 53 spot in the top500.
  • Preliminary fall training and education schedule

Details can be found below and are also available on the SciNet education website courses.scinet.utoronto.ca and the SciNet wiki docs.scinet.utoronto.ca.

SYSTEM NEWS

  • Scratch purging policy on Niagara starts July 16.
  • Niagara takes it place at the number 53 spot in the top500.
  • Burst Buffer, a fast storage tier between the general parallel file system and system memory, is sto;; available for groups with high I/O needs, upon request. See https://docs.scinet.utoronto.ca/index.php/Burst_Buffer

EVENTS COMING UP

Unless stated otherwise, all events listed below take place at the SciNet Teaching Room at our offices on the eleventh floor of the MaRS West Tower, suite 1140A (661 University Avenue, Toronto ON M5G 1M1).

Most events will be recorded and some are broadcast, but only some of the courses can be taken remotely for SciNet certificate credits, as indicated below.

Registration for SciNet courses is done by logging into https://support.scinet.utoronto.ca/education with your SciNet account.

More SciNet courses will be announced at the end of the summer.

  • INTRO TO SCINET/NIAGARA Wednesday Sept 23, 2018, 10:00 am – 11:30 am SciNet Boardroom (suite 1140, 661 University Avenue, Toronto ON M5G 1M1).

    This is a class of approximately 60-90 minutes to introduce SciNet and the new supercomputer Niagara and teach you how to use Niagara.

    Participation counts towards the SciNet HPC Certificate.

    For more information and (free) registration, go to https://support.scinet.utoronto.ca/education/go.php/396/index.php

  • COMPUTE ONTARIO SUMMER SCHOOL EAST June 11 – 15, University of Toronto, St. George Campus New College, University of Toronto, 40 Willcocks St., Toronto, M5S 1C6

    The Compute Ontario Summer School on Scientific and High Performance Computing is an annual educational event for graduate/undergraduate students, postdocs and researchers to learn and share knowledge and experience in high performance and technical computing on modern HPC platforms.

    As in previous years, the Summer School on High Performance Computing 2014 will have three installments:

    “West” May 28 – June 1 Western University, London “Central” June 11 – June 15 University of Toronto, Toronto “East” July 30 – August 3 Queen’s University, Kingston

    Registration for the school in London and Kingston is at https://www.sharcnet.ca/summerschool/2018, while registration for the summer school in Toronto is at https://support.scinet.utoronto.ca/education/go.php/368/index.php

    The Toronto summer school hosted by SciNet will have the following three streams: Stream 1: High Performance computing; Stream 2: Data Science; Stream 3: Biomedical. Instructors are from SciNet, SHARCNET, and CAMH.

    This event will not be held at the SciNet Offices, rather it will be at New College on the Downtown St. George Campus of the University of Toronto.

    Parts of this event count towards the SciNet Certificates.

    More details and registration can be found at https://support.scinet.utoronto.ca/education/go.php/368/index.php

  • ADVANCED PARALLEL SCIENTIFIC COMPUTING Mondays and Wednesdays, 1 pm – 2pm Four weeks starting Sept 11. Teaching Room 1140A (MaRS West Tower, 661 University Ave, Toronto)

    In this course, we will be exploring advanced examples of parallel computing in scientific research. Topics include HPC tools, CUDA, molecular dynamics, Monte Carlo and random number generation, smoothed particle hydrodynamics, N-body simulations and computational fluid dynamics.

    The format of the course will be two lectures of one hour, for four consecutive weeks. The participants are expected to choose a project that involves analyzing and improving a parallel research code from one of the topic presented in the lectures, and to present their findings two weeks after the end of the course.

    The lectures of this course will be broadcast and recorded. Remote participation for credit may be possible upon request.

    Familiarity with parallel programming (MPI/OpenMP/CUDA) in a compiled language (C/C++/Fortran) is a prerequisite of this course.

    This class counts as 12 credit-hours towards the SciNet HPC Certificate. It can also be taken as a modular course by Physics grad students and as a mini course by Astrophysics students.

    For sign up and more information, see https://support.scinet.utoronto.ca/education/go.php/317/index.php

  • INTRODUCTION TO CLINICAL BIOSTATISTICS (MSC1090) Tuesdays and Thursday, 11 am – 12 noon Twelve weeks starting Sept 12.

    In this course data analysis techniques utilizing the R statistical language, will be discussed and introduced, as well as, the basics of programming and scientific computing. The goal of this course is to prepare graduate students to perform scientific data analysis. Successful students will learn how to use statistical inference tools to gain insight into large and small data sets, as well as be exposed to cutting-edge techniques and best practises to store, manage and analyze (large) data. Topics include: R programming, version control, automation, modular programming and scientific visualization.

    Students willing to take the course as part of their graduate program have to enroll through Acorn/ROSI. This course is part of the IMS graduate program and to be taught at the UofT St. George campus (i.e., not in the SciNet classroom).

    Unfortunately, this course is full, but we are exploring the possibility of giving it again in the Winter term.

    For sign up and more information, see https://support.scinet.utoronto.ca/education/go.php/324/index.php

  • INTRO TO SCINET Sept 13, 10:00 am – 11:30 am

    In about 90 minutes, you will learn how to use the SciNet systems. Experienced users may still pick up some valuable pointers.

    Participation counts towards the SciNet HPC Certificate.

    For sign up and more information, see https://support.scinet.utoronto.ca/education/go.php/329/index.php

  • SCINET USER GROUP MEETING Sept 13, 12:00 noon – 1:00 pm Boardroom (MaRS West Tower, 661 University Ave, Toronto, suite 1140)

    Pizza, user discussion, and a tech talk: “ChIP-Seq analysis of the Interactive Bromodomain 1 protein (Ibd1) in Tetrahymena thermophila”, by Alejandro Saettone (Ryerson University)

    For sign up and more information, see https://support.scinet.utoronto.ca/education/go.php/301/index.php

  • INTRODUCTION TO NEURAL NETWORK PROGRAMMING Sept 25, 10:00 am – 3:00 pm Teaching Room 1140A (MaRS West Tower, 661 University Ave, Toronto)

    This workshop will introduce neural network programming concepts, theory, and techniques. The level of the material will be introductory, intended for those with no experience with neural networks. The programming language will be Python 2.7; experience with Python programming will be assumed. The Keras neural network framework, with a Theano back end, will be used for more-advanced programming; no experience with Keras or Theano will be expected. Students should come with the following Python packages installed on their laptops: numpy, matplotlib, scikit-learn, theano (version 0.9.0 or greater), keras.

    Participation counts towards the SciNet Data Science Certificate.

    You can also view this event’s broadcast, but this would not count towards the certificate.

    For sign up and more information, see https://support.scinet.utoronto.ca/education/go.php/338/index.php

  • INTRODUCTION TO THE LINUX SHELL Sept 27, 10:00 am – 12:00 noon Teaching Room 1140A (MaRS West Tower, 661 University Ave, Toronto)

    Learn the basics of how to use the unix shell in two hours. Very useful for new users of SciNet that have little or no experience with unix or linux.

    Participation counts towards the SciNet Scientific Computing Certificate.

    For sign up and more information, see https://support.scinet.utoronto.ca/education/go.php/330/index.php

  • ADVANCED NEURAL NETWORKS Mondays from 11:00 am to 12:00 noon Four weeks starting Oct 2. Teaching Room 1140A (MaRS West Tower, 661 University Ave, Toronto)

    This class will review advanced neural network programming theory and architectures. The level of the material will not be introductory, experience with neural networks will be assumed. This class is intended to continue the material covered in “Introduction to Neural Network Programming” (see above).

    Participation counts towards the SciNet Data Science Certificate.

    For sign up and more information, see https://support.scinet.utoronto.ca/education/go.php/325/index.php

  • INTRODUCTION TO PROGRAMMING Tuesdays and Thursdays, 1:00 pm – 2:00 pm Four weeks starting Oct 10. Teaching Room 1140A (MaRS West Tower, 661 University Ave, Toronto)

    New to programming? Learn the basics of programming using python in eight one-hour sessions over the course of four weeks. Sessions will consist of a mix of lectures and hands-on exercises.

    Participation counts towards the SciNet Scientific Computing Certificate.

    For sign up and more information, see https://support.scinet.utoronto.ca/education/go.php/328/index.php

  • ADVANCED SHELL PROGRAMMING Oct 25, 1:00 pm – 4:00 pm Teaching Room 1140A (MaRS West Tower, 661 University Ave, Toronto)

    Learn how to write bash script, use environment variables, how to control process, and much more. Requires some linux basic command line experience.

    Participation counts towards the SciNet Scientific Computing Certificate.

    For sign up and more information, see https://support.scinet.utoronto.ca/education/go.php/331/index.php

  • INTRO TO SCIENTIFIC COMPUTING WITH PYTHON Tuesdays and Thursdays, 1:00 pm – 2:00 pm Four weeks, starting Nov 7, 2017 Teaching Room 1140A (MaRS West Tower, 661 University Ave, Toronto)

    Learn about research computing even with little programming experience. Covers programming in python, best practices and visualization. Some experience with python is required. Four home work sets will be the basic of the evaluation.

    Participation counts towards the SciNet Scientific Computing Certificate.

    The lectures of this course will be broadcast and recorded. Remote participation for credit may be possible upon request.

    For sign up and more information, see https://support.scinet.utoronto.ca/education/go.php/327/index.php

  • SCIENTIFIC COMPUTING FOR PHYSICISTS (PHY1610) Winter 2018

    This course is aimed at reducing your struggle in getting started with computational projects, and make you a more efficient computational scientist. Topics include well-established best practices for developing software as it applies to scientific computations, common numerical techniques and packages, and aspects of high performance computing. While we will introduce the C++ language, in one language or another, students should already have some programming experience. Despite the title, this course is suitable for many physical scientists (chemists, astronomers, …).

    For sign up and more information, see https://support.scinet.utoronto.ca/education/go.php/326/index.php

2018 Compute Ontario Summer School Central

May 14, 2018 in blog, for_educators, for_researchers, for_users, news

The Compute Ontario Summer School on Scientific and High Performance Computing is an annual educational event for graduate/undergraduate students, postdocs and researchers to learn and share knowledge and experience in high performance and technical computing on modern HPC platforms.

Apart from the Compute Ontario Summer School Central, which will be hosted in Toronto by SciNet from June 11 to June 15, 2018, there are two other instances of the 2018 Compute Ontario summer school: One hosted by SHARCNET at Western University (May 28-Jun 1) one hosted by CAC at Queen’s University (Jul 30-Aug 3). Each of the three site has a slightly different list of courses, but all include both in-class lectures and hands-on sessions. Those who attend at least three full days cumulatively will receive an official certificate in HPC training.

The Toronto summer school (“Compute Ontario Summer School Central”), hosted by SciNet, will have the following three streams: Stream 1: High Performance computing; Stream 2: Data Science; Stream 3: Biomedical. Instructors are from SciNet, SHARCNET, and CAMH.

Location

Wilson Hall – New College
University of Toronto
St. George Campus
40 Willcocks St.
Toronto, ON M5S 1C6

Rooms: 524, 1016, 1017, 2006 (check the sessions for the room assignments)

Registration

Step 1: Log into the SciNet education site with your SciNet account, select the Compute Ontario Summer School in “Browse Courses”, and click on the “Register Me” link on the right. OR: if you do not have a SciNet account, register at tinyurl.com/toss2018reg, enter the required information such as login, password and email. In the latter case, you will receive an email with a link to confirm your email.

Step 2: Make your session selection (see program below). You can alter your selection at any time, but note that seats are limited.

Cost

The event is free of charge, though meals and lodging are at the participant’s own expense. We therefore thank the organizations who are providing the instructors (which they do free of
charge). This event is furthermore sponsored by Compute Ontario, the umbrella organization for Academic Advanced Research Computing in Ontario.

Program

HPC Stream Data Science Stream Biomedical Stream
Mon, Jun 11
Morning: 09:00-12:30
Welcome and Introduction to HPC and SciNet Welcome and Introduction to HPC and SciNet Welcome and Introduction to HPC and SciNet
Afternoon: 13:30-16:30
Programming Clusters with Message Passing Interface Introduction to the Linux Shell Python for MRI analysis
Tue, Jun 12
Morning: 09:30-12:30
Programming Clusters with Message Passing Interface Introduction to R Image Analysis at Scale
Afternoon: 13:30-16:30
Programming Clusters with Message Passing Interface Introduction to Python Machine Learning for Neuroimaging
Wed, Jun 13
Morning: 09:30-12:30
Programming GPUs with CUDA Parallel Python PLINK
Afternoon: 13:30-16:30
Programming GPUs with CUDA Machine Learning with Python Next Generation Sequencing
Thu, Jun 14
Morning: 09:30-12:30
Programming GPUs with CUDA Neural Networks with Python RNASeq Analysis
Afternoon: 13:30-16:30
Programming GPUs with CUDA Scientific Visualization Suites R for MRI analysis
Fri, Jun 15
Morning: 09:30-12:30
Shared Memory Parallel Programming with OpenMP Debugging, Profiling and Bring-Your-Own-Code Lab Public Datasets for Neuroimaging
Afternoon: 13:30-16:30
Shared Memory Parallel Programming with OpenMP Debugging, Profiling and Bring-Your-Own-Code Lab HCP with HPC: Surface Based Neuroimaging Analysis

Launch of the Niagara Supercomputer at SciNet

March 5, 2018 in for_educators, for_press, for_researchers, for_users, frontpage, in_the_news, news, Road_to_Niagara

The Niagara supercomputer was officially launched on March 5th, 2018. We were honoured by the presence and remarks of Reza Moridi (Ontario Minister of Research, Innovation and Science), Nizar Ladak (Compute Ontario President and CEO), Dr. Roseann O’Reilly Runte (CFI President and CEO), Prof. Vivek Goel (Vice-president of Research and Innovation at the University of Toronto), and Prof. W. Richard Peltier (Scientific Director of SciNet).

SciNet’s CTO Daniel Gruner gave an overview of the new system:

Niagara is located at University of Toronto and operated by the university’s high-performance computing centre SciNet, but the system is open to all Canadian university researchers.

Niagara is the fastest computer system in the country and is able to run a single job across all 60,000 cores thanks to a high-performance network which interconnects all the nodes. For more information on the configuration, see here.

A time-lapse of the building of Niagara is available (part of SciNet’s YouTube channel):

This system is jointly funded by the Canada Foundation for Innovation, the Government of Ontario, and the University of Toronto.

Road to Niagara 3: Hardware setup

March 5, 2018 in blog-technical, for_press, for_researchers, for_users, news, Road_to_Niagara, Uncategorized

This is the fourth of a series of posts on the transition to SciNet’s new supercomputer called “Niagara”, which will replace the General Purpose Cluster (GPC) and Tightly Coupled Cluster (TCS). The transition to Niagara will take place in the fall of 2017, and the system is planned to be available to users in early 2018.

The University of Toronto has awarded the contract for Niagara to Lenovo, and some of the details of the hardware specifications of the Niagara system have been released:

The system will have the following hardware components:

  • 1,500 nodes.
  • Each node will have 40 Intel Skylake cores (making a total of 60,000 cores) at 2.4 GHz.
  • Each node will have 200 GB (188 GiB)of DDR4 memory.
  • The interconnect between the nodes will be Mellanox EDR Infiniband in a Dragonfly+ topology.
  • A ~9PB usable shared parallel filesystem (GPFS) will be mounted on all nodes.
  • A 256TB Excelero burst buffer (NVMe fabric, up to 160 GB/s) will be available for fast I/O.
  • Peak theoretical speed: 4.61 PetaFLOPS

Niagara is estimated to be installed and operational towards in March 2018, and ready for users not too long after.

Even before official ready-date, there will a period in which select users can try out and port their codes to Niagara.

After the friendly-user period, all current users of the GPC (and former users of the TCS) will get access to Niagara.

The large core count, ample memory per core, and fast interconnect support Niagara’s intended purpose to enable large parallel compute jobs of 512 cores or more.

The software setup will also be tailored to large parallel computations. Nonetheless, there will still be a fair amount of backfill opportunity for smaller jobs.

The setup of Niagara is intended to be similar in spirit to the GPC, but different in form: scheduling per node, a home, scratch and possibly project directory defined in environment variables, a module system, and access to our team of analyst to help you get your codes running, and running well.

International HPC Summer School 2018 in Ostrava, by SciNet, XSEDE, PRACE and Riken

December 22, 2017 in for_educators, for_researchers, for_users, frontpage, in_the_news, news

A High-Performance Computing Summer Institute
July 8-13, Ostrava, Czech Republic
Expenses-paid program
Apply by February 13, 2018
Website: http://ihpcss18.it4i.cz


Graduate students and postdoctoral scholars from institutions in Canada, Europe, Japan and the United States are invited to apply for the ninth International Summer School on HPC Challenges in Computational Sciences, to be held July 8 to 13, 2018, in Ostrava, in the Czech Republic, and hosted by the IT4Innovations National Supercomputing Centre.

Applications are due Feb 13, 2018. The summer school is organized by the Partnership for Advanced Computing in Europe (PRACE), the Extreme Science and Engineering Discovery Environment (XSEDE), the RIKEN Advanced Insti­tute for Computational Science (RIKEN AICS), and the SciNet HPC Consortium.

Leading computational scientists and HPC technologists from the U.S., Europe, Japan and Canada will offer instructions on a variety of topics and also provide advanced mentoring. Topics include:

  • HPC challenges by discipline
  • HPC programming proficiencies
  • Performance analysis & profiling
  • Algorithmic approaches & numerical libraries
  • Data-intensive computing
  • Scientific visualization
  • Canadian, EU, Japanese and U.S. HPC-infrastructures

The expense-paid program will benefit scholars from Canadian, European, Japanese and U.S. institutions who use advanced computing in their research. The ideal candidate will have many of the following qualities, however this list is not meant to be a “checklist” for applicants to meet all criteria:

  • Familiar with HPC, not necessarily an HPC expert, but rather a scholar who could benefit from including advanced computing tools and methods into their existing computational work
  • A graduate student with a strong research plan or a postdoctoral fellow in the early stages of their research efforts
  • Regular practice with parallel programming (i.e., student utilizes parallel programming generally on a monthly basis or more)
  • May have a science or engineering background, however, applicants from other disciplines are welcome provided their research activities include computational work.

Students from underrepresented groups in computing are highly encouraged to apply (i.e., women, racial/ethnic minorities, persons with disabilities, etc.). If you have any questions regarding your eligibility or how this program may benefit you or your research group, please do not hesitate to contact the individual associated with your region below.

Interested students should apply by February 13, 2018. Participation, meals and housing will be covered for the selected participants, also support for intercontinental travel will be given (contingent to funding).

Further information and application, see http://ihpcss18.it4i.cz.

Expired: Job Opportunity at SciNet: HPC Analyst

October 20, 2017 in for_researchers, for_users, HPC Jobs, HPC Jobs Ontario

The SciNet HPC Consortium is looking to augment their team with an HPC Analyst.

Title of job: Scientific Applications Analyst

Location: SciNet HPC Consortium, University of Toronto, Toronto, Ontario, Canada

Summary of job:

The Scientific Applications Analyst provides senior IT services and training in parallel programming, data science applications, and scientific computing workflows for the SciNet High Performance Computing (HPC) consortium which serves researchers at the University of Toronto including faculty, postdoctoral fellows and graduate students in all disciplines and fields (e.g. science and engineering, medicine, finance, languages, etc.). The incumbent is involved in massively-parallel computing (single jobs that make use of up to 40,000 cores) and data analytics and machine learning on large data sets (100TB and up). S/he works with researchers and research teams to plan, develop, install and optimize the initial SciNet systems for various research programs and provides technical consultation to researchers on their system needs for research operations. S/he also takes part in delivering and developing SciNet’s training and education program. SciNet operates large High Performance Computing (HPC) systems, and acquires the largest HPC cluster in Canada by the end of 2017. SciNet provides HPC resources and support to researchers at the University of Toronto, the affiliated research hospitals, and other Canadian universities.

Salary: CAD$94,672 with an annual step progression to a maximum of CAD$121,069. Pay scale and job class assignment is subject to determination pursuant to the Job Evaluation/Pay Equity Maintenance Protocol.

Closing date: Oct 26, 2017, 11:59:00 PM EST

For more details, see the job site of the University of Toronto.

2017 Compute Ontario Summer School Central

June 14, 2017 in blog, blog-general, for_educators, for_press, for_researchers, for_users, frontpage, news, Uncategorized

The Compute Ontario Summer School on Scientific and High Performance Computing is an annual educational event for graduate/undergraduate students, postdocs and researchers who are engaged in a compute intensive research. Held geographically in the west, centre and east of the province of Ontario, the summer school provides attendees with the opportunity to learn and share knowledge and experience in high performance and technical computing on modern HPC platforms.

Each site will have a slightly different list of courses. The summer school will include both in-class lectures and hands-on labs (done on the participants’ laptops). Those who attend at least three full days cumulatively will receive an official certificate in HPC training (i.e., a total of 6 full morning and afternoon sessions).

Instructors for this school have been provided by SciNet, CAMH and SHARCNET. Break refreshments are provided courtesy of Compute Ontario.

REGISTRATION

Registration for the central installment in Toronto from July 24-28, 2017 is now open!

The registration is free and is aimed at Compute Canada users as well as students, post-docs and other researchers from academic institutions. You do not need to have a SciNet account. Please be advised that seats are limited and tend to fill up.

More information and registration can be found on the summer school website.

SCHEDULE

High Performance Computing Stream Data Science Stream Biomedical Stream
Mon, Jul 24
Morning: 09:00-12:00
Welcome and Introduction to HPC and SciNet
Afternoon: 13:30-16:30
Shared Memory Programming with OpenMP Introduction to the Linux Shell PLINK
Tue, Jul 25
Morning: 09:00-12:00
Shared Memory Programming with OpenMP Introduction to R Next Generation Sequencing
Afternoon: 13:30-16:30
Programming Clusters with Message Passing Interface Data Science with Python RNASeq
Wed, Jul 26
Morning: 09:00-12:00
Programming Clusters with Message Passing Interface Parallel R for Data Science Python for MRI analysis
Afternoon: 13:30-16:30
Programming Clusters with Message Passing Interface Python for High Performance Computing (Parallel Python) Image Analysis at Scale
Thu, Jul 27
Morning: 09:00-12:00
Programming GPUs with CUDA Visualization with Python Machine Learning for Neuroimaging
Afternoon: 13:30-16:30
Programming GPUs with CUDA Scientific Visualization Suites R for MRI analysis
Fri, Jul 28
Morning: 09:00-12:00
Programming GPUs with CUDA Debugging, Profiling and Bring-Your-Own-Code Lab Public Datasets for Neuroimaging
Afternoon: 13:30-16:30
Programming GPUs with CUDA Debugging, Profiling and Bring-Your-Own-Code Lab Unit Testing / Neuroinformatics Pipeline Development

LOCATION

This event will be held in the Medical Science Building at the University of Toronto, 1 King Circle, Toronto, Ontario, M5S 1A8, Canada.

The nearest subway station is “Queen’s Park”. Paid parking is available on the St. George Campus.

LODGING

Lodging is not provided by the organization. If you require lodging, you will have to make arrangements yourself. It may be worthwhile checking out the University’s summer residence program at www.studentlife.utoronto.ca/hs/summer .

MEALS

Meals are *not* provided by the organization, but refreshments will be provided during the morning and afternoon breaks, courtesy of Compute Ontario.

COMPUTING FACILITIES

For the hands-on sessions, participants are to bring their own laptop with working wireless and with an ssh client with X-windows installed. The latter is needed to connect to one of SciNet or SHARCNET supercomputers, to which the participant will get access for the duration of the School.

CERTIFICATES

Participants that complete at least three days worth of instruction (i.e., a total of 6 morning and afternoon sessions combined) are to receive a Compute Ontario Summer School Certificate on the last day of the School. Note that this certificate is separate from the SciNet certificates, but parts of the school may count towards a SciNet certificate as well.

Grand Opening of the ArcNet Space at MaRS

May 15, 2017 in blog-general, for_educators, for_press, for_researchers, for_users, frontpage, news, Uncategorized

On May 9, 2017, the Grand Opening of the ArcNet space took place (despite having moved a while ago). What is ArcNet? It is a space where expertise and support of Advanced Research Computing (the “ARC” in ArcNet) from three organizations come together. SciNet is the oldest of the three; It is the supercomputing consortium at the University of Toronto, which has been providing Canadian researchers with computational resources and expertise necessary to perform their research on scales not previously possible in Canada, from the biomedical sciences and aerospace engineering to astrophysics and climate science. SOSCIP is a research and development consortium that pairs academic and industry researchers with advanced computing tools to fuel Canadian innovation. The third organization, Compute Ontario, partners with the four academic computing consortia in Ontario aims to drive advanced computing to accelerate research and enhance competitiveness in the global marketplace resulting in a more prosperous Ontario.

The Grand Opening brought together many of our stakeholders, and was also attended by the Ontario Minister of Research, Innovation and Science, MPP Rezi Moridi, by the President of the University of Toronto Meric Gertler, by the Vice-President of Research at the University of Toronto Prof. Vivek Goel, and by the Scientific Directory of SciNet Prof. W. Richard Peltier. We were honoured that each was willing to say a few words about the opening of this space.

“This facility is a true partnership between the University of Toronto’s SciNet High Performance Computing Consortium, SOSCIP and Compute Ontario. Bringing them together in a state-of-the-art facility will strengthen their partnership and undoubtably create new opportunities to drive innovation through advanced computing in Ontario.” said Prof. Goel, one of the driving forces of the creation of the space.

The President of the University of Toronto reminded us that “With ARCNet, we have created an amazing hub of talent and technology that fosters collaboration between the public and private sector.”

Minister Rezi Moridi remarked that “Today we are here to celebrate the grand opening of the Advanced Research Computing facility. It is great to see that three organizations got together and set up this wonderful facility: SOSCIP,
Compute Ontario, and University of Toronto’s SciNet. I wish you all the best in serving our research community.”


The speeches were given in the new teaching and visualization room. This room holds up to 40 students and is already frequently used for courses and other events. It features a large visualization wall, i.e., a 13 x 7.5 feet ultrahigh resolution screen (8K, to be precise).


Two presentations were given by SciNet analysts to demonstrate the capabilities of this visualization wall; Ramses van Zon showed how to get insight into the complexity of the software installed on SciNet’s main cluster by using graph visualizations, while Marcelo Ponce showed visualizations of several aspects of interacting neutron stars, with data from numerical general relativity simulations.

You can see the capabilities of the visualization wall in the following video: