SciNet News September 2018: Training and Other Events

September 14, 2018 in blog-general, for_educators, for_researchers, for_users, news, newsletter, Uncategorized

Below you will find the salient points of what happened or changed recently, and what will happen in the near future, at SciNet (including with the new Niagara supercomputer). Among these are the announcement of SciNet course offerings in the 2018-2019 academic year. Note that users that take a sufficient number of courses can earn one of the SciNet certificates. In addition, many of our courses can now be taken for credit towards PhD and MSc programs by graduate students from several departments across the University of Toronto, including Institute of Medical Sciences, Ecology and Evolutionary Biology, Chemistry, Astrophysics, and Physics.

As always, further details can be found below on the SciNet courses siteand the SciNet wiki.

SUMMARY

  • Niagara takes the number 53 spot in the June 2018 TOP500 list of supercomputers (https://www.top500.org/list/2018/06).
  • Scratch purging policy on Niagara is in effect.
  • Burst buffer available on demand.
  • Various SciNet courses and events to start next week, including a “Intro to Niagara/SciNet” session and a TechTalk on “Machine Learning Cosmic Structure Formation” on Sept 12.
  • SciNet’s Jupyterhub with access to files on Niagara is online.
  • my.SciNet website with access to your Niagara jobs records is online.
  • Courses website now accessible with your Compute Canada password.

SYSTEM NEWS

EVENTS COMING UP

Registration for SciNet courses is done by logging into https://courses.scinet.utoronto.ca with your Compute Canada username
and password.

Many of the events are at the teaching room or boardroom in the SciNet offices on the eleventh floor of the MaRS West Tower, suite 1140A (661 University Avenue, Toronto ON M5G 1M1). SciNet events are often recorded and broadcast (see the courses site for links).

  • INTRO TO SCINET AND NIAGARA
    Wednesday Sept 12, 2018, 10:00 am – 11:30 am
    SciNet Boardroom (suite 1140, 661 University Avenue, Toronto).

    This is a class of approximately 90 minutes to introduce SciNet and the new supercomputer Niagara and teach you how to use Niagara.

    Participation counts towards the SciNet HPC Certificate.

    For more information and (free) registration, go to https://courses.scinet.utoronto.ca/404

  • SCINET USER GROUP MEETING
    Wednesday Sept 12, 2018, 12:00 noon – 1:00 pm
    SciNet Boardroom (suite 1140, 661 University Avenue, Toronto).

    The SciNet Users Group (SNUG) meetings are every month on the second Wednesday (except during the summer), and involve pizza, user discussion, feedback, and a half-hour talk on topics or technologies of interest to the SciNet community.

    The TechTalk will be on

    MACHINE LEARNING COSMIC STRUCTURE FORMATION

    by George Stein (Dept. of Astronomy-UofT, CITA).

    Abstract: In modern astrophysics and cosmology, accurate simulations of the large scale structure of the universe are necessary. Usually, this is accomplished by so called N-body simulations, which calculate the full gravitational collapse of a region of the universe over its 14 billion year history. Instead of calculating this costly gravitational evolution, we trained a three-dimensional deep Convolutional Neural Network (CNN) to identify dark matter proto-haloes directly from the cosmological initial conditions, and showed that a CNN of this type can be a viable alternative in some cases. In this talk I will discuss current cosmological simulations and the invasion of machine learning techniques, with a focus on our work. For more information see https://arxiv.org/abs/1805.04537.

    For sign up and more information, see https://courses.scinet.utoronto.ca/410

  • INTRODUCTION TO COMPUTATIONAL BIOSTATISTICS WITH R (MSC1090)
    Tuesdays and Thursday, 11 am – 12 noon
    Twelve weeks starting Sept 12.

    In this course data analysis techniques utilizing the R statistical language, will be discussed and introduced, as well as, the basics of programming and scientific computing. The goal of this course is to prepare graduate students to perform scientific data analysis. Successful students will learn how to use statistical inference tools to gain insight into large and small data sets, as well as be exposed to cutting-edge techniques and best practises to store, manage and analyze (large) data. Topics include: R programming, version control, automation, modular programming and scientific visualization.

    Students willing to take the course as part of their graduate program have to enroll through Acorn/ROSI. This course is part of the IMS graduate program and to be taught at the UofT St. George campus (i.e., not in the SciNet classroom). Contact us if you wish to audit the course without credit.

    This course will be recorded, but not broadcast.

    For more information, see https://courses.scinet.utoronto.ca/399

  • INTRODUCTION TO THE LINUX SHELL
    Wednesday Sept 19, 1:00 pm – 4:00 pm
    Teaching Room 1140A (MaRS West Tower, 661 University Ave, Toronto)

    Working with many of the HPC systems in Ontario involves using the Linux/UNIX command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. In this half-day session, you can become initiated with this course. This hands on session will cover basic commands and scripting. It could be a great boon for your productivity!

    Participation counts towards the SciNet Scientific Computing Certificate.

    For sign up and more information, see https://courses.scinet.utoronto.ca/407

  • INTRODUCTION TO PROGRAMMING
    Tuesdays and Thursdays, 1:00 pm – 2:00 pm
    Four weeks starting Oct 2.
    Teaching Room 1140A (MaRS West Tower, 661 University Ave, Toronto)

    New to programming? Learn the basics of programming using python in eight one-hour sessions over the course of four weeks. Sessions will consist of a mix of lectures and hands-on exercises.

    Participation counts towards the SciNet Scientific Computing Certificate.

    For sign up and more information, see https://courses.scinet.utoronto.ca/401

  • INTRO TO SCINET AND NIAGARA
    Wednesday Oct 10, 2018, 10:00 am – 11:30 am
    SciNet Boardroom (suite 1140, 661 University Avenue, Toronto).

    This is a class of approximately 90 minutes to introduce SciNet and the new supercomputer Niagara and teach you how to use Niagara.

    Participation counts towards the SciNet HPC Certificate.

    For more information and (free) registration, go to https://courses.scinet.utoronto.ca/405

  • SCINET USER GROUP MEETING
    Wednesday Oct 10, 2018, 12:00 noon – 1:00 pm
    SciNet Boardroom (suite 1140, 661 University Avenue, Toronto).

    The SciNet Users Group (SNUG) meetings are every month on the second Wednesday (except during the summer), and involve pizza, user
    discussion, feedback, and a half-hour talk (TBA) on topics or technologies of interest to the SciNet community. We’ll likely
    discuss the upcoming Resource Allocation Competition.

    For sign up and more information, see https://courses.scinet.utoronto.ca/411

  • ADVANCED SHELL PROGRAMMING
    Wednesday Oct 17, 2018, 1:00 pm – 4:00 pm
    Teaching Room 1140A (MaRS West Tower, 661 University Ave, Toronto)

    Learn how to write bash script, use environment variables, how to control process, and much more. Requires some linux basic command line experience.

    Participation counts towards the SciNet Scientific Computing Certificate.

    For sign up and more information, see https://courses.scinet.utoronto.ca/409

  • NUMERICAL COMPUTING WITH PYTHON
    Tuesdays and Thursdays, 1:00 pm – 2:00 pm
    Four weeks, starting Nov 6, 2017 (skipping the week of Nov 12-16)
    Teaching Room 1140A (MaRS West Tower, 661 University Ave, Toronto)

    Learn about research computing even with little programming experience. Covers programming in python, best practices and
    visualization. Some experience with python is required. Four home work sets will be the basic of the evaluation.

    Participation counts towards the SciNet Scientific Computing Certificate.

    For sign up and more information, see https://courses.scinet.utoronto.ca/402

  • INTRO TO SCINET AND NIAGARA
    Wednesday Nov 14, 2018, 10:00 am – 11:30 am
    SciNet Boardroom (suite 1140, 661 University Avenue, Toronto)

    This is a class of approximately 90 minutes to introduce SciNet and the new supercomputer Niagara and teach you how to use Niagara.

    Participation counts towards the SciNet HPC Certificate.

    For more information and (free) registration, go to https://courses.scinet.utoronto.ca/406

  • INTRODUCTION TO THE LINUX SHELL
    Wednesday Nov 21, 1:00 pm – 4:00 pm
    Teaching Room 1140A (MaRS West Tower, 661 University Ave, Toronto)

    Working with many of the HPC systems in Ontario involves using the Linux/UNIX command line. This provides a very powerful interface, but it can be quite daunting for the uninitiated. In this half-day session, you can become initiated with this course. This hands on session will cover basic commands and scripting. It could be a great boon for your productivity!

    Participation counts towards the SciNet Scientific Computing Certificate.

    For sign up and more information, see https://courses.scinet.utoronto.ca/407

  • SCIENTIFIC COMPUTING FOR PHYSICISTS (PHY1610)
    Winter 2019
    Teaching Room 1140A (MaRS West Tower, 661 University Ave, Toronto)

    This course is aimed at reducing your struggle in getting started with computational projects, and make you a more efficient computational scientist. Topics include well-established best practices for developing software as it applies to scientific computations, common numerical techniques and packages, and aspects of high performance computing. While we will introduce the C++ language, in one language or another, students should already have some programming experience. Despite the title, this course is
    suitable for many physical scientists (chemists, astronomers, …).

    This course is part of the physics graduate program. Students willing to take the course as part of their graduate program have to enroll through Acorn/ROSI.

    For more information, see https://courses.scinet.utoronto.ca/398

  • QUANTITATIVE APPLICATIONS FOR DATA ANALYSIS
    Winter 2019
    University of Toronto Scarborough Campus

    In this course data analysis techniques utilizing Python and R statistical language, will be discussed and introduced, as well as the basics of programming and scientific computing. The goal of this course is to prepare graduate students to perform scientific data analysis. Successful students will learn how to use statistical inference tools to gain insight into large and small data sets, as well as be exposed to cutting-edge techniques and best practises to store, manage and analyze (large) data.

    Topics include: Python and R programming, version control, automation, modular programming and scientific visualization.

    Students willing to take the course as part of their graduate program have to enroll through Acorn/ROSI. This course is part of the EES graduate program and to be taught at the UTSc campus.

    For more information, see https://courses.scinet.utoronto.ca/403

Road to Niagara 3: Hardware setup

March 5, 2018 in blog-technical, for_press, for_researchers, for_users, news, Road_to_Niagara, Uncategorized

This is the fourth of a series of posts on the transition to SciNet’s new supercomputer called “Niagara”, which will replace the General Purpose Cluster (GPC) and Tightly Coupled Cluster (TCS). The transition to Niagara will take place in the fall of 2017, and the system is planned to be available to users in early 2018.

The University of Toronto has awarded the contract for Niagara to Lenovo, and some of the details of the hardware specifications of the Niagara system have been released:

The system will have the following hardware components:

  • 1,500 nodes.
  • Each node will have 40 Intel Skylake cores (making a total of 60,000 cores) at 2.4 GHz.
  • Each node will have 200 GB (188 GiB)of DDR4 memory.
  • The interconnect between the nodes will be Mellanox EDR Infiniband in a Dragonfly+ topology.
  • A ~9PB usable shared parallel filesystem (GPFS) will be mounted on all nodes.
  • A 256TB Excelero burst buffer (NVMe fabric, up to 160 GB/s) will be available for fast I/O.
  • Peak theoretical speed: 4.61 PetaFLOPS

Niagara is estimated to be installed and operational towards in March 2018, and ready for users not too long after.

Even before official ready-date, there will a period in which select users can try out and port their codes to Niagara.

After the friendly-user period, all current users of the GPC (and former users of the TCS) will get access to Niagara.

The large core count, ample memory per core, and fast interconnect support Niagara’s intended purpose to enable large parallel compute jobs of 512 cores or more.

The software setup will also be tailored to large parallel computations. Nonetheless, there will still be a fair amount of backfill opportunity for smaller jobs.

The setup of Niagara is intended to be similar in spirit to the GPC, but different in form: scheduling per node, a home, scratch and possibly project directory defined in environment variables, a module system, and access to our team of analyst to help you get your codes running, and running well.

New Courses and New Initiatives for this Coming Semester

September 2, 2017 in for_educators, for_press, frontpage, news, Uncategorized


Excited about the beginning of a new academic year?

We, at SciNet, certainly are!

SciNet has created several new courses for this coming fall semester and we are really excited about that!
Take a look at our education website to learn about all the courses and workshops that we will be offering.

In addition to the traditional courses on Scientific Computing, we have also added courses on Computational BioStatistics, Machine Learning and Neural Networks, and basic level introductory courses for students without any previous background on computing or programming!

Additionally, several members of our team have obtained Graduate restricted appointed positions at the Institute of Medical Sciences and the Physics Department!

The number of SciNet courses that are listed as U of T graduate courses continues to increase (no small feat for a non-teaching unit like SciNet). Our full-term graduate courses in 2017/2018 are

Finally, starting this September we want to officially launch our “Research Initiative Program”!

This is a collaborative program, aimed to partner with research groups across the University, in order to boost and empower research.

Of course, research support is something that we have been doing since the beginning of SciNet, by providing technical support and the infrastructure to researchers for tackling their computational needs.
This program will go beyond that, by allowing researchers to explicitly partner with SciNet’s scientists, in order to pursue short and long term research projects.

More information about this program, ongoing collaborations and areas of expertise can be found at the
Research @ SciNet page.

2017 Compute Ontario Summer School Central

June 14, 2017 in blog, blog-general, for_educators, for_press, for_researchers, for_users, frontpage, news, Uncategorized

The Compute Ontario Summer School on Scientific and High Performance Computing is an annual educational event for graduate/undergraduate students, postdocs and researchers who are engaged in a compute intensive research. Held geographically in the west, centre and east of the province of Ontario, the summer school provides attendees with the opportunity to learn and share knowledge and experience in high performance and technical computing on modern HPC platforms.

Each site will have a slightly different list of courses. The summer school will include both in-class lectures and hands-on labs (done on the participants’ laptops). Those who attend at least three full days cumulatively will receive an official certificate in HPC training (i.e., a total of 6 full morning and afternoon sessions).

Instructors for this school have been provided by SciNet, CAMH and SHARCNET. Break refreshments are provided courtesy of Compute Ontario.

REGISTRATION

Registration for the central installment in Toronto from July 24-28, 2017 is now open!

The registration is free and is aimed at Compute Canada users as well as students, post-docs and other researchers from academic institutions. You do not need to have a SciNet account. Please be advised that seats are limited and tend to fill up.

More information and registration can be found on the summer school website.

SCHEDULE

High Performance Computing Stream Data Science Stream Biomedical Stream
Mon, Jul 24
Morning: 09:00-12:00
Welcome and Introduction to HPC and SciNet
Afternoon: 13:30-16:30
Shared Memory Programming with OpenMP Introduction to the Linux Shell PLINK
Tue, Jul 25
Morning: 09:00-12:00
Shared Memory Programming with OpenMP Introduction to R Next Generation Sequencing
Afternoon: 13:30-16:30
Programming Clusters with Message Passing Interface Data Science with Python RNASeq
Wed, Jul 26
Morning: 09:00-12:00
Programming Clusters with Message Passing Interface Parallel R for Data Science Python for MRI analysis
Afternoon: 13:30-16:30
Programming Clusters with Message Passing Interface Python for High Performance Computing (Parallel Python) Image Analysis at Scale
Thu, Jul 27
Morning: 09:00-12:00
Programming GPUs with CUDA Visualization with Python Machine Learning for Neuroimaging
Afternoon: 13:30-16:30
Programming GPUs with CUDA Scientific Visualization Suites R for MRI analysis
Fri, Jul 28
Morning: 09:00-12:00
Programming GPUs with CUDA Debugging, Profiling and Bring-Your-Own-Code Lab Public Datasets for Neuroimaging
Afternoon: 13:30-16:30
Programming GPUs with CUDA Debugging, Profiling and Bring-Your-Own-Code Lab Unit Testing / Neuroinformatics Pipeline Development

LOCATION

This event will be held in the Medical Science Building at the University of Toronto, 1 King Circle, Toronto, Ontario, M5S 1A8, Canada.

The nearest subway station is “Queen’s Park”. Paid parking is available on the St. George Campus.

LODGING

Lodging is not provided by the organization. If you require lodging, you will have to make arrangements yourself. It may be worthwhile checking out the University’s summer residence program at www.studentlife.utoronto.ca/hs/summer .

MEALS

Meals are *not* provided by the organization, but refreshments will be provided during the morning and afternoon breaks, courtesy of Compute Ontario.

COMPUTING FACILITIES

For the hands-on sessions, participants are to bring their own laptop with working wireless and with an ssh client with X-windows installed. The latter is needed to connect to one of SciNet or SHARCNET supercomputers, to which the participant will get access for the duration of the School.

CERTIFICATES

Participants that complete at least three days worth of instruction (i.e., a total of 6 morning and afternoon sessions combined) are to receive a Compute Ontario Summer School Certificate on the last day of the School. Note that this certificate is separate from the SciNet certificates, but parts of the school may count towards a SciNet certificate as well.

Grand Opening of the ArcNet Space at MaRS

May 15, 2017 in blog-general, for_educators, for_press, for_researchers, for_users, frontpage, news, Uncategorized

On May 9, 2017, the Grand Opening of the ArcNet space took place (despite having moved a while ago). What is ArcNet? It is a space where expertise and support of Advanced Research Computing (the “ARC” in ArcNet) from three organizations come together. SciNet is the oldest of the three; It is the supercomputing consortium at the University of Toronto, which has been providing Canadian researchers with computational resources and expertise necessary to perform their research on scales not previously possible in Canada, from the biomedical sciences and aerospace engineering to astrophysics and climate science. SOSCIP is a research and development consortium that pairs academic and industry researchers with advanced computing tools to fuel Canadian innovation. The third organization, Compute Ontario, partners with the four academic computing consortia in Ontario aims to drive advanced computing to accelerate research and enhance competitiveness in the global marketplace resulting in a more prosperous Ontario.

The Grand Opening brought together many of our stakeholders, and was also attended by the Ontario Minister of Research, Innovation and Science, MPP Rezi Moridi, by the President of the University of Toronto Meric Gertler, by the Vice-President of Research at the University of Toronto Prof. Vivek Goel, and by the Scientific Directory of SciNet Prof. W. Richard Peltier. We were honoured that each was willing to say a few words about the opening of this space.

“This facility is a true partnership between the University of Toronto’s SciNet High Performance Computing Consortium, SOSCIP and Compute Ontario. Bringing them together in a state-of-the-art facility will strengthen their partnership and undoubtably create new opportunities to drive innovation through advanced computing in Ontario.” said Prof. Goel, one of the driving forces of the creation of the space.

The President of the University of Toronto reminded us that “With ARCNet, we have created an amazing hub of talent and technology that fosters collaboration between the public and private sector.”

Minister Rezi Moridi remarked that “Today we are here to celebrate the grand opening of the Advanced Research Computing facility. It is great to see that three organizations got together and set up this wonderful facility: SOSCIP,
Compute Ontario, and University of Toronto’s SciNet. I wish you all the best in serving our research community.”


The speeches were given in the new teaching and visualization room. This room holds up to 40 students and is already frequently used for courses and other events. It features a large visualization wall, i.e., a 13 x 7.5 feet ultrahigh resolution screen (8K, to be precise).


Two presentations were given by SciNet analysts to demonstrate the capabilities of this visualization wall; Ramses van Zon showed how to get insight into the complexity of the software installed on SciNet’s main cluster by using graph visualizations, while Marcelo Ponce showed visualizations of several aspects of interacting neutron stars, with data from numerical general relativity simulations.

You can see the capabilities of the visualization wall in the following video:




Expired: HPCS 2017, Kingston June 5-9, 2017

April 25, 2017 in blog-general, for_researchers, for_users, news, Uncategorized

HPCS (the High Performance Computing Symposium) is Canada’s premiere Advanced Research Computing (ARC) conference, bringing together top researchers from across Canada and around the world, as well as major industry partners.

This year’s conference is being held in Kingston, Ontario – June 5th – 9th, and will include a range of keynote sessions and technical workshops designed to appeal to the research community and ARC professionals. Topics will include “traditional” HPC disciplines, as well as emerging areas such as cognitive computing – and there will have sessions exploring future technologies.

For more information see 2017.hpcs.ca

Note that Ontario students can go for fee (see the flyer).

International HPC Summer School 2017 in Boulder, CO

January 25, 2017 in for_educators, for_researchers, for_users, frontpage, news, Uncategorized

boulder-jay-05apr2016-cropped

Apply by 6 March, 2017
Expenses-paid program
Sponsored by PRACE, XSEDE, Riken, and Compute Canada
website: https://confluence.xsede.org/display/IH17/International+HPC+Summer+School+2017

The eighth International Summer School on HPC Challenges in Computational Sciences will be held from June 25-July 30, 2017, Boulder, Colorado, USA. This is an advanced summer school on High Performance Computing which targets graduate students who already have some experience in HPC parallel programming (for instance, MPI, OpenMP, or CUDA/OpenCL), preferably on software used in successful research projects.

The organizers of this summer school are XSEDE, PRACE, Compute Canada, and RIKEN.

Leading American, Canadian, European and Japanese computational scientists and HPC technologists will offer instruction on a variety of topics. The program is still being finalized, but previous summer schools included the following:

  • Access to EU, Canadian, Japanese and U.S. HPC-infrastructures
  • HPC challenges by discipline (e.g., bioinformatics, computer science, chemistry, and physics)
  • HPC Programming Proficiencies
  • Performance analysis & profiling
  • Algorithmic approaches & numerical libraries
  • Data-intensive computing
  • Scientific visualization

Participation in the summer school is decided through an application process. Meals, housing, and travel will be covered for the selected participants. Applications from students in all science and engineering fields are welcome. Ten out of 80 student participants will be from Canada. Preference will be given to applicants with parallel programming experience, and a research plan that will benefit from using high performance computing systems.

Applications are due by March 6, 2017
For further information and to apply online, please click here.

Expired: General Purpose Cluster (GPC)

November 9, 2016 in Uncategorized


The General Purpose Cluster (GPC) is our extremely large “workhorse” cluster (ranked 16th in the world at its inception, then the fastest in Canada) and is where most computations are done at SciNet – it has already performed more than 45,000,000 computations for Canadian researchers.  It is an IBM iDataPlex cluster based on Intel’s Nehalem architecture (one of the first in the world to make use of the new chips).

The GPC consists of 3,780 nodes (IBM iDataPlex DX360M2) with a total of 30,240 cores (Intel Xeon E5540) at 2.53GHz, with 16GB RAM per node (2GB per core) with some larger-memory nodes up to 32GB.   The nodes run Linux.   Approximately one quarter of the cluster is connected with non-blocking DDR InfiniBand while the rest of the nodes are connected with 5:1 blocked QDR InfiniBand. The compute nodes are accessed through a queuing system that allows jobs with a maximum wall time of 48 hours and a minimum time, in the batch queue, of 15 minutes.

A “quickstart” guide to using SciNet’s GPC can be found on our technical documentation wiki.

High Performance Storage System (HPSS)

November 8, 2016 in for_researchers, for_users, Systems, Uncategorized


The High Performance Storage System (HPSS) is a tape-backed hierarchical storage system that provides a significant portion of the allocated storage space at SciNet. It is a repository for archiving data that is not being actively used. Data can be returned to the active filesystem on the compute clusters when it is needed.

SciNet’s HPSS currently stores nearly 9 PB of data.

For more information, see the technical documentation on the SciNet wiki

SOSCIP GPU Cluster (SGC)

September 3, 2016 in Systems, Uncategorized

The SOSCIP GPU Cluster (SGC)is a Southern Ontario Smart Computing Innovation Platform (SOSCIP) resource located at the University of Toronto’s SciNet HPC facility. The SOSCIP multi-university/industry consortium is funded by the Ontario Government and the Federal Economic Development Agency for Southern Ontario.

The SOSCIP GPU Cluster consists of of 14 IBM Power 822LC “Minsky” Servers each with 2x10core 3.25GHz Power8 CPUs and 512GB Ram. Similar to Power 7, the Power 8 utilizes Simultaneous MultiThreading (SMT), but extends the design to 8 threads per core allowing the 20 physical cores to support up to 160 threads. Each node has 4x NVIDIA Tesla P100 GPUs each with 16GB of RAM with CUDA Capability 6.0 (Pascal) connected using NVlink.

Allocations of this system are done through Southern Ontario Smart Computing Innovation Platform (SOSCIP)

A quickstart guide to using this GPU cluster can be found on SciNet’s technical documentation wiki.