Road to Niagara 3: Hardware setup

March 5, 2018 in blog-technical, for_press, for_researchers, for_users, news, Road_to_Niagara, Uncategorized

This is the fourth of a series of posts on the transition to SciNet’s new supercomputer called “Niagara”, which will replace the General Purpose Cluster (GPC) and Tightly Coupled Cluster (TCS). The transition to Niagara will take place in the fall of 2017, and the system is planned to be available to users in early 2018.

The University of Toronto has awarded the contract for Niagara to Lenovo, and some of the details of the hardware specifications of the Niagara system have been released:

The system will have the following hardware components:

  • 1,500 nodes.
  • Each node will have 40 Intel Skylake cores (making a total of 60,000 cores) at 2.4 GHz.
  • Each node will have 200 GB (188 GiB)of DDR4 memory.
  • The interconnect between the nodes will be Mellanox EDR Infiniband in a Dragonfly+ topology.
  • A ~9PB usable shared parallel filesystem (GPFS) will be mounted on all nodes.
  • A 256TB Excelero burst buffer (NVMe fabric, up to 160 GB/s) will be available for fast I/O.
  • Peak theoretical speed: 4.61 PetaFLOPS

Niagara is estimated to be installed and operational towards in March 2018, and ready for users not too long after.

Even before official ready-date, there will a period in which select users can try out and port their codes to Niagara.

After the friendly-user period, all current users of the GPC (and former users of the TCS) will get access to Niagara.

The large core count, ample memory per core, and fast interconnect support Niagara’s intended purpose to enable large parallel compute jobs of 512 cores or more.

The software setup will also be tailored to large parallel computations. Nonetheless, there will still be a fair amount of backfill opportunity for smaller jobs.

The setup of Niagara is intended to be similar in spirit to the GPC, but different in form: scheduling per node, a home, scratch and possibly project directory defined in environment variables, a module system, and access to our team of analyst to help you get your codes running, and running well.

2017 Compute Ontario Summer School Central

June 14, 2017 in blog, blog-general, for_educators, for_press, for_researchers, for_users, frontpage, news, Uncategorized

The Compute Ontario Summer School on Scientific and High Performance Computing is an annual educational event for graduate/undergraduate students, postdocs and researchers who are engaged in a compute intensive research. Held geographically in the west, centre and east of the province of Ontario, the summer school provides attendees with the opportunity to learn and share knowledge and experience in high performance and technical computing on modern HPC platforms.

Each site will have a slightly different list of courses. The summer school will include both in-class lectures and hands-on labs (done on the participants’ laptops). Those who attend at least three full days cumulatively will receive an official certificate in HPC training (i.e., a total of 6 full morning and afternoon sessions).

Instructors for this school have been provided by SciNet, CAMH and SHARCNET. Break refreshments are provided courtesy of Compute Ontario.

REGISTRATION

Registration for the central installment in Toronto from July 24-28, 2017 is now open!

The registration is free and is aimed at Compute Canada users as well as students, post-docs and other researchers from academic institutions. You do not need to have a SciNet account. Please be advised that seats are limited and tend to fill up.

More information and registration can be found on the summer school website.

SCHEDULE

High Performance Computing Stream Data Science Stream Biomedical Stream
Mon, Jul 24
Morning: 09:00-12:00
Welcome and Introduction to HPC and SciNet
Afternoon: 13:30-16:30
Shared Memory Programming with OpenMP Introduction to the Linux Shell PLINK
Tue, Jul 25
Morning: 09:00-12:00
Shared Memory Programming with OpenMP Introduction to R Next Generation Sequencing
Afternoon: 13:30-16:30
Programming Clusters with Message Passing Interface Data Science with Python RNASeq
Wed, Jul 26
Morning: 09:00-12:00
Programming Clusters with Message Passing Interface Parallel R for Data Science Python for MRI analysis
Afternoon: 13:30-16:30
Programming Clusters with Message Passing Interface Python for High Performance Computing (Parallel Python) Image Analysis at Scale
Thu, Jul 27
Morning: 09:00-12:00
Programming GPUs with CUDA Visualization with Python Machine Learning for Neuroimaging
Afternoon: 13:30-16:30
Programming GPUs with CUDA Scientific Visualization Suites R for MRI analysis
Fri, Jul 28
Morning: 09:00-12:00
Programming GPUs with CUDA Debugging, Profiling and Bring-Your-Own-Code Lab Public Datasets for Neuroimaging
Afternoon: 13:30-16:30
Programming GPUs with CUDA Debugging, Profiling and Bring-Your-Own-Code Lab Unit Testing / Neuroinformatics Pipeline Development

LOCATION

This event will be held in the Medical Science Building at the University of Toronto, 1 King Circle, Toronto, Ontario, M5S 1A8, Canada.

The nearest subway station is “Queen’s Park”. Paid parking is available on the St. George Campus.

LODGING

Lodging is not provided by the organization. If you require lodging, you will have to make arrangements yourself. It may be worthwhile checking out the University’s summer residence program at www.studentlife.utoronto.ca/hs/summer .

MEALS

Meals are *not* provided by the organization, but refreshments will be provided during the morning and afternoon breaks, courtesy of Compute Ontario.

COMPUTING FACILITIES

For the hands-on sessions, participants are to bring their own laptop with working wireless and with an ssh client with X-windows installed. The latter is needed to connect to one of SciNet or SHARCNET supercomputers, to which the participant will get access for the duration of the School.

CERTIFICATES

Participants that complete at least three days worth of instruction (i.e., a total of 6 morning and afternoon sessions combined) are to receive a Compute Ontario Summer School Certificate on the last day of the School. Note that this certificate is separate from the SciNet certificates, but parts of the school may count towards a SciNet certificate as well.

Grand Opening of the ArcNet Space at MaRS

May 15, 2017 in blog-general, for_educators, for_press, for_researchers, for_users, frontpage, news, Uncategorized

On May 9, 2017, the Grand Opening of the ArcNet space took place (despite having moved a while ago). What is ArcNet? It is a space where expertise and support of Advanced Research Computing (the “ARC” in ArcNet) from three organizations come together. SciNet is the oldest of the three; It is the supercomputing consortium at the University of Toronto, which has been providing Canadian researchers with computational resources and expertise necessary to perform their research on scales not previously possible in Canada, from the biomedical sciences and aerospace engineering to astrophysics and climate science. SOSCIP is a research and development consortium that pairs academic and industry researchers with advanced computing tools to fuel Canadian innovation. The third organization, Compute Ontario, partners with the four academic computing consortia in Ontario aims to drive advanced computing to accelerate research and enhance competitiveness in the global marketplace resulting in a more prosperous Ontario.

The Grand Opening brought together many of our stakeholders, and was also attended by the Ontario Minister of Research, Innovation and Science, MPP Rezi Moridi, by the President of the University of Toronto Meric Gertler, by the Vice-President of Research at the University of Toronto Prof. Vivek Goel, and by the Scientific Directory of SciNet Prof. W. Richard Peltier. We were honoured that each was willing to say a few words about the opening of this space.

“This facility is a true partnership between the University of Toronto’s SciNet High Performance Computing Consortium, SOSCIP and Compute Ontario. Bringing them together in a state-of-the-art facility will strengthen their partnership and undoubtably create new opportunities to drive innovation through advanced computing in Ontario.” said Prof. Goel, one of the driving forces of the creation of the space.

The President of the University of Toronto reminded us that “With ARCNet, we have created an amazing hub of talent and technology that fosters collaboration between the public and private sector.”

Minister Rezi Moridi remarked that “Today we are here to celebrate the grand opening of the Advanced Research Computing facility. It is great to see that three organizations got together and set up this wonderful facility: SOSCIP,
Compute Ontario, and University of Toronto’s SciNet. I wish you all the best in serving our research community.”


The speeches were given in the new teaching and visualization room. This room holds up to 40 students and is already frequently used for courses and other events. It features a large visualization wall, i.e., a 13 x 7.5 feet ultrahigh resolution screen (8K, to be precise).


Two presentations were given by SciNet analysts to demonstrate the capabilities of this visualization wall; Ramses van Zon showed how to get insight into the complexity of the software installed on SciNet’s main cluster by using graph visualizations, while Marcelo Ponce showed visualizations of several aspects of interacting neutron stars, with data from numerical general relativity simulations.

You can see the capabilities of the visualization wall in the following video:




Expired: HPCS 2017, Kingston June 5-9, 2017

April 25, 2017 in blog-general, for_researchers, for_users, Uncategorized

HPCS (the High Performance Computing Symposium) is Canada’s premiere Advanced Research Computing (ARC) conference, bringing together top researchers from across Canada and around the world, as well as major industry partners.

This year’s conference is being held in Kingston, Ontario – June 5th – 9th, and will include a range of keynote sessions and technical workshops designed to appeal to the research community and ARC professionals. Topics will include “traditional” HPC disciplines, as well as emerging areas such as cognitive computing – and there will have sessions exploring future technologies.

For more information see 2017.hpcs.ca

Note that Ontario students can go for fee (see the flyer).

Resource Allocation Competition for 2017 Opened

October 12, 2016 in blog, for_researchers, for_users

Yesterday, Compute Canada launched its annual Resource Allocation Competitions (RAC), a peer-reviewed process to grant priority access to Compute Canada’s advanced research computing resources.

The process for getting these resources has been restructured. If your research group needs computing or storage resources on Compute Canada systems in 2017, then you will need to understand the renewed process for getting such resources.

Names for various kinds of resource allocations have changed. For instance, what was formally called a RAC application, now is called an application to the RRG competition (“Resources for Research Groups”), and what was formally a default allocation is now RAS (“Rapid Access Service”). New competitions have been added as well, such as the RPP (Research Platforms and Portals) competition. The RAC acronym is now used as an overall term for all of these competitions.

So if you held a RAC allocation in previous years and want to apply for one in 2017, or you were thinking of applying, you likely want an RRG allocation.

On the Compute Canada site, you can find more information on the RGG and RPP, and on the RAS.

To explain these and other changes to the process, on Thursday Oct 13, 2016, from 12 noon to 1:30 pm, there will be an on-line Q&A session. SciNet will broadcast this session in our boardroom, where SciNet analysts will also be present to take questions (note that SciNet has recently moved to the MaRS building).

Gravitation waves detected, again!

June 15, 2016 in blog, blog-general, blog-technical, for_press, for_researchers, for_users, in_the_news, news, success_story, Testimonials

We congratulate the LIGO and Virgo collaborations to the second-ever observation of gravitational waves from colliding black holes.

SciNet is proud to have contributed to the computation of the waveform templates that were used in this latest discovery of LIGO. LIGO measured about 55 gravitational wave cycles for this new binary black hole system. This large number of cycles made detailed computations of the expected wave-shapes more important than for the first detected black hole merger that was announced in February.

Canada is a leader of numerical calculations of colliding black holes, research led by Professor Harald Pfeiffer, Canada Research Chair for Numerical Relativity and Gravitational Wave Astrophysics at the University of Toronto. Pfeiffer states: I am very grateful for the sustained support of the SciNet team during the last 7 years; their support and the access to computing time on SciNet’s supercomputers have been crucial for my research program and its profound contributions to the LIGO discovery.

frame06508

Above: The in-spiral and collision of two black holes similar to GW151226. The top portion of the frame shows the horizons of the two holes, in this case, at the moment close to the merger of the black holes. The middle portion of the frame shows the gravitational waveform projected onto the LIGO Livingston detector. The bottom part shows the frequency of the gravitational waves, gradually increasing from about 35Hz to above 700Hz. For this system, LIGO could observe many more gravitational wave cycles than for the first discoved system (named GW150914).

Visualization done by University of Toronto Undergraduate student Aliya Babul & Prof. Harald Pfeiffer, within the SXS Collaboration/www.black-holes.org.

2016 Ontario HPC Summer School – Toronto

June 2, 2016 in blog, blog-general, for_educators, for_press, for_researchers, for_users, frontpage, news

ohpcss

The 2016 Ontario Summer School on High Performance Computing is a week-long academic workshop that provides attendees with opportunities to learn and share knowledge and experiences in high performance computing, technical computing, and data science. The Ontario HPC Summer School is given in three locations, in different weeks. The first one was from May 30 to June 3 in Hamilton, the second will be in Toronto from July 11 to 15, hosted by SciNet, while a third will take place in Ottawa from August 8 to 12.

The format of the school is that of a five-day workshop with mixed lectures and hands-on sessions on a number of selected topics, including shared memory programming, distributed memory programming and general purpose graphics processing unit programming, and data science. The program varies slightly per location.

Most sessions are given in parallel. When registering for the Toronto event, you can pick the sessions you are interested in from the following schedule (note that you cannot register for only part of a session):

SCHEDULE

Monday July 11, 2016
9:30 am – 12:30 pm
single stream: Intro to high performance computing and SciNet
1:30pm – 4:30 pm
stream 1: Shared memory programming with OpenMP, part 1 of 2
stream 2: Intro to the Linux Shell

Tuesday July 12, 2016
9:30 am – 12:30 pm
stream 1: Shared memory programming with OpenMP, part 2 of 2
stream 2: R for data science
1:30pm – 4:30 pm
stream 1: Programming Clusters with MPI, part 1 of 3
stream 2: Parallel R for data science

Wednesday July 13, 2016
9:30 am – 12:30 pm
stream 1: Programming Clusters with MPI, part 2 of 3
stream 2: Python for scientific computing
1:30pm – 4:30 pm
stream 1: Programming Clusters with MPI, part 3 of 3
stream 2: Python for high performance computing

Thursday July 14, 2016
9:30 am – 12:30 pm
stream 1: Programming GPUs with CUDA, part 1 of 4
stream 2: Visualization, part 1 of 2
1:30pm – 4:30 pm
stream 1: Programming GPUs with CUDA, part 2 of 4
stream 2: Visualization, part 2 of 2

Friday July 15, 2016
9:30 am – 12:30 pm
stream 1: Programming GPUs with CUDA, part 3 of 4
stream 2: Debugging
1:30pm – 4:30 pm
stream 1: Programming GPUs with CUDA, part 4 of 4
stream 2: Bring your own code

LOCATION

This event will be held in the Mechanical Engineering Building at the University of Toronto

Wallberg Building
Rooms 116 and 119
University of Toronto
St. George Campus
184-200 College Street
Toronto, Ontario, M5S 3E5
Canada.

The nearest subway station is “Queen’s Park”.
Paid parking is available on the St. George Campus.

LODGING

Lodging is not provided by the organization. If you require lodging, you will have to make arrangements yourself. It may be worthwhile checking out the University’s summer residence program at www.studentlife.utoronto.ca/hs/summer .

MEALS

Meals are not provided by the organization, but refreshments will be provided during the morning and afternoon breaks.

COMPUTING FACILITIES

For the hands-on sessions, participants are to bring their own laptop with working wireless and with an ssh client with X-windows installed. The latter is needed to connect to one of SciNet’s supercomputers, to which the participant will get access for the duration of the School.

CERTIFICATES

Participants that complete at least three days worth of instruction are to receive an Ontario Summer School Certificate on the last day of the School. Note that this certificate is separate from the SciNet certificates, but parts of the school may count towards a SciNet certificate as well.

MORE INFORMATION AND REGISTRATION

For more information on the sessions and for registration, please go to
www.sharcnet.ca/events/ss2016

The registration is free and is aimed at Compute Canada users as well as students, post-docs and other researchers from academic institutions. You do not need to have a SciNet account. Please be advised that seats are limited and tend to fill up.

Gravitational Waves detected!!!

February 12, 2016 in blog, for_educators, for_press, for_researchers, for_users, in_the_news, news, success_story, Uncategorized

On February 11th, 2016 (10:30AM EST), scientists from Caltech, MIT and the LIGO Scientific Collaboration together with representatives of the National Science Foundation, announced in a live press conference, the first direct detection of gravitational waves.
The event detected, named GW150914, was produced by two colliding black holes, inspariling and merging together. This signal was detected by LIGO on September 14th, 2015.

Details of this discovery can be found in the following papers:

Many of the researchers running simulations and analysing data in several of our clusters are directly involved in the efforts for accurately modelling, simulating and detecting gravitational wave signals.
As a matter of fact, one of the simulations used for visualizing one of the movies screened during the announcement of the discovery of gravitational waves, was performed on SciNet’s General Purpose Cluster (GPC).

“The first detection of gravitational waves passing through Earth is a tremendous discovery. These waves were generated by never before observed astronomical objects, colliding black holes. I am very grateful for SciNet and Compute Canada to provide the computing resources needed to explore the properties of binary black holes, research that was instrumental in building the waveform-templates that LIGO used in its momentous discovery.”
Prof. Harald Pfeiffer (Canadian Institute for Theoretical Astrophysics, University of Toronto)

plane0060plane0320 plane0490 plane0590 plane0830
Simulation of a binary black-hole system emitting gravitational radiation (gravitational waves) performed on SciNet’s General Purpose Cluster (GPC).
These simulations were performed by solving Einstein’s equations of General Relativity using a code co-developed on SciNet, by the SXS/CITA collaboration.
Credit: SXS/CITA/SciNet
Another of the videos displayed on the press-conf, was also produced by solving Einstein equations using this same computer code co-developed on SciNet by the SXS/CITA collaboration.

Big Data Challenge for HighSchool Students 2016

February 12, 2016 in blog-general, for_educators, for_press, frontpage, success_story

IMG_20160204_091035 IMG_20160204_144339 IMG_20160204_144158

SciNet in partnership with STEM Fellowship (http://stemfellowship.org/), SAS and Open Data Toronto, organized the second edition of the “Big Data Challenge for High School Students”.

On Feb. 4th, the 2015/2016 Big Data Challenge for high school students took place. 8 teams from several schools across the GTA presented their research on data analytic in front of peers and judges.

Inspired by “Big Data in the City” theme, students gathered data from Open Data Toronto, analysed and investigated topics such as: immigration relocation strategies, emergency response for first responders services, identification of clusters in Toronto, environmental analysis of Toronto neighbourhoods, debt risk analysis of the city, collision patterns and prevention, data mining from social media related to energy efficient companies, among many others.

SciNet members, in addition to organize this event participated evaluating the initial proposals and judging the final 8 qualified for the final presentation.

IMG_20160204_143755

Participants of the Big Data Challenge will be participating in tours to SciNet’s datacenter, as an unique opportunity to experience and visit the home of the largest super-computers in Canada!

 

Congratulations to all the participants!!!

 

Further information can be found in the following links:

http://stemfellowship.org/bigdata

http://journal.stemfellowship.org/doi/abs/10.17975/sfj-2015-013

https://support.scinet.utoronto.ca/education/go.php/230/index.php/ib/1//p_course/230

International HPC Summerschool 2016 in Slovenia

December 16, 2015 in blog, for_educators, for_researchers, for_users, frontpage, news, Uncategorized

Ljubljana_panorama

Apply by 15 February 2016, decisions on March 9, 2016
Expenses-paid program
Sponsored by PRACE, XSEDE, Riken, and Compute Canada
website: http://ihpcss2016.hpc.fs.uni-lj.si/

The seventh International Summer School on HPC Challenges in Computational Sciences will be held from June 26-July 1, 2016, Ljubljana, Slovenia. This is an advanced summer school on High Performance Computing which targets graduate students and postdocs who already have some experience in HPC parallel programming (for instance, MPI, OpenMP, or CUDA/OpenCL), preferably on software used in successful research projects.

The organizers of this summer school are XSEDE, PRACE, Compute Canada, and RIKEN.

Leading American, Canadian, European and Japanese computational scientists and HPC technologists will offer instruction on a variety of topics. The program is still being finalized, but previous summer schools included the following:

  • Access to EU, Canadian, Japanese and U.S. HPC-infrastructures
  • HPC challenges by discipline (e.g., bioinformatics, computer science, chemistry, and physics)
  • HPC Programming Proficiencies
  • Performance analysis & profiling
  • Algorithmic approaches & numerical libraries
  • Data-intensive computing
  • Scientific visualization

Participation in the summer school is decided through an application process. Meals, housing, and travel will be covered for the selected participants. Applications from students in all science and engineering fields are welcome. Although the school targets graduate students and postdocs, applications from research assistants and faculty are also welcome. Preference will be given to applicants with parallel programming experience, and a research plan that will benefit from the utilization of high performance computing systems.

Applications are due by February 15, 2016.
For further information and to apply online, please click here.