Tag Archives: featured

New ARC Cluster : Huckleberry

ARC released a new cluster named Huckleberry in late 2017. The Huckleberry system, accessed at huckleberry1.arc.vt.edu, was installed with deep learning applications in mind. To this end, it consists of 14 IBM “Minsky” S822LC nodes and NVIDIA’s proprietary NVLink interconnect network. This system enables highly parallel and highly distributed workloads. IBM unveiled its deep learning AI toolkit called PowerAI alongside the launch of Minsky nodes that leverage CPUs linked to Power CPUs with NVLink making it possible to have high speed high performance computing. PowerAI is available under /opt/DL in Huckleberry.

Each compute node on Huckleberry (i.e. IBM “Minsky” nodes) consists of :

  • Two IBM Power8 with 10 cores, 8 threads per core and memory bandwidth 115gb/s per socket
  • Four NVIDIA P100 GPUs advertised to have 21 teraFLOPS of 16-bit floating-point performance ideal for deep learning applications deliver high performance, massive parallelism
  • NVIDIA’s NVLink technology which provides high bandwidth data transfers between CPUs and GPUs; an improvement over PCI-Express
  • Mellanox EDR Infiniband (100 GB/s) interconnect used to connect compute nodes

The PowerAI toolkit contains Caffe, TensorFlow etc. which are optimized for the Power servers. IBM provides support for it as well.

While the rest of the clusters make use of the PBS batch systems, Huckleberry makes use of the Slurm batch system using the command sbatch.

Individuals may request a Huckleberry account.  Instructors can get set up class accounts for Huckleberry as well.

Gas-surface interactions are everywhere

Even as you read this, molecules of gas are colliding with your skin and the walls around you. These kinds of interactions between gases and surfaces are important in many different fields. For example, consider an important problem in the life sciences: respiration. Breathing itself is a process that involves a gas-surface interaction.

As oxygen molecules sprint around the atmosphere, eventually they will end up in your blood stream. Somewhere along the way they have interacted with the surfaces of your lungs and passed into your blood stream. This is what Diego Troya, an associate professor of Chemistry at Virginia Tech, is interested in. His lab is trying to understand the interactions between gases and surfaces.

 As oxygen molecules sprint around the atmosphere, eventually they will end up in your blood stream. Somewhere along the way they have interacted with the surfaces of your lungs and passed into your blood stream. This is what Diego Troya, an associate professor of Chemistry at Virginia Tech, is interested in. His lab is trying to understand the interactions between gases and surfaces.

His lab is trying to carry out simulations of the molecular dynamics of collisions between the gases of the low-Earth orbit atmosphere and models of the polymers that are used as thermal blankets or protective paints on the spacecraft surface. These simulations will provide valuable information about the microscopic reaction mechanisms of erosion processes that cannot be determined from in-orbit or ground-based experiments.

His lab is also studying the adsorption of chemical-warfare agents on surfaces. Recent news has reported the use of substances like Sarin as a warfare agent. This gas is a nerve agent that deactivates the enzyme that controls muscle relaxation, and can be fatal in large concentrations.

Use of these subtances in the battlefield also results in contamination of equipment and other materials like sand that might inadvertently get shipped back to the United States. In the Troya lab, modeling work is used to find out how this agent sticks to common surfaces so that more efficient decontamination trategies can be developed. Because nerve agents like Sarin are extremely toxic, studies in a laboratory setting are challenging. Therefore, computational simulations can fill the experimental void and help to get a complete picture of the surface chemistry of this gas.

Because the lab is engaged in research of the interactions of gas moleculs with extended surfaces, Dr. Troya’s lab uses over a million CPU hours a year. In the spring of 2013, Troya’s lab started using BlueRidge, Virginia Tech Advanced Research Center’s new supercomputer, and his lab has benefitted tremendously from this resource. Blueridge provides the type of cutting-edge research equipment that provides results very quickly, and it has been an incredible gift for their laboratory. “BlueRidge is many times faster than the prior computer we used to operate. A lot of problems that we couldn’t tackle before Blueridge hav now become tractable,” said Prof. Troya.

Since the lab is using this state-of-the-art equipment, the researchers have to be properly trained to exploit its capabilities. In his laboratory, Dr. Troya is a hands-on adviser to his graduate students (Angela Edwards, Robert Chapleski, and Jacky Chan). He always tries to work with them individually, and it is not rare to find him sitting with them as they learn to write computer programs or analyze the result of the simulations.