Menu

Scientific Computing Resources for Research

Scientific computing encompasses modeling, simulation, analysis, and artificial intelligence, and is vital to making significant scientific breakthroughs and driving innovation. High-performance computing (HPC), also known as supercomputing, is essential in performing these complex operations quickly and efficiently, as well as managing large amounts of data. By leveraging HPCs, along with advancements in software, algorithms, methods, and workflows, researchers gain access to powerful tools that enable them to study complex systems that would otherwise be impractical or impossible to investigate through traditional means.

DHSU offers both on-premise and cloud-based High Performance Computing (HPC) resources for research. These resources are managed by the HPC Specialist, Dr. Scott Bunnell, under the guidance of the HPC Faculty Advisory Committee and the Senior Vice President for Research (SVPR) Office. Below, we provide a summary of the available HPC resources, as well as guidelines on how to make use of them.

On-premise HPC

Description

The on-premise HPC was obtained through NIH S10 award and made available in early 2023. It includes a modern framework with compute and storage resources. These will be available to the entire DHSU research community. The HPC characteristics are as follows:

  • Compute: A compute cluster with over 1000 cores, comprising 16 64-core, each with a 7 TB hard drive and 512 GB of RAM memory; one 32-core node with 2 high-end nVIDIA A30 24-GB GPUs, 1024 GB of RAM and 5TB hard drive; and one 16-core "dev" node with 128 GB of RAM and 400 GB hard drive.
  • Storage: The cluster is served by a 1.2 PB storage device (NVMe/spinning), running the Lustre parallel file system. 
  • Networking: Compute cluster nodes and storage are interconnected through a fast 100-Gb Ethernet network. Offsite access is available via an institutional (science-DMZ) gateway.
  • Software: The job-scheduling software is Grid Engine.

Our hope is to gradually grow the on-premise HPC by incorporating new resources funded through extramural funding. We, therefore, encourage DHSU researchers to budget grant funds to extend the HPC – these resources can be prioritized exclusively to their team, but any  unused resources can be made available to other DHSU researchers. 

Maintenance Fee

Access to the on-premise HPC requires renting storage space on at a rate of $50/TB/year. This fee will be used to support continued HPC maintenance. The fee will provide the research group users with access to both storage and compute resources, and can be covered through extramural funding, department/institute funds or any other appropriate source of funding. If such funds are not available, research groups can submit a request with the appropriate justification for this fee to be covered by the SVPR Office (maximum of 20 TB/year). These requests will be evaluated by the HPC Faculty Advisory Committee and the SVPR Office. 

Resource Allocation

The on-premise HPC will include a workload manager / job scheduler software with policies to ensure equitable resource distribution among DHSU users. If the demand exceeds the resources, the committee reserves the right to prioritize those researchers with NIH funding, and particularly to those that were listed as potential users in the NIH grant that funded the HPC. More details can be found in the Policy on Allocation of HPC resources. These priorities will be implemented through the job scheduler software. 

Getting Started

To get started please complete this online form. This form requests the following information:

  • HIPAA declaration: All users must include a legally binding declaration that at no time will identifiable protected health information, regulated under HIPAA, be stored or processed on the scientific computing framework. 
  • Fee payment information: Please specify how your research group will cover the HPC maintenance fee of $50/TB/year, and may include extramural funding, department/institute funds or any other appropriate source of funding. To request this fee to be covered by the SVPR office, please specify: 1) the lack of funding sources and plans to obtain funding to cover this fee in the future; and 2) why you need on-premise HPC resources for your research.
  • Storage space requested: Number of TB/year of storage requested. This will provide access to compute resources. 
  • Desired username: This will be used to access the HPC
  • Mobile phone number: Will be used to assign a password to access the HPC.

Further details on how to access and use the on-premise HPC can be found in the HPC User Guide

For more information, please contact the HPC Lead (scott.bunnell@downstate.edu).

Cloud HPC

Description 

DHSU recognizes the important role that cloud HPC plays in modern research. Cloud HPC may be useful for those requiring compute and storage resources beyond what is currently offered at DHSU's on-premise HPC, Please note that PHI data cannot be stored or processed on Google Cloud under our current agreement.

Cloud HPC resources are available at DHSU via Google Cloud. Specifically, DHSU has signed the Internet2 NET+ Google Cloud service agreement with the Burwood Group Inc, an official Google Reseller. This agreement provides significant benefits for all DHSU researchers using Google Cloud, including: 

  • Discount of 5% on all products/services
  • Waived egress fees, i.e. network fees for moving out of the cloud
  • Discount of up to 25% on products/services for NIH-funded researchers through the NIH STRIDES program. Details on the NIH STRIDES program, which includes additional incentives, can be found here: https://datascience.nih.gov/strides
  • Multiple other benefits are described here: https://internet2.edu/services/google-cloud-platform/

Additionally, as part of this agreement, the Burwood Group and Google will provide the following services to DHSU Google Cloud users:

  • An easy-to-use web platform to track Google Cloud expenses and facilitate individualized billing 
  • Technical support and guidance on Google Cloud usage and services
  • Training opportunities  

Payment of Google Cloud Expenses

DHSU researchers are expected to cover any Google Cloud expenses, either through extramural funding, department/institute funds or any other appropriate source of funding. Burwood will provide a monthly detailed breakdown of Google Cloud expenses to facilitate separate payments by different research groups.

DHSU researchers who require cloud HPC but do not have funds to cover the costs are welcome to submit a request for SVPR funding to cover cloud HPC costs (see the Getting Started section below).

Google Cloud also provides a program that provide Google Cloud credits to researchers:  https://edu.google.com/programs/credits/research  

Resource Allocation

Google Cloud resources are on-demand and will be covered separately by the different research groups. DHSU will ensure appropriate mechanisms are in place to facilitate this payment, and will ensure research groups are fully compliant with any required payments. For details see point 6 of the Policy on Allocation of HPC resources [LINK].

Getting Started

To get started please complete this online form. It requests the required information to get you set up on Google Cloud and the Burwood billing platform. This includes contact information,  a description of the Google Cloud services that will be used and the funding method (grant, institution/department, etc).  

DHSU researchers who require cloud HPC but do not have funds to cover the costs are welcome to submit a request for SVPR funding to cover their cloud HPC costs via the same online form. The SVPR funding request section will request the following information: 

  • Project description 
  • Total budget requested for the 1 year period.
  • Justification for cloud need and requested resources (e.g. specific service not available on-premise, or need to complement on-premise with additional compute/storage)
  • Whether the project involved HIPAA-regulated data
  • How the cloud resources will contribute to future extramural funding and publications.

The proposals will be evaluated by the HPC Faculty Advisory Committee on a bi-monthly basis, and funding recommendations will be made to the SVPR office. The main criteria will be adequate justification for the need for cloud HPC resources and the potential for future funding/publications. 

For additional information please contact the HPC Lead (scott.bunnell@downstate.edu).

More details on how to access and use Google Cloud can be found in their online documentation: https://cloud.google.com/docs.

 

The following links provide useful information for researchers interested in using Google Cloud:

Other Cloud Resources