ARC’s primary mission is to facilitate breakthrough research at Virginia Tech. To this end, ARC uses an allocation system to ensure that system time is distributed in a manner appropriate to research needs while allowing faculty members and PIs the flexibility to ensure that the time allocated to a given project is managed (e.g., among graduate students) so as to maximum productivity. An allocation is a system time account requested and managed by a single person (e.g., a project PI). Many users (e.g., Co-PIs or graduate students) can then be granted access to charge against a single allocation.
Note: The ARC systems currently governed by an allocation policy are BlueRidge, NewRiver, Dragonstooth and Cascades. Users do not need to be an authorized user of an approved allocation request to run jobs on other ARC system.
Note: Allocation applications can also include requests for other resources (e.g., additional storage) required to make a project successful.
There are two types of allocations, which differ somewhat in how they are awarded:
- Research Allocations are provided for research projects and usually managed by the project’s Principal Investigator (PI) (see Eligibility for Research Allocations, below). They are typically granted for a single year and can be renewed annually for the length of the project. Multi-year research allocations, such as for inclusion in a proposal submission, may be granted through negotiation with ARC.
- Instructional Allocations support academic classes and are managed by the faculty member or instructor responsible for the course. Instructional allocations are typically smaller, available for shorter time periods (e.g., for the duration of the associated course), and may be limited to a select set of systems.
System unit allocations on ARC’s systems are intended to ensure that users have the computing resources required to complete their research while also ensuring that no single user or group of users dominates the systems to the detriment of others. As such, allocations are awarded on a project-by-project basis and intended to be managed by the individual responsible for overseeing the research.
In order to manage a research project or allocation on ARC’s systems, a user must fall into one of the following categories:
- Be a current faculty member or post-doctoral researcher at Virginia Tech, OR
- Be an employee of Virginia Tech and the Principal Investigator (PI) for a research computing-related project, OR
- Be an employee of Virginia Tech and the Co-PI for a research computing-related project led by a non-Virginia Tech PI
Adjunct professors must provide a letter from their department chair, indicating that they are qualified to lead an internal research project, before their project and allocation requests can be approved.
Undergraduate and graduate students are not eligible to apply directly for projects and allocations, but must instead work under the sponsorship of a qualified researcher.
Undergraduate and graduate students should ask their advisor or research project PI to submit an allocation request. Once the request has been granted, they can be added to the project and submit jobs.
Allocations are measured in “system units”, which are calculated as simply the number of cores* multiplied by the wall clock time of a job. So if you run a job that uses 32 cores and takes 12 hours, your allocation will be charged 384 (32 x 12) system units. Because CPUs and GPUs have different computing power, ARC is currently reviewing whether system units should be calculated differently for CPU-GPU systems.
*Please note that the number of cores here refers to the number of cores reserved by your job. If you run a sequential (1 core) job on a node with 4 cores, you may be making all 4 cores unavailable to other researchers. In this case, your allocation will be charged for the use of all 4 cores. This encourages users to maximize their utilization of the resources that they reserve.
ARC uses a three tiered system for awarding research allocations. Larger allocations will eventually take longer to review, but in this initial rollout of the allocation system and BlueRidge, ARC will review them as soon as possible:
- Less than 200,000 system units: Requires a 200 word abstract
- 200,000 to 1 million system units: Requires a 1-2 page justification
- More than 1 million system units: Requires a 3-5 page justification
Justifications should describe the research problem or question being addressed. But the bulk of the justification should be devoted to the computational approach, such as:
- What are your software and hardware (e.g. CPU, GPU, memory, network) requirements?
- What are the types of code or programs that will be used?
- Is the code “mature” or is it still under development?
- How efficiently does the code or program scale across cores/nodes?
There are two types of allocation applications:
- New Application: New research projects or instructional allocations
- Renewal: For renewal of a previously granted research allocation, applicants will provide a short summary of the future year’s plans and be asked to update the project’s research output (grants and publications) in ARC’s research database. Allocation renewals are not currently supported for Instructional allocations.
The application for a new allocation involves the following steps:
- If you have not done so already, create a project in ARC’s research database to be associated with the allocation that you would like to request. In the Project Code field, enter the name that you would like to use for your allocation fund.
- Once you’ve created the project, if appropriate, add any grants or publications associated with the project.
- Once the project is complete, create an allocation request. You will be asked to provide a write-up commensurate with the allocation tier being requested.
- The application will be reviewed by the appropriate committee.
- If the allocation is approved, you will be notified and then be able to add users to the allocation on the project page. (See this FAQ for detailed instructions.)
If a project is still ongoing at the end of an allocation, the allocation manager (PI, etc) may request renewal of the allocation. Whereas initial allocation decisions require a justification, the process for renewal is based more on production during the previous allocation. Applying for allocation renewal involves the following steps:
- Update the associated project in ARC’s research database, including the project itself as well as its associated grants and publications.
- Complete ARC’s Allocation Request Form and make sure to set the Allocation Type to “Renewal”.
- The application will be reviewed by the appropriate committee and you will be notified if the allocation is approved.
Once you have an allocation, you should periodically review its status. As noted above, PIs can add users to or remove users from their allocation in the Users section of the page for the associated project. (See this FAQ for detailed instructions.) ARC hopes to add other functionality to the website, but for now other management tasks can be accomplished using the following commands when logged into BlueRidge:
- glsaccount: List your projects and their users
- glsfund -h -a <accountname>: List allocation size, remaining balance, and start and end dates for the <accountname> account
- gstatement -h -a <projectname>: Detailed list of jobs run against the <projectname> account. There are a variety of options for the gstatementcommand, such as start and end times, that might be of use as well – see gstatement --help for a list.