Policies

Pegasus Access and Allocations

CCS Account Registration
CCS Project Request

CCS Accounts

All users of the Pegasus supercomputer are required to have a CCS account.  To qualify for a CCS account, you must be affiliated with the University of Miami. All CCS accounts must be linked with a valid corresponding University of Miami account. CCS resources are open to all University of Miami employees and students.

Once you have a valid University of Miami user account, you may register for a CCS account through the CCS Portal. Your registration will be approved automatically and your CCS account will be created with the same user name as your CaneID. This website is only available within UM secure networks.

Pegasus Allocations

After registering an account with CCS, all University of Miami students and faculty are granted access to the Pegasus supercomputer with the following limits:

  • Students and Post-Doctoral Assistants: Up to 50,000 CPU hours per year (fiscal)
  • Faculty and full-time staff: Up to 100,000 CPU hours per year (fiscal)

All active users are allocated a HOME directory of 250GB. Pegasus cluster resources are assigned and managed on a project basis for collaborative work under /scratch. Users without project membership may request personal scratch allocations of 512GB from hpc@ccs.miami.edu. Users are encouraged to join projects for access to larger scratch spaces.

UM faculty members and principal investigators (PIs) may submit a Project Request in excess of the default amounts by providing justification to the allocations committee for the additional requested resources. Additional justifications are also required for experiments that will use more than 2TB of scratch space. The allocations committee requires start and end dates for all large resource requests. Project allocations are divided into different classes and have different justification requirements.

Be aware that project allocations are granted by fiscal year.  If you exceed your requested allocation, your running priority will drop.  You will be able to request one additional allocation within the same fiscal year.  Faculty members/PIs will need to coordinate among their groups and separately apply for allocation larger than the 50K and 100K levels, as noted above. CCS strongly encourages users to apply for allocations above the minimum levels that reflect a realistic expectation of computations required for a given project.  The allocation data are strictly for tracking purposes, not for determining appropriateness of requests.  CCS will make every effort to accommodate all users of the Pegasus system.

Disk allocations on Pegasus are only for data that is currently being processed. Data left after processing will be purged automatically. Information about your account, including CPU hours used, is available when you log into Pegasus.

Pegasus Usage

Job Submission

ALWAYS submit your production jobs to the cluster using LSF. DO NOT run production jobs on the head nodes. To learn how to use LSF visit our running jobs web page. You may compile and test your code on the head nodes. However, make sure your test case can finish within 30 minutes and is not resource-intensive. Any jobs exceeding these limits without prior approval from the Director of HPC will be killed and the user account will be disabled. All ssh sessions are closed automatically after 50 minutes of inactivity.

File Systems

Pegasus has two parallel file systems available to users:

  • /nethome (permanent, quota’d, not backed-up)
  • /scratch (high-speed purged storage)

NO backups are performed on Pegasus file systems. Files on /scratch are subject to purging after 21 days

All active users are allocated a logical storage area, HOME directory. Hard storage quotas are not currently implemented at the user level, but are generally limited to 250GB for HOME. Home directories are intended primarily for basic account information, source codes, and binaries.

/scratch directories should be used for compiles and run-time input and output files. HOME and scratch are available on all nodes of the cluster. Data in the /scratch directories are NOT backed up and untouched data older than 3 weeks are subject to purging. Data in HOME directory is also NOT backed up. However, it is not subject to purging unless the account is deleted.

Accounts exceeding the 250GB Home directory limit will be suspended. The user will not able to submit jobs. Once the usage is under 250GB, the account will be enabled in one business day. After 90 days, suspended accounts will be disabled. The user will not be able to login. At this point the user must submit a request to have their account temporarily enabled, so they can clean up their data. Once the usage is under 250GB, the account will be enabled in one business day. Disabled accounts will be deleted after 30 days. Their data will be archived to backup. The backup will be kept for 30 days.