The Cloud core is focused on providing a flexible SaaS (Storage as a Service) and PaaS (Platform as a Service) architecture to UM researchers. At over 7 PB, the W.A.D.E. storage cloud is one of the largest data provisioning clouds in academic research.
A high-speed InfiniBand network connecting storage backed by a state-of-the-art file system supports IO-intensive workloads that can be scaled out across thousands of cpus. This flexible cloud architecture allows UM faculty and students to solve complex problems using applications that require high bandwidth, high-speed data storage and networking, and a high number of cpu cycles.
The Advanced Computing group is continuously working to increase the ability to scale out services in a clustered architecture while balancing the need for the increase in I/O operations, throughput, and capacity.
Current efforts include:
- use of the latest parallel filesystem software such as SpectrumScale which provides concurrent high-speed file access to applications executing on multiple nodes in the system
- collection and analysis of filesystem statistics and constant monitoring and tuning of the filesystems in response to workloads
- use of object storage technology, in which files are organized as objects in a flat directory rather than a traditional tree hierarchy
This can boost performance for large collections of files since the system no longer needs to navigate a tree like structure to locate data but instead uses indexed data often held entirely in memory.
- implementation of a tiered storage model in which data can be stored according to how it is accessed
This allows the data required by running applications which must be accessed with very low latency and high throughput to be stored differently than archival data which is accessed infrequently.
- deployment of multi-protocol gateways (NFS, CIFS, sftp, Globus) to allow students and faculty convenient access to data