Storage
To select the storage a system, one must first specify the requirements in terms of capacity and storage read/write performance.
Based on your size requirements, a choice will have to be made in terms of technology:
- Spinning disks
will give you a large capacity in TBs for very low cost, but have a high latency in their access times.
- Solid State Drives
have a lower latency, but usually lower capacity and at a higher price
- NVMe SSDs
will give you the best access times but with the lowest capacities at the highest price
MTBF: Mean Time Between Failures
Enterprise storage devices usually have a much higher mean time between failures (MTBF) compared to consumer products. This is a measure of how often a device will fail under normal operational load.
Because of this, it makes sense to store critical information on products with a higher lifetime. If the information is not critical, money could be saved for these parts.
Enterprise HDD |
Desktop HDD |
|
|---|---|---|
MTBF |
2.5 million hours |
1.2 million hours |
Storage in a System
At the minimum, each system will require a system disk for an operating system. Since compute nodes as usually seen as replaceable, it doesn’t make sense to spend much money on the system disks.
However, a better storage on each node will be needed to support scratch file systems and burst buffers for calculations, which are much more demanding.
Finally, larger storage pools can be created with multiple disks and even a mixture of different technologies in order to host user data and data sets. Typically these larger pools aren’t on all cluster systems, but on dedicated storage systems.
Storage Redundancy: RAID
For critical data a choice will have to be made about data redundancy.
This is because as a storage system grows and the number of devices increases, the probability of an individual drive failure increases.
The usual way to handle this is to introduce redundancy into a storage system by using RAID (redundant array of independent disks). RAID can be implemented both in software and in hardware. There are multiple levels:
- RAID 0
only combines multiple disks into one larger one without any redundancy. This increase throughput, since it aggregates the I/O performance of all disks.
- RAID 1
mirroring of disks, so everything is written twice. If a disk fails, the second disk still holds all the data.
- RAID 10
combines multiple RAID1 arrays to one large disk using RAID0. This gives you basic redundancy with aggregated performance due to multiple storage pools.
- RAID6
this requires at least 4 disks. Because of how the data is written, it reduces your total capacity a bit, but tolerates the failure of up to two disks, while still having all the data.
- RAID60
combines multiple RAID6 arrays to one large disk using RAID0. This gives you redundancy with aggregated performance due to multiple storage pools.
Warning
Communicate to your users that RAID is not a backup! RAID does not protect them from a catastrophic failure. So if you need to absolutely make certain data is safe, you will have to implement a backup strategy with multiple systems, possible with an offsite backup.