Ensuring high availability and recoverability of acquired data

Every time one runs a shot, or simulation, exorbitant amounts of data are collected and sent off to live a life in storage. This data is important to our livelihood as a scientific research community, and to the goals of our mission of sustainable energy. Therefore it will behoove all to ensure the integrity of this data. Many mechanisms are available to store and ensure the availability of this data, from Hardware Raid, to Software Raid, and backups. Is the right amount of data redundancy being utilized in order to ensure data is safe? What are the scenarios in which these redundancies could fail? How can one ensure that each type of failure is accounted for with the least amount of overhead? When using Hardware Raid on the storage networks, each Raid group is allowed a certain number of failures, before the whole group fails beyond recovery. Software Raid, specifically ZFS raid-z or mirroring, can check for "soft errors," and provide a way to recover, even if a hard disk fails or a device is prematurely removed. Finally, backups are only as good as the policy and resources provided to the system. As with many engineering decisions, it is often not clear what the best solution is. Alone, each one of these mechanisms provides a certain level of data redundancy or availability. However, when one would combine these resources, it will ensure that no matter what scenario, data will be available and recoverable.