Users should avoid storing data on local HPC systems, such as individual compute nodes or servers. User data generally belongs on network file systems (GPFS), with data protection and snapshots to protect data. This includes /home, /juno (including juno/res), /lila (including /data), and /oscar/warm.
Planned and unplanned maintenance can require reinstalling the operating system on any individual system, and may also destroy local data. /scratch filesystems are normally undisturbed, but we cannot guarantee this. Important data must not be stored only on server/node local drives.
The HPC Team performs daily backups of local drives in HPC servers, but does not keep snapshots or archive older data. Before routine maintenance, we refresh our backups in case of unexpected problems, but this is on a best effort basis.
All systems in the lilac cluster (including lilac.mskcc.org and lab login nodes) may be reinstalled and reimaged at any time as part of routine or emergency maintenance. We will notify users (at least via #hpc_mskcc Slack channel) in advance for planned work.
We will notify the CMO and BIC before planned reinstallation of juno servers (at least via #hpc_mskcc Slack channel).