Overview:  

Standby is a new storage offering, built as an alternative to Tier 2. Standby should be leveraged for infrequently accessed data that is still directly available for reference, retrieval or analysis.  Standby can act as an intermediary location; a space to organize and prepare research data for long-term retention.  

Standby can also be used to store large data sets, transferring portions of the data back and forth to the Active space for analysis.  Data stored in Standby does not need to be recalled or downloaded for access, the data is available immediately.  

Standby is accessible via desktop or transfer nodes on O2, providing a similar degree of data protection, including snapshots and off-site backup.  

 

How to Access: 

Multi-protocol Access 

Standby Storage can be provisioned in both NFS and SMB. Windows and MacOS clients typically access shared storage over SMB, whereas Linux hosts access Standby Storage via NFS. In some rare cases, Standby Storage can be made available in both NFS and SMB. 

  • Connect to remote desktop to mount research.files.med.harvard.edu and standby.files.med.harvard.edu  
  • Note: Transferring files while using VPN will be quite slow and can interfere with other users accessing HMS resources over the VPN.  
  • If a remote desktop is not available, please contact [email protected] to discuss further options.  

Access via:  

  • Windows: \\standby.files.med.harvard.edu  
  • OSX: smb://standby.files.med.harvard.edu/  
  • Connect to your lab’s folder(s)  

Standby is accessible via desktop or transfer nodes on O2. To access /n/standby on the O2 cluster, you must login to the transfer cluster at transfer.rc.hms.harvard.edu; /n/standby is not available from O2 login or compute nodes.

Eligibility  

HMS, HSDM, and HSPH quad-based faculty and staff are currently eligible to use Standby storage. Other HMS affiliates may also qualify for access. Please email [email protected] to discuss available storage options.  

 

Migration of Data from Active to Standby  

HMS Storage Services will be performing initial migrations to Standby for researchers who have previously identified Tier 2 or Tier 3 datasets.  

To migrate data from an existing Active folder to a Standby folder, we recommend you utilize one of these approved methods:  

  • rsync - Copies files either to or from a remote host, or locally on the current host  
    • (e.g. rsync -a /n/groups/lab/tier2 /n/standby/hms/dept/lab/) 
  • cp - Copies the contents of the source_file to the target_file. 
    • (e.g. cp -a /n/groups/lab/tier2 /n/standby/hms/dept/lab/) 
  • mv – Moves each file named by a source operand to a destination file in the existing directory named by the directory operand. 
    • (e.g. mv /n/groups/lab/tier2 /n/standby/hms/dept/lab/)

 

Directory Structure  

For new labs, a Standby storage folder will be created as part of the Active storage request, to encourage integration of Standby in your lab’s data workflow. If you belong to an established lab, you can request creation of a Standby folder by filling out the Standby Storage Form. Modification of the directory structure is available to all users of the directory. Researchers are advised to structure the folders to correspond to how the records were generated, to complement proposed or existing workflows. 

  

Permissions 

Standby folder permissions will mirror permissions found on Active lab folders. In general, Standby folders will be configured to allow access to all lab members, unless otherwise defined. This is designed to promote, as part of the general offboarding process, the migration of data to long-term storage. Please email [email protected] if you have questions regarding Standby folder permissions.  

 

Filesystem Quotas 

Filesystem quotas will be applied to all lab level directories. They enable HMS to appropriately scale and lifecycle stored research data. Defining and limiting filesystem sizes allows for adequate provisioning of standby capacity to the HMS community. 

 

Quota Notifications  

If your group reaches the maximum storage amount allowed by the storage quota, the responsible party (group leader or PI) will be notified via email. If your lab has identified a Lab Data Manager, they may also be informed. The quota email will be sent on a daily basis until the data storage amount drops below the limit or an indicated grace period has passed. If the data storage amount does not decrease, the ability to write to the directory will be restricted until storage usage is reduced. 

 

Request Storage:  

Allocation amounts dependent on lab needs and available resources.  

To request storage: Standby Storage Request Form

 

Contact:  

Research Data Management: [email protected]