Using Transparent cloud tiering, you can migrate files in both co-resident and non-resident states depending on file heat (how often a file is accessed or modified).
Normally, when a file is migrated to the cloud storage tier, the status of the file becomes non-resident. This means that, an application must completely recall the file data from the cloud to be able to perform any read or write operation on that file. For performance reasons, this is an issue when the data is warm. It is also a cost issue on public clouds since there is a high fee for reading data externally from cloud services. Also, the calling application must recall the file every time it needs to perform any operation on the file, and this can be resource-intensive. The solution to this issue is to pre-migrate files, leaving them in the co-resident state.
In pre-migration, irrespective of the file size, the files are archived to cloud storage in the co-resident state such that the file is still present on the file system cluster as well as having a copy of the data on cloud storage. This allows file system applications to have continued access to the files without issuing any recall commands, but at the same time, ensures that data is at least available on the cloud if there is any type of disaster that causes loss of the Spectrum Scale cluster. As data gets colder, the files are migrated in the non-resident state.
You can use the following command to migrate files in the co-resident state:
mmcloudgateway files migrate --co-resident-state file1
To verify that the file is migrated in the co-resident state, issue the following command:
mmcloudgateway files list file1
The system displays an output similar to this:
File name : /gpfs0/file1
On-line size : 46
Used blocks : 0
Data Version : 1
Meta Version : 1
State : Co-resident
Base Name : 57FA294111831B2B.10D9F57158C1628B.0063C1580F522F09.0000000000000000.5713748E.0000000000009E2B
Typically, a policy is used for co-migration of data to the cloud using the mmchpolicy command. A sample policy is available here:
- As a Spectrum Scale administrator, I want an archive copy of my stable data (not likely to be deleted or changed) external to my cluster on object storage so that if my cluster fails I have DR archive access to the data.
- As a Spectrum Scale administrator, I want an archive copy of my stable data (not likely to be deleted or changed) external to my cluster on object storage so that if my data is accidentally deleted or incorrectly modified, I can restore the data from Object Storage via the Transparent cloud tiering service.