There might be situations where customer scenario would require sharing of tiered block storage volumes of varied IOPS to be shared across computes in different availability zones. This article leverages SSHFS to overcome this limitation using a very basic service of SSH which is availabile across operating systems flavours. This recipe is specific to IBM Cloud VPC network construct.
Note: NFS offering in VPC is going to be GAed soon, but one can setup NFS servers and clients and manage them. In this article we are using sshfs to complement NFS to share the volume across VSI located in different AZ.
This recipe is based on Implementing below architecture where in block storage for VPC is mounted on virtual server based out of Dallas 1 zone. This volume is then mounted on virtual servers in zones – Dallas2 and Dallas3. So whatever data is written to Dallas1 volume is replicated to Dallas2 and Dallas3. I will take virtual server hosted in Dallas1 data center as primary server or master, where as servers hosted on Dallas2 and Dallas3 as secondary servers or Slaves.
To Implement this scenario one would need to establish passwordless authentication between secondary servers and primary server or master as shown in below diagram. IBM Cloud VPC comes with private key authentication mechanism by default and one can’t establish passwordless authentication that. Hence one would need to change SSH configuration from private key authentication to password based authentication. Install fuse-sshfs library on secondary servers by running below command :
Reboot the machine once after installing fuse-sshfs as many times it throws below error during mounting:
“Transport endpoint is not connected“
or one can manually download the latest verion:
The details of virtual Instances are mapped across availability zones as below:
Attach Block Storage Volume to Primary Server
I tested this solution with 100G storage attached to vs1-az1-instance.
Verify in Instance that volume is attached.
Create Volume on Attached Storage
Verify that disk is mounted in OS.
Create Volume that we will mount on other two Instances spread across zones.
Set xfs file system on the partition.
Mount the partition to Posix compliant directory.
Make entry in /etc/fstab to make mount permanent.
Reboot the virtual server.
Passwordless Authentication Between Master Server and Slaves Servers
Make entry in /etc/hosts of virtual servers in az2 and az3 for virtual server in az1.
Enable passwordless authentication between virtual server on az2, az3 – Secondary and that of az1 – Primary.
This establishes the passwordless SSH authentication between secondary and primary server.
Mount Remote File System of Primary Instance on Secondary Instace
We have now reached the stage where we need to mount volume that we provisioned in Primary instance in availability zone 1 named as vs1-az1-instance on storage of Secondary instances – vs1-az2-instance and vs1-az3-instance.
The command that I will use for mounting is –
sshfs root@vs1-az1-instance:/shareddata <<Mount_Point>>
where mount points are :
a) /root/vs1-az2-mount – Availability Zone 2
b) /root/vs1-az3-mount – Availability Zone 3
Make mount permanent by making entry in /etc/fstab
Note: To auto-remount on primary server reboot make below entries in your /etc/fstab.
root@vs1-az1-instance:/shareddata /root/vs1-az2-mount fuse.sshfs allow_other,_netdev,noatime,reconnect,auto 0 0
Note: If one wants to use private key of primary host to mount the file system, he can leverage below string. IdentityFie reference is what needs to be given.
sshfs#user@server:/remote/folder /local/mount/dir fuse IdentityFile=sshkeyfile,Port=XXX,uid=1000,gid=1000,allow_other,_netdev,ServerAliveInterval=45,ServerAliveCountMax=2,reconnect,noatime,auto 0 0
Which if not there would not remount the volume in case of primary server reboot and one would need to reboot secondary servers to make that happen.
Reboot machine and verify if mount still persists.
We are now ready for testing the solution.
Testing the Solution
The virtual servers and storage that we configured above are as below:
Using SSHFS is a no cost solution to synchronize the data across virtual servers located in different availability zones.