Use an open-source tool to move large data to cloud object storage

Today, hybrid clouds enable data and computations to be partitioned across clouds. Hybrid cloud and private cloud users can archive their data or migrate their work to the public cloud, where cloud object storage is most common. This means users need an efficient tool to move large data from Hadoop Distributed File System to IBM Cloud Object Storage. This is where COSDistCp comes in. COSDistCp is based on the popular community tool DistCp. The DistCp tool is a traditional Hadoop application designed to work with file system operations, which are different in the object storage world. Because DistCp wasn’t designed to work with object storage, it suffers from performance inefficiencies that we managed to resolve in COSDistCp.