The purpose of this study was to review the capabilities of IBM General Parallel File System (GPFS) as a file system for IBM BigInsights Hadoop deployments and to test the performance advantages of Mellanoxâ€™s Remote Direct Memory Access (RDMA) for BigInsights applications using GPFS. To provide a basis of comparison, tests were run comparing the use of GPFS with Apache Hadoop Distributed File System (HDFS). Benchmark results show GPFS improves application performance over HDFS by 35% on the analytics benchmark (Terasort benchmark), 35% on write tests and 50% on read tests using the Enhanced TestDFSIO benchmark. This paper provides details on the test architecture, methodology, results and conclusions reached during the study.
Author: Mellanox Technologies