The purpose of this study was to review the capabilities of IBM General Parallel File System (GPFS) as a file system for IBM BigInsights Hadoop deployments and to test the performance advantages of Mellanox’s Remote Direct Memory Access (RDMA) for BigInsights applications using GPFS. To provide a basis of comparison, tests were run comparing the use of GPFS with Apache Hadoop Distributed File System (HDFS). Benchmark results show GPFS improves application performance over HDFS by 35% on the analytics benchmark (Terasort benchmark), 35% on write tests and 50% on read tests using the Enhanced TestDFSIO benchmark. This paper provides details on the test architecture, methodology, results and conclusions reached during the study.

Author: Mellanox Technologies


Join The Discussion

Your email address will not be published. Required fields are marked *