The purpose of this study was to review the capabilities of IBM General Parallel File System (GPFS) as a file system for IBM BigInsights Hadoop deployments and to test the performance advantages of Mellanox’s Remote Direct Memory Access (RDMA) for BigInsights applications using GPFS. To provide a basis of comparison, tests were run comparing the use of GPFS with Apache Hadoop Distributed File System (HDFS). Benchmark results show GPFS improves application performance over HDFS by 35% on the analytics benchmark (Terasort benchmark), 35% on write tests and 50% on read tests using the Enhanced TestDFSIO benchmark. This paper provides details on the test architecture, methodology, results and conclusions reached during the study.

Author: Mellanox Technologies

PDF: https://www.ibm.com/developerworks/community/wikis/form/anonymous/api/wiki/fa32927c-e904-49cc-a4cc-870bcc8e307c/page/f0cc9b82-a133-41b4-83fe-3f560e95b35a/attachment/49c31c28-b42e-45a0-a564-3f0aabda4ec4/media/Big%20Insights%20with%20Mellanox%20Infiniband%20RDMA%20-%20April%202014.pdf

Join The Discussion

Your email address will not be published. Required fields are marked *