Overview

Skill Level: Any Skill Level

By combining Aspera with IBM Power and IBM cloud, customers have a practical way to transport massive data files to and from the cloud. Showcase the differences between Aspera FASP with that of the standard AIX scp command in transfer performance.

Ingredients

System requirements for IBM Aspera Enterprise Server

 

  • Product-specific Aspera license file.
  • SSH Server. Version 5.2 or higher is recommended and OpenSSH 4.4 or newer
  • Verify that the machine's host’s file has an entry for "127.0.0.1 localhost." Check /etc/hosts
  • Verify that SELINUX is disabled
  • If you have a local firewall on your server, then need to allow the UDP port (55001, by default) for multicast traffic
  • To standardize the port for use in Aspera transfers, Aspera recommends using TCP/33001 and turn off Port 22.
  • Secure SSH settings (Follow Aspera-Enterprise / Client Documentation)
  • Aspera transfer products use system accounts for connection authentication, and these accounts require additional configuration for Aspera transfers. You can specify user-based settings, such as bandwidth, document root (docroot) and file handling rules in aspera.conf (Aspera transfer product's configuration file) file located at /opt/aspera/etc/ aspera.conf. An example of aspera.conf in the following location at /opt/aspera/etc/samples/aspera-everything.conf
     

Step-by-step

  1. Introduction & Recipe Objective

    By combining Aspera with IBM Power and IBM cloud, customers have a practical way to transport massive data files to and from the cloud. Aspera makes cloud computing even faster, more predictable and more cost effective for big data transfers such as enterprise storage backup, sharing virtual images or bursting to the cloud for increased computing capacity.

    What is Aspera?

    Aspera solutions enable organizations to move, share and synchronize large files and data sets, digital assets and media quickly and securely. These highly scalable solutions are built to handle the largest data requirements at maximum speed, regardless of data size, type, and distance or network conditions.

    What are the benefits of Aspera?

    At the core of the Aspera solutions is the patented Aspera Fast, Adaptive, and Secure Protocol (FASP®), a breakthrough transfer protocol that uses existing wide area network (WAN) infrastructure and commodity hardware to achieve speeds up to hundreds of times faster than FTP and HTTP. FASP transport technology provides enterprise-grade security and exceptional reliability and bandwidth control.

     

    TCP Vs FASP

     TCP (Transmission Control Protocol) FASP (Fast, Adaptive and Secure Protocol) 
     

    It provides reliable data delivery under ideal conditions, but has an inherent throughput bottleneck that becomes obvious, and severe, with increased packet loss and latency found on long-distance WANs.

    Adding more bandwidth does not change the effective throughput. File transfer speeds do not improve and expensive bandwidth is underutilized.

    TCP’s security and monitoring are also insufficient for today’s business concerns about security of their digital assets.

    It uses the mechanism that identifies and retransmits precisely the real packet loss on the channel. At 10% packet loss, FASP achieves 90% bandwidth utilization with less than 1% redundant data overhead.

    The FASP protocol provides complete built-in security without compromising transfer speed. The security model, based solely on open standards cryptography, consists of SSH authentication, on-the-fly data encryption using strong cryptography (AES-128) for privacy of the transferred data, and integrity verification per data block, to safeguard against man-in-the-middle and anonymous UDP attacks.

     Aspera Enterprise Server

    For businesses that cannot afford the delays and risks of shipping physical media across the world, or the slow, unreliable and insecure FTP transfers, Aspera transfer servers such as Aspera Enterprise Server offers solutions that guarantees results regardless of transfer distances, file sizes or network conditions. Combining patented FASP® technology, optimized cloud I/O for cloud deployments, and comprehensive transfer and user management, our transfer servers move large data sets over the WAN with unrivaled speed (100X faster than FTP or HTTP), flexible bandwidth control, 100% reliability and bullet-proof security.

    Objective:

    Our aim was to compare Aspera FASP with that of the standard AIX scp command to see the differences in transfer performance and processor load on Power systems.  We wished to see if Aspera FASP could provide performance improvement not just in a WAN configuration but also in a LAN environment as well.  In addition we wanted to characterize how performance changed when multiple simultaneous transfers were taking place.

  2. Methodology

    This documents is focused on to showcases the performance gain when using the Aspera FASP protocol in environments with high round-trip  time, packet  loss and when large files are transferred over networks.  We have configured Aspera Enterprise Server and Aspera Client in two environments. Below are the details of two main server configurations.

    Configuration-1: Pair of servers within the same lab, local network connection.

     server1-1

    Network Configuration

    server1-2

    LPAR statistics

    server1-3

    Aspera License Cap on Server & Client: All Licensed Aspera Clients and Servers have a bandwidth capacity limit.

    server1-4

    Configuration-2: Pair of servers located in different locations (India and Austin TX)

     server2-1

    Network Configuration

    server2-2

    LPAR statistics

    server2-3

    Aspera License:

    server2-4

     

     

     

     

  3. Install Aspera Enterprise Server & Aspera Client

    Install Aspera Enterprise Server & Aspera Client. Run it as root using the following command:

    bash aspera-entsrv-3.6.0.120993-aix-6.1-ppc32.sh

    bash aspera-scp-client-3.6.0.120993-aix-6.1-ppc32.sh

    step3

    Things to be considered for using Aspera FASP transfer solution with ascp cli are as below:

    •  Aspera ascp is a fasp transfer program that is used to transfer data at the rate that fulfills the maximum bandwidth capacity.  The transfer policy specified along with maximum and minimum transfer, determines the utilization of network resource by fasp transfer.
    • Aspera Target Rate: Aspera Clients always request a target rate. Aspera servers will either allow the client to transfer at the rate requested, or will throttle the client to a rate which is set by the server.

                Ex: ascp -P 33001 -QT -l 1000M -m 1M root@destination_host:tmp/1GB /tmp

                      -T:    Disable Encryption

                      -P:    Port used to Transfer

                      -l:    Set the target transfer rate (1000Mbps)

                    -m:    Set the minimum transfer rate (1Mbps)

    •  Currently the ascp process is not multithreaded, so if you have multiple concurrent transfers, it is important to have multiple CPU or multi core system, so you can spread the CPU load.
    • To improve the transfer speed, you need to look into components- physical bandwidth, CPU and I/O resources. The bandwidth is one of the most common limitations on transfer performance; CPU can be a bottleneck, especially when encryption is enabled. The I/O limitation is more common with high concurrency or high packet loss.
    • The network bandwidth capacity can be determined by isolating it from other performance limiting factors using the ascp cli options: –no-write &  –no-read. These options simulate reading and writing. In addition to this, you can enable or disable encryption to see the impact of it on bandwidth.

             Ex: ascp -QT -l 500M –no-read –no-write bigfile user@destination:/
     

    §  The –-no-read –-no-write testing is only for the network and not the storage

    §  To test the CPU speed of the end systems (enabled encryption – remove “T”)

    §  To test the endpoints’ write speed, you the give option –-no-write

    §  To test the receiving side’s ability to write to storage, you mention –-no-read

  4. Test Cases

    We performed data transfer testing with standard SCP and Aspera FASP transfer solution on IBM Power Systems located in same lab where RTT: 0ms and in environment that has high RTT: 262ms where the IBM Power systems are located at Austin and India. The tests show that there is a significant higher throughput achieved when using Aspera FASP than with standard SCP transfers. List of Test Cases performed in the two server configurations:

    Case 1: Transfer speed of large files (10GB, 5GB, 500MB, 200MB) using Aspera FASP and SCP

    Case 2: Performance test runs to test the bottleneck using ascp (without reading and writing to disk)

    Case 3: To test the parallel functioning of TCP and FASP over the same network.

    • Case 4: Test with Aspera Demo Server
  5. Results

    Test Case 1: Transfer of data between pair of servers within the same lab

    To test the download transfer rate, data of 39GB that comprises of various files of size -100MB, 200MB, 500MB, 1GB, 5GB and 10GB; was downloaded from Aspera Enterprise Server to Aspera Client using the fasp program transfer: ascp cli. In this case, the servers are in same lab that are of 1Gbps network with RTT: 0ms.

    C1-1

    C1-2

        Observations: It is observed that Aspera transfer rate was 700Mbps and the download completed in 9.4min, whereas using the secure copy SCP, it took 22min to complete the download.

     

    Test Case 2: Transfer of data between pair of servers located in India (Aspera Client) and Austin (Aspera Enterprise Server)

    To test the download transfer rate, small data set of 28GB that comprises of various files of size -100MB, 200MB, 500MB, 1GB, 5GB and 10GB and large data set of 500GB; was downloaded from Aspera Enterprise Server(Austin-TX) to Aspera Client(India) using the fasp program transfer: ascp cli. In this case, the servers are dispersed geographically and are of 1Gbps network with RTT: 262ms.

    C2-1

    C2-2

    Observations/Recommendations:

    • With Aspera transfer, it took 12 min to transfer small data set of 28GB and for large data set of 500GB, it took 3 hours to complete, at a rate of 350Mbps – 450Mbps, whereas with standard SCP, at transfer rate 400KB/s it took almost 14hrs for download a 28GB file.
    • The bandwidth is too low than that of the target rate as the transfer was over intranet bound with firewall. As firewall implements QOS (Quality of Service) and performs deep packet inspection (IDS/IPS), this is likely to cause slow transfers.
    • As an Aspera best practice, disable the power saving feature to ensure it does not interfere with optimal performance on Aspera Client and Enterprise Server.

     

    Test Case3: Performance test for transfer data at target rate 1000Mbps

    The Aspera transfer options (–no-read –no-write), eliminates either the reading from or writing to disk, as part of a transfer.

    C3-1

    Observations/Recommendations

    • The Aspera transfer options –no-read –no-write shows that the network capacity, independent of storage throughput. (Note: Encryption is disabled). If the transfer rate is less than the target rate, then the transfer is either limited by network congestion, or endpoint CPU speed.

                         §  Pair of servers within the same lab: Transfer rate achieved is 800Mbps with no packet loss.

                         §  Pair of servers located at India & Austin: Transfer rate achieved is 414Mbps

    • With encryption enabled, if the transfer rate is noticeably slower than that attained with disabled encryption, then it is for sure that the CPU will be a bottleneck with encrypted transfers.

                         §  Pair of servers within the same lab: Transfer rate is 280Mbps

                         §  Pair of servers located at India & Austin: 340Mbps

    • Enabling write allows testing the receiving side’s ability to write to storage. If the transfer rate is less than the target rate 1000Mbps, then it could be due to storage write speed limitation, which needs to be improving with some tuning.

                         §  Pair of servers within the same lab: Transfer rate achieved is 650Mbps

                         §  Pair of servers located at India & Austin: 320 Mbps

    • Enabling reading allows you to test the endpoints’ read/write speed.  If the transfer rate is close to the target rate (1000Mbps), then there is no sender side storage limitation. But, if the transfer rate is considerably slower, then you do have a sender side storage limitation, which may be responsive to tuning.

                        §  Pair of servers within the same lab: Transfer rate is 840Mbps -870Mbps

                        §  Pair of servers located at India & Austin: 410Mbps

     

    Test Case 4:  Parallel transfer

     This is to test the parallel functioning of FASP, where multiple ascp processes runs and with TCP, where scp and ascp traffic compete each other.

    C4-1

    C4-2

    Observations/Recommendations

    • Aspera FASP works well in mixed environments where standard scp and ascp run parallel over the same network.
    • With multiple ascp processes, it is observed that the CPU utilization on Aspera Client and Enterprise Server is about 50% and the transfer rate achieved is about 425Mbps in case of servers in the same lab, whereas the same in the servers located at different locations, it is observed that the CPU utilization is 20-40% and the transfer rate is 270Mbps.
    •  The same in case of multiple standard scp processes, it is observed that the ssh and scp processes together consume 70% of CPU and the transfer rate achieved is 21MB/s on the servers within the same lab. In case of pair of servers located at India and Austin, the CPU utilization is 2-4% with transfer rate 430KB/s.
    • In case of cross traffic where ascp and ssh and scp process run, it is observed that in case of servers in same lab, the CPU utilization is 65% with transfer rate 500Mbps for ascp and 20MB/s for SCP, whereas in the case of servers located at different locations, the CPU utilization is about 17-25% with transfer rate of 315Mbps with aspera and 450KB/s with standard scp.
    •  In case of parallel transfer on a dual-core system, you can configure to use multiple UDP ports say: 33001 & 33002 along with transfer rate(ensure that together the transfer rate is less or equal to the target rate). The two commands need to be executed in different terminal windows:

                       > ascp -C 1:2 -O 33001 -l 100m /file root@10.0.0.2:/remote-dir

                       > ascp -C 2:2 -O 33002 -l 100m /file root@10.0.0.2:/remote-dir

     

    Test Case 5: Download & upload Test from Aspera Demo Server: To make sure the software is working properly download and upload transfers between your system and the Aspera Demo Server.

     https://demo.asperasoft.com/aspera/user/

    Username / Password: aspera/demoaspera

     Tested downloading a 10GB file from demo.asperasoft.com to Aspera Enterprise Server RTT is 69ms – 71ms. It took ~2min to download 10GB file.

    Ex: ascp -P 33001 -QT -l 1000M -m 1M aspera@demo.asperasoft.com:aspera-test-dir-large/10GB /tmp

     

  6. Troubleshooting

    • The general syntax for testing ascp connectivity is the following:

                    [ASPERA_SCP_PASS=remote_password] ascp -wf -P ssh_port remote_user@remote_host:/

    • aspera processes do log with syslog and facility local2. To redirect aspera logs to a file, configure syslog accordingly.

                    echo ‘local2.info        /var/log/aspera.log’ >> /etc/syslog.conf (give tab instead of space)
                    touch /var/log/aspera.log
                    refresh -s syslogd

  7. Conclusion

    IBM® Aspera Enterprise Server is an innovator in moving the world’s secure data at maximum speed. It delivers maximum transfer speed and optimal bandwidth utilization—regardless of file/data set size, network condition, or deployment model. It overcomes the limitations of TCP and reduces transmission times. In case of multiple concurrent transfers, CPU can be a bottleneck, especially when encryption is enabled, but with Aspera configured Power System, you can spread the CPU load.

     

     

     

    Contributors:

    Andrew Wack – An architect and expert for IBM System p Cluster Testing. He played a key role in identifying appropriate test techniques, tools and guidelines to enable better test coverage and resources requirements for the test effort. Andrew can be reached at awack@us.ibm.com

    Leena Kushwaha – An Openstack SME and storage consultant. Involved in evaluating, establishing and conducting proof of concepts for Hybrid Cloud Solutions. Leena can be reached at leena.kushwaha@in.ibm.com

1 comment on"High Speed Data Transfer with IBM Aspera on IBM Power"

Join The Discussion