page-brochureware.php

Porting x86 Linux applications to IBM POWER
Step 2: Plan for the port

Many packages are trivial to port to POWER, often requiring a simple recompile or make command. However, if you tried building your package and found that it wasn’t trivial, don’t worry, this guide will help you work through most any porting situation.

The first, and possibly most important step, is to assess your environment and build a plan that addresses your specific requirements. After you review the information and recommendations in this topic you should have a solid plan for porting your application to the POWER platform.

Evaluate your application and it’s dependencies

Generally speaking, porting an application to POWER is a straight forward process and depending on what language it’s written in, the application may “just run” or at most require a recompile to run. Answer these questions to get started:

1. What language is your application written in?

Many of the most commonly used programming languages have already been ported to the POWER platform and ship with the distributions, while some are maintained by the community. This should simplify things as you prepare your own development environment. The list of programming languages available on the platform is long and continuously changing. If you’re not sure whether your language is supported, check OSPAT to find out.

My application is written in an interpreted or scripting language

Effort required to port: Test only

Most applications written in Java, which is designed to be cross-platform portable, or in interpreted languages such as, Python and Perl, are easy to port because they rely on a specific program (the language interpreter) to make that same program run on different architectures. This means that most x86/Linux applications with no platform specific dependencies will run as is on Power.

Here’s some general information about a few of the most popular programming languages on the platform. If your application is written in a different language, check to see if it’s been ported to Power by using the Open Source Availability Tool (OSPAT) search engine.


Java

Many open source projects are built with OpenJDK and it is usually a safe choice for your development needs. OpenJDK is available from the distributions. You’ll find install instructions here: http://openjdk.java.net/install/

Make sure to download and install the latest IBM Java SDK from here: https://developer.ibm.com/javasdk/downloads/


Node.js

Node.js is available and kept current here: https://nodejs.org/en/download/. You can also install a Docker container from the same page. IBM has tested almost all NPMs on the platform:

  • Automated testing, using each tool’s own supplied tests.
  • Found some errors with NPMs pulling down Intel based binaries, but generally there were very few problems found.

Python

Python versions 2 and 3 are available from the distributions. Python 2 continues to be provided for backwards compatibility and Python 3 should be the new standard.

There is also a miniconda environment for ppc64le available here: https://repo.continuum.io/miniconda/Miniconda2-4.3.14-Linux-ppc64le.sh

A repository of ppc64le packages ported to the miniconda environment can be found here: https://repo.continuum.io/pkgs/

My application is written in a compiled language

Effort required to port: Recompile and test

C, C++, Go, and FORTRAN are examples of languages that are compiled to instructions for a specific machine/platform. The C language in particular can expose more of the underlying machine architecture to the program and is thus, slightly less portable. That said, most applications written in a compiled language with no platform dependencies will only require a recompile to run on Power. The “Prepare your development environment” section lists some development tools that we recommend to help ease the process.

2. Does your application require a database?

If your application requires a database or has dependencies, you should verify that they have been ported to Power. The best way to do this is to search for it by using OSPAT.

**Note** In a hybrid IT environment, it is possible to continue to use your existing database server while you port your applications in stages – even if the database is not supported on the Power platform. However, if (or when) you’re ready to migrate to a database supported on the Power platform, consider these recommendations, based on the type of database your application requires:

NoSQL databases

POWER processor-based systems were designed for various big data and analytics workloads by providing 4X more threads per core, memory bandwidth, and cache than other platform options. These benefits translate into superior performance gains for NoSQL solutions and make it an ideal solution for managing your big data workloads on Power Systems running Linux.

MongoDB Enterprise Server

MongoDB is a NoSQL document store designed for unstructured or semi-structured big data. Run it on IBM POWER and you gain a high-performance DBaaS platform that delivers an integrated, real-time view of all your data – throughout your enterprise.


Neo4j

Neo4j is an industry leading open source graph database, that when run on IBM POWER, resets the scalability benchmark for what real-time graph processing can deliver your data.


Redis

Redis NoSQL open source data engine running on Power Systems facilitates developer processes, delivering next-generation applications with near-zero latency, high availability and seamless scalability to meet user expectations.


ScyllaDB

Scylla is an open source NoSQL database that is a drop-in alternative to Apache Cassandra that leverages the best features of Cassandra while providing 10 times better performance than Cassandra and low levels of latency. Scylla’s highly scalable architecture enables it to handle one million database transactions per second on a single server.


Cassandra

Apache Cassandra is a free and open-source distributed NoSQL database management system designed to handle large amounts of data across many commodity servers, providing high availability with no single point of failure. Learn more:

Relational databases

Switch to an open source relational database that runs on the Power platform and you’ll achieve improved price performance, while leveraging open innovation and exceeding the capabilities of your currently installed system.

EnterpriseDB (EDB) Postgres Advanced Server

EDB Postgres Advanced Server is an open source object relational database for enterprise data management. Run it on IBM POWER LC servers for big data and gain a 1.8x price-performance advantage guarantee over x86 servers.


Kinetica

Kinetica is a GPU-accelerated database for real-time analysis of large and streaming datasets. Kinetica on POWER with NVIDIA NVLink Technology and Tesla P100 GPUs delivers 2.4x more Kinetica queries per hour than accelerated x86 solutions.

Data frameworks

Hortonworks Data Platform (HDP)

100% open source Apache Hadoop and Spark on IBM Power Systems built with OpenPOWER technology. Hortonworks Data Platform on OpenPOWER LC delivers 1.7X Hadoop workload performance compared to x86.

3. Does your application have software dependencies?

If your application has prerequisite or dependent packages, you’ll need to make sure they are available on the platform and Linux distribution you plan to use.

Which Linux distribution(s) will you install?

IBM, in partnership with the major distributions, Ubuntu, Red Hat, and SUSE, provides a complete support solution for implementing and managing the Linux operating system for your enterprise server and cloud environments. Learn more

In addition, many of the most popular community distributions also support the platform, including Fedora, OpenSUSE, Debian, and CentOS. The right distribution for your porting project will be the one that addresses your most critical requirements.

One of the biggest distinctions between distributions is whether they focus on technology currency or long term stability and support. If you are developing cloud, cognitive, blockchain, or AI workloads that use the latest technologies, you may choose a distribution that focuses on currency more than long term stability. However, if you are dependent on key applications which are infrequently updated and tested against well known, stable distributions, you may choose a distribution that focuses on long term stability instead. It’s important to note that all distributions attempt to maintain some balance between currency and long term stability, though. Here are some recommendations:

I need a distribution that provides the latest and greatest technology

Ubuntu, Debian, and Fedora typically have frequent releases that integrate the latest open source projects. They are good choices for initial development in cloud, containers, cognitive, HPC, and other areas where the latest technology is absolutely necessary. Ubuntu is also very focused on distribution availability in the Cloud – with its free licensing model enabling quick and easy spin-up of workloads on any of the major cloud platforms (see http://www.canonical.com/partners for more details).

I need a distribution that provides the best support

RHEL, SLES, and Ubuntu, in partnership with IBM, provide long term distribution support and are common targets for applications that expect long term stability. The Red Hat distribution is further enhanced by the Extra Packages for Enterprise Linux (EPEL) repository, Red Hat OpenStack Platform, other add-on technologies (see http://www.redhat.com/en/technologies for other options), as well as Fedora for the very latest packages.

Distributions like Ubuntu (and its many variants, such as kubuntu), CentOS, and OpenSUSe provide community supported distributions with much of the same application support as their enterprise versions but without the corresponding support costs and without a known champion to track your defects or preemptively repair them.

Is your development environment setup for the port?

You most likely already have a development environment in place that includes the tools and processes that work for you and your particular situation. However, if you’re looking for some suggestions specific to porting and developing on the Power platform, the following recommendations may help.

Development tools recommendations

IBM Advance Toolchain
Open source development tools and runtime libraries that allow you to take advantage of the latest POWER hardware features for Linux. This self-contained toolchain does not rely on the system toolchain and requires minimum dependencies. Nor does it override the default Linux distribution toolchain (if it’s installed in /opt).

Note that when an application is built with the Advance Toolchain, it then has a dependency on the Advance Toolchain runtime. So, if that application is to be deployed elsewhere, the Advance Toolchain runtime package must be installed there as well. Because the runtime is free, this is not of significant concern, but something of which to be aware.


IBM Software Development Kit for Linux on Power
The SDK for Linux on Power includes a diverse set of tools that enhance the porting, optimization, analysis, and tuning of C/C++ applications and libraries for Linux on Power. Get more information about the various download options here: https://developer.ibm.com/linuxonpower/sdk-download/ or get the source code from GitHub: https://github.com/open-power-sdk

Some of the available tools, which will help you with your port (and beyond) and can be individually installed from GitHub, are described below:

  • Migration Advisor: Some issues that are identifiable by code inspection can be automatically identified with the Migration Advisor — consider running this tool as a very early step in your porting effort. Get the code: https://github.com/open-power-sdk/migration-advisor
  • Source Code Advisor: Uses FDPR (described below) to analyze a representative performance scenario and report performance issues which could not be identified during compilation. Get the code: https://github.com/open-power-sdk/source-code-advisor
  • CPI Breakdown: Profiles a representative performance scenario and report a hierarchical set of information about where the program is spending its time. Get the code: https://github.com/open-power-sdk/cpi-breakdown
  • IBM Feedback Directed Program Restructuring (FDPR): An offering from IBM Research that optimizes an existing binary without the need for source code changes, recompiling, or relinking. This is also known as post-link optimization. There are open source wrapper scripts in the fdpr_wrap package. Both FDPR and fdpr_wrap packages can be found at: https://developer.ibm.com/linuxonpower/sdk-packages/

Learn more about the SDK for Linux on Power, find links to downloads, and get tips and best practices:

Integrated Development Environment (IDE) recommendations

Eclipse
Eclipse is one of the most widely used IDEs. It contains a base workspace and an extensible plug-in system for customizing your development environment. Learn more here: http://www.eclipse.org/.


Kubernetes
Kubernetes is an open source system for managing containerized applications across multiple hosts. It provides basic mechanisms for deploying, maintaining, and scaling applications. Learn more here: https://github.com/kubernetes/kubernetes

Continuous Integration (CI) framework recommendations

Jenkins CI
Jenkins is a free, open source automation server that provides hundreds of plugins to support building, deploying and automating your open source project. Learn more about Jenkins and download it from here: http://jenkins-ci.org

  • Recommendation for open source projects
    The POWER CI is a Jenkins service managed by the Oregon State University (OSU) – Open Source Lab (OSL) and is intended to give developers easy access to the POWER architecture for building and managing their open source projects via Jenkins.
  • Recommendation for ISV and enterprise projects
    The Continuous Integration/Continuous Development (CICD) is a Jenkins service managed by IBM and is intended to give ISV developers access to the POWER architecture for building and managing their open source projects via Jenkins.

Travis CI
Travis CI will be supported on the platform soon.

Library recommendations

In addition to the typical development libraries such as libc, libm, libpthread, Zlib, OpenSSL, Boost, Tcmalloc, Intel TBB, and SPHDE (which are included in the latest release of IBM Advance Toolchain), consider these math libraries that will boost the performance of your HPC and deep learning applications.

ESSL
Engineering and Scientific Subroutine Library (ESSL) is a collection of high performance mathematical subroutines providing a wide range of functions for many common scientific and engineering applications. Languages: Fortran, C and C++ serial, SMP and SPMD. Download ESSL


libblas
The Basic Linear Algebra Subprograms (BLAS) are libraries used for optimizing mathematical computation on POWER8.


MASS
Mathematical Acceleration Subsystem (MASS) for Linux consists of libraries of mathematical intrinsic functions tuned specifically for optimum performance on POWER architectures. Languages: C, C++ and Fortran. Download MASS


Here’s a good blog that describes how to deploy and optimize these libraries on Power: Building Optimized Libraries for Deep Learning on OpenPOWER Linux Systems

Compiler recommendations

Many of the most popular compilers are available and optimized for POWER, including the following:

GNU Compiler Collection (GCC)

GCC is available from the distributions (and is also integrated into the the IBM Advance Toolchain for Linux on Power if you choose to install it). Install GCC from your distribution by using apt-get install or yum install


Clang

Clang is available from the distributions. Install it using apt-get install or yum install


XL C/C++ for Linux

There are three ways to get the community version of XL C/C++ for Linux, which is a no-charge, fully functional product for developers who do not require official IBM support.

Purchase and download the fully supported version of XL C/C++ for Linux from the IBM Marketplace.


XL Fortran for Linux

XL Fortran for Linux Community Edition (on little endian distributions) is a no-charge, fully functional product for developers who do not require official IBM support. You can get it three ways:

Purchase and download the fully supported version of XL Fortran for Linux from the IBM Marketplace.


Compiler optimization tips

Performance profiling tools recommendations

Use the following suite of tools to identify performance problems in your application and to understand how your application interacts with the Linux kernel.

OProfile
The OProfile tool profiles code that is based on hardware-related events, such as cache misses or processor cycles. For example, OProfile can help you determine the source routines that cause the most cache misses. OProfile uses hardware performance counters provided in many processors, including IBM POWER8. Learn more about OProfile.


perf
Perf is a powerful performance analysis tool with two different components: a userspace tool and a kernel infrastructure. You can install it using the yum install perf command on Fedora/RHEL or the apt-get install linux-tools-common command on Ubuntu. Learn more:


Valgrind
Valgrind is an suite of profiling tools used for detecting memory leaks, memory debugging, and performing detailed profiling to find blockages in programs. Learn more:

Do you have test cases and tools prepared?

Creating of a viable test plan is critical to the success of any porting project. You’ll test during various stages of the port and, of course, your plan will be tailored to the specific requirements of your project.

Most applications come with their own unit and functional tests. Some of those require configuration, such as setting up multiple servers or applications. Some require additional tools, such as jMeter, Selenium, Electric Fence (eFence), or insure++, for example. Check OSPAT to see if your favorite is available on the platform.

Do you have access to Power hardware?

If you already have access to Power hardware, great, you should be ready to begin the port. If not, don’t worry, we’ve got you covered.

There are Power cloud resources available throughout the world

Go to the cloud resources page, to quickly and easily find regional and global sites that offer access to Power development tools and platforms. Choose any of the available cloud resources based on your region and type of access, instance, accelerator and OS required.

Recommendations

I need short-term, no-cost access to Power hardware

I need long-term access to Power hardware

  • IBM Hardware Mall
    IBM Power Systems and IBM System Storage products are made available for your use in developing, evaluating, porting, migrating, optimizing, and or demonstrating your programs and solutions.
  • Nimbix
    The Nimbix Jarvice Cloud platform features ready to deploy instances for deep learning and NVLink application deployment and development. Request access >

I want try the port through a simulator

  • Install the IBM Software Developer Kit (SDK) for Linux on Power on your (Linux-based) desktop and then run your Power binaries on the same desktop using emulation technologies. Both QEMU (user-mode emulation) and the IBM Power Functional Simulator (full system simulation) are provided with the SDK. In this mode, you can develop and port applications without the need for Power hardware.
  • Provision ppc64le Docker-Machine on Intel. Learn about using the Docker Machine tool to provision a Power (ppc64le) Docker environment on any Intel (x86_64) virtual machine, running either locally or in the cloud (AWS, GCP). Currently only Ubuntu 16.04 is supported as the host to run ppc64le docker machine.
  • Set up an x86 system to build and package software for IBM POWER. Learn how to set up an x86 system to build and package software to run on an IBM POWER processor-based system running the Linux operating system. This is useful if you want to develop and build software on your x86 notebook or desktop, but your customers want to use the software you develop on their IBM POWER hardware running Linux.

When you have your hardware access plan in place, you should be ready to begin the port.