page-brochureware.php

Porting x86 Linux applications to IBM POWER
The definitive how-to guide

Get support

Get support from the experts

If you have comments or would like to contribute to this guide, we welcome your feedback.   Learn how

Suggest new content or make a comment on the existing content by using one of these methods.

Note: If you’re a non-IBMer and you want to comment – please use the email option for now.

  1. Log an issue in GitHub.
    • Go here: https://github.ibm.com/linuxonpower/porting-guide
    • Note which page and section you are referring to in the issue title. For example, “Add dancing chipmunk graphic to the Get ready page:Why Linux section”
    • Include the page URL in the Comment section and add your comment. Use the markdown function if you have formatting suggestions or add a file if that is appropriate.
  2. Download and edit source files in GitHub
    • Go here: https://github.ibm.com/linuxonpower/porting-guide
    • Download the baseline .ODT file. There are three files that contain baseline content for each major section of the guide: Get ready, Get set, and Go.
    • Make your changes directly to the file with Track changes on.
    • Upload your updated file and create a pull request for it.
  3. Send an email to lhalkire@us.ibm.com. Include page name/section, URL, and as much detail about the comment as possible.

Plan for the port

Many packages are trivial to port to POWER, often requiring a simple recompile or make command. However, if you tried building your package and found that it wasn’t trivial, don’t worry, this guide will help you work through most any porting situation.

The first, and possibly most important step, is to assess your environment and build a plan that addresses your specific requirements. After you review the information and recommendations in this topic you should have a solid plan for porting your application to the POWER platform.

What language is your application written in?

Generally speaking, porting an application to POWER is a straight forward process and depending on what language it’s written in, the application may “just run” or at most require a recompile to run. Many of the most commonly used programming languages have already been ported to the POWER platform, which should simplify things as you prepare your own development environment. Check whether your language has been ported to the Power platform.

Programming languages ported to Power

Many of these languages ship with the distributions, some are maintained by the community, and in addition, this list is changing all the time. If you don’t see your language or want more information about where to get the package files for the languages listed below, use the Open Source Availability Tool (OSPAT) to search for the package.

  • Ada
  • Angular.js
  • Awk
  • Bash
  • C# (Mono)
  • Clang
  • Clang++ (LLVM)
  • Clojure (JVM)
  • D (LLVM)
  • DoT.js
  • Erlang
  • Fixedheader.js
  • G++
  • GCC
  • GNU Fortran
  • GNU Go
  • GNU Objective C
  • GNU Objective C++
  • Go lang
  • Haskell
  • HHVM
  • Java (OpenJDK)
  • JQuery
  • JRuby
  • Julia (LLVM)
  • Lua
  • Modula 2/3
  • Node.js / V8S
  • OCaml
  • Octave
  • Perl
  • Phantom.js
  • PHP
  • Pypy
  • Python
  • R lang
  • Ruby
  • Rust (LLVM)
  • Scala (JVM)
  • SpiderMonkey
  • SQL
  • Swift (LLVM)

My application is written in a interpreted or scripting language

Effort required to port: Test only

Most applications written in Java, which is designed to be cross-platform portable, or in interpreted languages such as, Python and Perl, are easy to port because they rely on a specific program (the language interpreter) to make that same program run on different architectures. This means that most x86/Linux applications with no platform specific dependencies will run as is on Power.

Here’s some general information about a few of the most popular programming languages on the platform. If your application is written in a different language, check to see if it’s been ported to Power by using the Open Source Availability Tool (OSPAT) search engine.


Java

Many open source projects are built with OpenJDK and it is usually a safe choice for your development needs. OpenJDK is available from the distributions. You’ll find install instructions here: http://openjdk.java.net/install/

Make sure to download and install the latest IBM Java SDK from here: https://developer.ibm.com/javasdk/downloads/


Node.js

Node.js is available directly from IBM and is maintained and kept current here: https://developer.ibm.com/node/. You can also install a Docker container from the same page. IBM has also tested almost all NPMs on the platform:

  • Automated testing, using each tool’s own supplied tests.
  • Found some errors with NPMs pulling down Intel based binaries, but generally there were very few problems found.

Python

Python versions 2 and 3 are available from the distributions. Python 2 continues to be provided for backwards compatibility and Python 3 should be the new standard.

There is also a miniconda environment for ppc64le available here: https://repo.continuum.io/miniconda/Miniconda2-4.3.14-Linux-ppc64le.sh

A repository of ppc64le packages ported to the miniconda environment can be found here: https://repo.continuum.io/pkgs/

My application is written in a compiled language

Effort required to port: Recompile and test*

C, C++, and FORTRAN languages are compiled to instructions for a specific machine/platform. The C language in particular can expose more of the underlying machine architecture to the program and is thus, slightly less portable. That said, most applications written in a compiled language with no platform dependencies will only require a recompile to run on Power.

If you’re porting an application written in C/C++, you should seriously consider installing the IBM Software Development Kit for Linux on Power (SDK). The SDK is a free, Eclipse-based Integrated Development Environment (IDE) that integrates C/C++ source development, the IBM Advance Toolchain, Post-Link Optimization, and classic Linux performance analysis tools, including Oprofile, Perf, and Valgrind. The SDK also includes powerful tools to help you port to Linux on Power including:


Migration Advisor

The Migration Advisor scans your project source code and reports issues that are likely due to portability or performance.


Build Advisor

The Build Advisor scans your project build output and recommends “best practices” you can implement for improved results. It looks for compilers (type and version), optimization levels, processor-specific optimization, and other compiler and linker flags.


Source Code Advisor

The Source Code Advisor leverages technology developed by IBM Research to analyze the run-time characteristics of a program for performance issues that cannot be determined at compile-time. These issues are reported with explanations and suggestions for possible remedies. In addition, for most issues, the Source Code Advisor offers to implement the remedy directly in the source code with a single click.

Download the SDK from here: https://developer.ibm.com/linuxonpower/sdk-download/ or in source form from here: https://github.com/open-power-sdk.

Does your application require a database?

No, my application does not require a database

Great, you can continue on to the choose Linux distribution planning topic.

Yes, my application requires a database

According to Gartner, by 2018, more than 70% of new in-house applications will be developed on an open source database management system (OSDBMS), and 50% of existing commercial RDBMS instances will have been converted. And guess what? Open source databases running on POWER8 deliver 1.8-2X+ greater value versus equivalent x86 solutions.

If your application requires a database, you should verify that the database has been ported to Power. The best way to do this is to search for it using OSPAT.

Does the database run on Power?

Yes, the database runs on Power

Great, you can and continue on to the choose Linux distribution planning topic.

No, but I plan to migrate to one that does and need recommendations

In a hybrid IT enviornment, it is possible to continue to use your existing database server while you port your applications in stages – even if the database is not supported on the Power platform.

However, if (or when) you’re ready to migrate to a database supported on the Power platform, consider these recommendations, based on the type of database your application requires:

NoSQL databases

POWER processor-based systems were designed for various big data and analytics workloads by providing 4X more threads per core, memory bandwidth, and cache than other platform options. These benefits translate into superior performance gains for NoSQL solutions and make it an ideal solution for managing your big data workloads on Power Systems running Linux.

MongoDB Enterprise Server

Type: Document Store | Latest version: v3.4 | Distribution support: RHEL 7.1, 7.2 and Ubuntu 16.04

MongoDB is a NoSQL document store designed for unstructured or semi-structured big data. Run it on POWER8 and you gain a high-performance DBaaS platform that delivers an integrated, real-time view of all your data – throughout your enterprise. Learn more:

Go to the MongoDB website


Neo4j

Type: Graph store | Latest version: 3.2 | Distribution support: RHEL 7.3 and Ubuntu 16.04

Neo4j is an industry leading open source graph database, that when run on POWER, resets the scalability benchmark for what real-time graph processing can deliver your data. Learn more:

Go to the Neo4j website


Redis

Type: In-memory key value store | Latest version: 3.2.8 | Distribution support: RHEL 7.3, Ubuntu 14.04, 16.04

Redis NoSQL open source data engine and Power Systems facilitates developer processes, delivering next-generation applications with near-zero latency, high availability and seamless scalability to meet user expectations. Learn more:

Go to the Redis Labs website


ScyllaDB

Type: Wide-column store

Scylla is an open source NoSQL database that is a drop-in alternative to Apache Cassandra that leverages the best features of Cassandra while providing 10 times better performance than Cassandra and low levels of latency. Scylla’s highly scalable architecture enables it to handle one million database transactions per second on a single server. Learn more:

Go to the Scylla website


Cassandra

Type: Wide-column store | Latest version: 3.7 | Distribution support: RHEL 7.3, Ubuntu 16.04

Apache Cassandra is a free and open-source distributed NoSQL database management system designed to handle large amounts of data across many commodity servers, providing high availability with no single point of failure. Learn more:

Go to the Apache Cassandra website

Relational databases

Switch to an open source relational database that runs on the Power platform and you’ll achieve improved price performance, while leveraging open innovation and exceeding the capabilities of your currently installed system.

EnterpriseDB (EDB) Postgres Advanced Server

Type: Relational DBMS | Latest version: 9.5 | Distribution support: RHEL 7.3

EDB Postgres Advanced Server is an open source object relational database for enterprise data management. Run it on IBM POWER LC servers for big data and gain a 1.8x price-performance advantage guarantee over x86 servers. Learn more:

Go to the EDB website


Kinetica

Type: In-memory accelerated by GPU | Latest version: 6.0 | Distribution support: RHEL 7.3, Ubuntu 16.04

Kinetica is a GPU-accelerated database for real-time analysis of large and streaming datasets. Kinetica on POWER8 with NVIDIA NVLink Technology and Tesla P100 GPUs delivers 2.4x more Kinetica queries per hour than accelerated x86 solutions. Learn more:

Go to the Kinetica website

Data frameworks

Hortonworks Data Platform (HDP)

100% open source Apache Hadoop and Spark on IBM Power Systems built with OpenPOWER technology. Hortonworks Data Platform on OpenPOWER LC delivers 1.7X Hadoop workload performance compared to x86. Learn more:

Go to the Hortonworks website

Does your application have software dependancies?

If your application has prerequisite or dependant packages, you’ll need to make sure they are available on the platform and Linux distribution you plan to use.

No, my application does not have any software dependancies

Great, you can continue on to the choose Linux distribution planning topic.

Yes, my application has software dependencies

Use the Open Source POWER Availability Tool (OSPAT) search engine to check if the correct open source and 3rd party (aka Enterprise) packages and versions are available on the distribution of your choice.

Which Linux distribution(s) will you install?

IBM, in partnership with the major distributions, Ubuntu, Red Hat, and SUSE, provides a complete support solution for implementing and managing the Linux operating system for your enterprise server and cloud environments. Learn more

In addition, many of the most popular community distributions also support the platform, including Fedora, OpenSUSE, Debian, and CentOS. The right distribution for your porting project will be the one that addresses your most critical requirements.

I need a distribution that provides the latest and greatest technology

Ubuntu, Debian, and Fedora typically have frequent releases that integrate the latest open source projects. They are good choices for initial development in cloud, containers, cognitive, HPC, and other areas where the latest technology is absolutely necessary.

I need a distribution that provides the best support

  • Red Hat Enterprise Linux (RHEL), SUSE Enterprise Linux (SLES), and Ubuntu, in partnership with IBM, provide long term distribution support and are a common target for key ISV applications that expect long term stability.
  • Distributions like Ubuntu (and its many variants, such as kubuntu), CentOS, and OpenSUSe provide community supported distributions, which provide much of the same application support as their enterprise versions but without the corresponding support costs.

Is your development environment setup for the port?

You most likely already have a development environment in place that includes the tools and processes that work for you and your particular situation. However, if you’re looking for some suggestions specific to porting and developing on the Power platform, the following recommendations may help.

Tip! IBM development tools to consider

IBM has several tools that make porting to and developing on the Power platform a snap. If your application is written in a complied language such as C/C++ or Go, you should consider installing them to ease the porting process.

I need IDE recommendations

Eclipse
Eclipse is one of the most widely used Integrated Development Environments (IDEs). It contains a base workspace and an extensible plug-in system for customizing your development environment. Learn more here: http://www.eclipse.org/.


Kubernetes
Kubernetes is an open source system for managing containerized applications across multiple hosts. It provides basic mechanisms for deploying, maintaining, and scaling applications. Learn more here: https://github.com/kubernetes/kubernetes

I need Continuous Integration (CI) framework recommendations

Jenkins CI
Jenkins is a free, open source automation server that provides hundreds of plugins to support building, deploying and automating your open source project. Learn more about Jenkins and download it from here: http://jenkins-ci.org

  • Recommendation for open source projects
    The POWER CI is a Jenkins service managed by the Oregon State University (OSU) – Open Source Lab (OSL) and is intended to give developers easy access to the POWER architecture for building and managing their open source projects via Jenkins.
  • Recommendation for ISV and enterprise projects
    The Continuous Integration/Continuous Development (CICD) is a Jenkins service managed by IBM and is intended to give ISV developers access to the POWER architecture for building and managing their open source projects via Jenkins.

Travis CI
Travis CI will be supported on the platform soon.

I need library recommendations

In addition to the typical development libraries such as libc, libm, libpthread, Zlib, OpenSSL, Boost, Tcmalloc, Intel TBB, and SPHDE (which are included in the latest release of IBM Advance Toolchain), consider these math libraries that will boost the performance of your HPC and deep learning applications.

ESSL
Engineering and Scientific Subroutine Library (ESSL) is a collection of high performance mathematical subroutines providing a wide range of functions for many common scientific and engineering applications. Languages: Fortran, C and C++ serial, SMP and SPMD. Download ESSL


libblas
The Basic Linear Algebra Subprograms (BLAS) are libraries used for optimizing mathematical computation on POWER8.


MASS
Mathematical Acceleration Subsystem (MASS) for Linux consists of libraries of mathematical intrinsic functions tuned specifically for optimum performance on POWER architectures. Languages: C, C++ and Fortran. Download MASS


Here’s a good blog that describes how to deploy and optimize these libraries on Power: Building Optimized Libraries for Deep Learning on OpenPOWER Linux Systems

I need compiler recommendations

Many of the most popular compilers are available and optimized for POWER, including the following:

GNU Compiler Collection (GCC)

GCC is available from the distributions (and is also integrated into the the IBM Advance Toolchain for Linux on Power if you choose to install it). Install GCC from your distribution by using apt-get install or yum install


Clang

Clang is available from the distributions. Install it using apt-get install or yum install


XL C/C++ for Linux

There are three ways to get the community version of XL C/C++ for Linux, which is a no-charge, fully functional product for developers who do not require official IBM support.

Purchase and download the fully supported version of XL C/C++ for Linux from the IBM Marketplace.


XL Fortran for Linux

XL Fortran for Linux Community Edition (on little endian distributions) is a no-charge, fully functional product for developers who do not require official IBM support. You can get it three ways:

Purchase and download the fully supported version of XL Fortran for Linux from the IBM Marketplace.


Compiler optimization tips

I need performance profiling tools recommendations

Use the following suite of tools to identify performance problems in your application and to understand how your application interacts with the Linux kernel.

Tip! OProfile, perf, and Valgrind, plus gcov and gprof are integrated into the SDK for Linux on Power to provide quick and easy profiling of your applications. Results are displayed in a space-efficient hierarchical layout, which can be expanded down to the line of source code or even the hardware instruction.


OProfile
The OProfile tool profiles code that is based on hardware-related events, such as cache misses or processor cycles. For example, OProfile can help you determine the source routines that cause the most cache misses. OProfile uses hardware performance counters provided in many processors, including IBM POWER8. Learn more:


perf
Perf is a powerful performance analysis tool with two different components: a userspace tool and a kernel infrastructure. You can install it using the yum install perf command on Fedora/RHEL or the apt-get install linux-tools-common command on Ubuntu. Learn more:


Valgrind
Valgrind is an suite of profiling tools used for detecting memory leaks, memory debugging, and performing detailed profiling to find blockages in programs. Learn more:

Do you have test cases and tools prepared?

Creating of a viable test plan is critical to the success of any porting project. You’ll test during various stages of the port and, of course, your plan will be tailored to the specific requirements of your project.

Most applications come with their own unit and functional tests. Some of those require configuration, such as setting up multiple servers or applications. Some require additional tools, such as jMeter, Selenium, Electric Fence (eFence), or insure++, for example. Check OSPAT to see if your favorite is available on the platform.

Do you have access to Power hardware?

If you already have access to Power hardware, great, you should be ready to begin the port. If not, don’t worry, we’ve got you covered.

There are Power cloud resources available throughout the world

Go to the cloud resources page, to quickly and easily find regional and global sites that offer access to Power development tools and platforms. Choose any of the available cloud resources based on your region and type of access, instance, accelerator and OS required.

Recommendations

I need short-term, no-cost access to Power hardware

I need long-term access to Power hardware with GPUs, CAPI, or NVlink technologies

  • IBM Hardware Mall
    IBM Power Systems and IBM System Storage products are made available for your use in developing, evaluating, porting, migrating, optimizing, and or demonstrating your programs and solutions.
  • Nimbix
    The Nimbix Jarvice Cloud platform is the first commercial cloud environment to feature POWER8 with NVLink. It features ready to deploy instances for deep learning and NVLink application deployment and development. Request access

I want try the port through a simulator

  • Install the IBM Software Developer Kit (SDK) for Linux on Power on your (Linux-based) desktop and then run your Power binaries on the same desktop using emulation technologies. Both QEMU (user-mode emulation) and the IBM Power Functional Simulator (full system simulation) are provided with the SDK. In this mode, you can develop and port applications without the need for Power hardware.
  • Provision ppc64le Docker-Machine on Intel. Learn about using the Docker Machine tool to provision a Power (ppc64le) Docker environment on any Intel (x86_64) virtual machine, running either locally or in the cloud (AWS, GCP). Currently only Ubuntu 16.04 is supported as the host to run ppc64le docker machine.

When you have your hardware access plan in place, you should be ready to begin the port.