Quick start: Download, cd lpcpu, run “lpcpu.sh”.

lpcpu gathers data from both x86 and Power Linux based systems.

We recommend downloading the latest copy on a regular basis.

Update, Aug 2018: The lpcpu code is now available on github:

https://github.com/jeniferh/lpcpu

To clone the repository:

# git clone https://github.com/jeniferh/lpcpu.git
Cloning into 'lpcpu'...
remote: Counting objects: 140, done.
remote: Compressing objects: 100% (113/113), done.
remote: Total 140 (delta 27), reused 138 (delta 25), pack-reused 0
Receiving objects: 100% (140/140), 3.43 MiB | 2.14 MiB/s, done.
Resolving deltas: 100% (27/27), done.
# cd lpcpu

To create a new “lpcpu.tar.bz2” tar ball for sharing purposes, simply run the create-tarball script from the lpcpu repository, ex:

# ./create-tarball.sh 
Create LPCPU Distribution Tarball Creating clean base directory Recording versions 1: 56ac01eb012e07136215b36bb2211b8e13af7132 2018-08-17 15:43:16 -0500 Adding lpcpu.sh to the base directory ‘lpcpu.sh’ -> ‘lpcpu/lpcpu.sh’ Updating lpcpu.sh [...]

Recent updates:
Aug 28: Migrated the user page here and added github info 🙂 note: old user page was here
July 25 2018: Added ethtool stat diff support, iotop
Feb 9 2017: Added support for lshw, lsb-release, and a new perf_annotate=yes option when running the perf profiler.
Mar 12 2017: Grab /etc/os-release; decimal number diffs with ndiff.py; “pmu” adds “–separate-cpu” to ocount; bug fixes
Sep 28 2017: Fix kvm-stat errors

Linux Performance Customer Profiler Utility (lpcpu)

This script captures a lot of potentially interesting performance data (profile information, system information, and some system configuration information) on a single pass approach. Gathering all of this information at once allows the context to be understood for the performance profiling data to be analyzed. The script will check to be sure you have the basic tools installed on your system.

This script takes advantage of all of the “normal” performance tools used on Linux.


    iostat
    mpstat
    vmstat
    perf
    meminfo
    top
    sar

In addition, relevant system information is gathered, with the profiler output, into a single tarball saved on your system. By default, the file is saved in /tmp.

The script is designed to run on both x86 and Power servers, it should work well on most Linux distributions.

The script creates a zipped tar-ball placed in /tmp by default. You can un-zip the file and poke around the data files captured to learn more about your system. Typically, the zipped tar-ball is returned to performance analysts at IBM who can help with problem determination and ideas.

In 95% of our interactions with product teams and customers, there is generally something easy to address first. There are naturally many other in-depth tools we might leverage in subsequent data runs, but first we want to be sure everyone is “on the same page”.

To test the script

To run the script just to test it, try this as the root user. Normally the script runs at least 120 seconds, in this case, you are specifying a run of 15 seconds.


# tar -jxf lpcpu.tar.bz2
# cd lpcpu
# ./lpcpu.sh duration=15

This checks for errors and attempts to run all of the default profilers. A typical error will be that the profiler being run is not installed. Obviously, in that case the profiler should be installed, or if not available, you can override the profiler list to skip that tool (but keep in mind that the data gathered may not be as useful).

You do need a number of rpm packages installed on your system, depending on what profiler data you want to collect, for ex:


sysstat
perf

The script does assume the Linux kernel with symbol information is available. This depends on your distro and version since they are generally packaged differently.

This script is not targeted or focused on Java applications, but it does serve well as the first pass data gathering tool.

Typical run

Typically, the workload being tested reaches a fairly steady state (has settled down) and performance data can be collected from the system.

In this case, you can run the script for the default 2 minutes, and the script will profile the information and gather everything together.

# ./lpcpu.sh

Over-riding values

The classic profilers and the parameters to override are listed in the script.

The parameters and some example values are listed here:


profilers="sar iostat mpstat vmstat oprofile top meminfo" ## (these are the list of profilers run by default)
extra_profilers="perf kvm" ## (space separated list of additional profilers to include)
duration=120 ## (in seconds)
interval=5
cmd=""
output_dir="/tmp"

These values are fairly self-explanatory.

* You can override the profilers statement and collect less data if you wish. Some product teams feel they need hardware counter data right up front, we generally discourage that until there’s a clear need to gather that level of detail. oprofile and perf can easily gather hardware related profiling data and CPI-type information.

* The default duration is 120 seconds and the default sampling interval for some of the commands is 5 seconds. This sampling interval doesn’t apply to oprofile.

* If you wish to profile the execution of a script/command, you can specify that script name in cmd=””. Typically we get the workload to a fairly steady-state and then run lpcpu.sh in a separate window.

Post-processing the results

Along with the script, we have the ability to format the results into a series of html pages and charts.

Take the lpcpu.tar.bz2 file, and unpack it.


# cd /tmp
# tar -jxf lpcpu.tar.bz2

Copy the data tarball to the system you would like to host your data on (this could be the test system or a workstation). Unpack the tarball and issues the following commands:

# pwd
/var/www/html
# tar -jxf lpcpu_data.mysystem.default.2012-04-04_1147.tar.bz2
# cd lpcpu_data.mysystem.default.2012-04-04_1147/
# ./postprocess.sh /tmp/lpcpu/
...lots of messages..
# ls -l summary.html

View that summary.html file in a browser. Depending on your browser, you may need a web server to make use of the new charting abilities. A Python script is included for running a simple web server should you not have one available. If you cannot run the Python script and do not have a web server available, please fall back on the old charting method (see the README for details).

Full Example Output

Here is an example of a default collection:


root@demo:/home# tar -jxf lpcpu.tar.bz2 
root@demo:/home# cd lpcpu/
root@demo:/home/lpcpu# ./lpcpu.sh

Running Linux Performance Customer Profiler Utility version 28605ac54c8a9485cae4e9ab507e2db8f60b4f2d 2016-09-19 17:04:47 -0500

Starting Time: Mon Jan 16 13:16:20 CST 2017
Setting up sar.
Setting up iostat.
Setting up mpstat.
Setting up vmstat.
Setting up top.
Setting up meminfo.
Setting up proc-interrupts.
Profilers start at: Mon Jan 16 13:16:20 CST 2017
starting sar.default [5]
starting iostat.default [5] [mode=disks]
starting mpstat.default [5]
starting vmstat.default [5]
Starting top.
starting meminfo.default [5] 
starting proc-interrupts.default [5]

Waiting for 120 seconds.
Stopping sar.
Stopping iostat.
Stopping mpstat.
Stopping vmstat.
Stopping top.
Stopping meminfo.
Stopping interrupts.
Profilers stop at: Mon Jan 16 13:18:20 CST 2017
Processing sar data.
Processing iostat data.
Processing mpstat data.
Processing vmstat data.
Processing top data.
Processing meminfo data.
Processing interrupts data.
Setting up postprocess.sh
Gathering system information
Finishing time: Mon Jan 16 13:18:22 CST 2017
Packaging data...data collected is in /tmp/lpcpu_data.demo.default.2017-01-16_1316.tar.bz2
 

After the collection is complete, here is example post processing output:


root@demo:/home/lpcpu# cd /tmp/
root@demo:/tmp# tar -jxf lpcpu_data.demo.default.2017-01-16_1316.tar.bz2 
root@demo:/tmp# cd lpcpu_data.demo.default.2017-01-16_1316/
root@demo:/tmp/lpcpu_data.demo.default.2017-01-16_1316# ./postprocess.sh /home/lpcpu/
Processing Directory : sar.breakout.default.001
SAR run queue data contains information on blocked processes
SAR memory data contains information on dirtymem
Current SAR version does not provide power mgmt data by default
Loaded block device hierarchy data from block-device-hierarchy.dat
Processing File : iostat.default.001
Discovered iostat_type=[disks]
Processing iostat data file with broken out read/write wait times
Processing File : mpstat.default.001
Using guest nice mode
postprocess-mpstat: Loading topology archive from ./system-topology.dump
Processing File : vmstat.default.001
Discovered vmstat_interval=[5]
Discovered vmstat has embedded timestamps
Processing File : meminfo-watch.default.001
Processing File : proc-interrupts.default.001
postprocess-proc-interrupts: Loading topology archive from ./system-topology.dump
    MTJP : No jobs to process, returning

Join The Discussion

Your email address will not be published. Required fields are marked *