Quick start: Download, cd lpcpu, run “lpcpu.sh”.
lpcpu gathers data from both x86 and Power Linux based systems.
We recommend downloading the latest copy on a regular basis.
Update, Aug 2018: The lpcpu code is now available on github:
To clone the repository:
# git clone https://github.com/jeniferh/lpcpu.git Cloning into 'lpcpu'... remote: Counting objects: 140, done. remote: Compressing objects: 100% (113/113), done. remote: Total 140 (delta 27), reused 138 (delta 25), pack-reused 0 Receiving objects: 100% (140/140), 3.43 MiB | 2.14 MiB/s, done. Resolving deltas: 100% (27/27), done. # cd lpcpu
To create a new “lpcpu.tar.bz2” tar ball for sharing purposes, simply run the create-tarball script from the lpcpu repository, ex:
Create LPCPU Distribution Tarball Creating clean base directory Recording versions 1: 56ac01eb012e07136215b36bb2211b8e13af7132 2018-08-17 15:43:16 -0500 Adding lpcpu.sh to the base directory â€˜lpcpu.shâ€™ -> â€˜lpcpu/lpcpu.shâ€™ Updating lpcpu.sh [...]
Aug 28: Migrated the user page here and added github info ðŸ™‚ note: old user page was here
July 25 2018: Added ethtool stat diff support, iotop
Feb 9 2017: Added support for lshw, lsb-release, and a new perf_annotate=yes option when running the perf profiler.
Mar 12 2017: Grab /etc/os-release; decimal number diffs with ndiff.py; “pmu” adds “–separate-cpu” to ocount; bug fixes
Sep 28 2017: Fix kvm-stat errors
Linux Performance Customer Profiler Utility (lpcpu)
This script captures a lot of potentially interesting performance data (profile information, system information, and some system configuration information) on a single pass approach. Gathering all of this information at once allows the context to be understood for the performance profiling data to be analyzed. The script will check to be sure you have the basic tools installed on your system.
This script takes advantage of all of the “normal” performance tools used on Linux.
iostat mpstat vmstat perf meminfo top sar
In addition, relevant system information is gathered, with the profiler output, into a single tarball saved on your system. By default, the file is saved in /tmp.
The script is designed to run on both x86 and Power servers, it should work well on most Linux distributions.
The script creates a zipped tar-ball placed in /tmp by default. You can un-zip the file and poke around the data files captured to learn more about your system. Typically, the zipped tar-ball is returned to performance analysts at IBM who can help with problem determination and ideas.
In 95% of our interactions with product teams and customers, there is generally something easy to address first. There are naturally many other in-depth tools we might leverage in subsequent data runs, but first we want to be sure everyone is “on the same page”.
To test the script
To run the script just to test it, try this as the root user. Normally the script runs at least 120 seconds, in this case, you are specifying a run of 15 seconds.
# tar -jxf lpcpu.tar.bz2 # cd lpcpu # ./lpcpu.sh duration=15
This checks for errors and attempts to run all of the default profilers. A typical error will be that the profiler being run is not installed. Obviously, in that case the profiler should be installed, or if not available, you can override the profiler list to skip that tool (but keep in mind that the data gathered may not be as useful).
You do need a number of rpm packages installed on your system, depending on what profiler data you want to collect, for ex:
The script does assume the Linux kernel with symbol information is available. This depends on your distro and version since they are generally packaged differently.
This script is not targeted or focused on Java applications, but it does serve well as the first pass data gathering tool.
Typically, the workload being tested reaches a fairly steady state (has settled down) and performance data can be collected from the system.
In this case, you can run the script for the default 2 minutes, and the script will profile the information and gather everything together.
The classic profilers and the parameters to override are listed in the script.
The parameters and some example values are listed here:
profilers="sar iostat mpstat vmstat oprofile top meminfo" ## (these are the list of profilers run by default)
extra_profilers="perf kvm" ## (space separated list of additional profilers to include)
duration=120 ## (in seconds)
These values are fairly self-explanatory.
* You can override the profilers statement and collect less data if you wish. Some product teams feel they need hardware counter data right up front, we generally discourage that until there’s a clear need to gather that level of detail. oprofile and perf can easily gather hardware related profiling data and CPI-type information.
* The default duration is 120 seconds and the default sampling interval for some of the commands is 5 seconds. This sampling interval doesn’t apply to oprofile.
* If you wish to profile the execution of a script/command, you can specify that script name in cmd=””. Typically we get the workload to a fairly steady-state and then run lpcpu.sh in a separate window.
Post-processing the results
Along with the script, we have the ability to format the results into a series of html pages and charts.
Take the lpcpu.tar.bz2 file, and unpack it.
# cd /tmp
# tar -jxf lpcpu.tar.bz2
Copy the data tarball to the system you would like to host your data on (this could be the test system or a workstation). Unpack the tarball and issues the following commands:
# tar -jxf lpcpu_data.mysystem.default.2012-04-04_1147.tar.bz2
# cd lpcpu_data.mysystem.default.2012-04-04_1147/
# ./postprocess.sh /tmp/lpcpu/
...lots of messages..
# ls -l summary.html
View that summary.html file in a browser. Depending on your browser, you may need a web server to make use of the new charting abilities. A Python script is included for running a simple web server should you not have one available. If you cannot run the Python script and do not have a web server available, please fall back on the old charting method (see the README for details).
Full Example Output
Here is an example of a default collection:
root@demo:/home# tar -jxf lpcpu.tar.bz2 root@demo:/home# cd lpcpu/ root@demo:/home/lpcpu# ./lpcpu.sh Running Linux Performance Customer Profiler Utility version 28605ac54c8a9485cae4e9ab507e2db8f60b4f2d 2016-09-19 17:04:47 -0500 Starting Time: Mon Jan 16 13:16:20 CST 2017 Setting up sar. Setting up iostat. Setting up mpstat. Setting up vmstat. Setting up top. Setting up meminfo. Setting up proc-interrupts. Profilers start at: Mon Jan 16 13:16:20 CST 2017 starting sar.default  starting iostat.default  [mode=disks] starting mpstat.default  starting vmstat.default  Starting top. starting meminfo.default  starting proc-interrupts.default  Waiting for 120 seconds. Stopping sar. Stopping iostat. Stopping mpstat. Stopping vmstat. Stopping top. Stopping meminfo. Stopping interrupts. Profilers stop at: Mon Jan 16 13:18:20 CST 2017 Processing sar data. Processing iostat data. Processing mpstat data. Processing vmstat data. Processing top data. Processing meminfo data. Processing interrupts data. Setting up postprocess.sh Gathering system information Finishing time: Mon Jan 16 13:18:22 CST 2017 Packaging data...data collected is in /tmp/lpcpu_data.demo.default.2017-01-16_1316.tar.bz2
After the collection is complete, here is example post processing output:
root@demo:/home/lpcpu# cd /tmp/ root@demo:/tmp# tar -jxf lpcpu_data.demo.default.2017-01-16_1316.tar.bz2 root@demo:/tmp# cd lpcpu_data.demo.default.2017-01-16_1316/ root@demo:/tmp/lpcpu_data.demo.default.2017-01-16_1316# ./postprocess.sh /home/lpcpu/ Processing Directory : sar.breakout.default.001 SAR run queue data contains information on blocked processes SAR memory data contains information on dirtymem Current SAR version does not provide power mgmt data by default Loaded block device hierarchy data from block-device-hierarchy.dat Processing File : iostat.default.001 Discovered iostat_type=[disks] Processing iostat data file with broken out read/write wait times Processing File : mpstat.default.001 Using guest nice mode postprocess-mpstat: Loading topology archive from ./system-topology.dump Processing File : vmstat.default.001 Discovered vmstat_interval= Discovered vmstat has embedded timestamps Processing File : meminfo-watch.default.001 Processing File : proc-interrupts.default.001 postprocess-proc-interrupts: Loading topology archive from ./system-topology.dump MTJP : No jobs to process, returning