Overview

Skill Level: Intermediate

Specjbb2015 Benchmark Measurements were taken on AIX 72D version ,SMT8 24core Tuleta P8 system.

Objective was to achieve Optimal performance on P8 using certain AIX Handtune options.

Ingredients

  • Set P8 power mode to "maximum performance" or "maximum frequency"
  • Latest IBMJDK or OpenJDK should be installed on the SUT.(IBM Java8 was used for the said measurements)

Step-by-step

  1. AIX Tunings

    1. smtctl -t 8 -w boot; bosboot -a; shutdown -Fr
    2. schedo -p -o sched_R=0
    3. schedo -p -o sched_D=0
    4. ulimit -s unlimited; ulimit -d unlimited; ulimit -m unlimited; ulimit -f unlimited; ulimit -n unlimited
    5. export LDR_CNTRL=STACKPSIZE=64K@DATAPSIZE=16M@TEXTPSIZE=64K@SHMPSIZE=16M
    6. export MEMORY_AFFINITY=MCM
    7. dscrctl -n -b -s 3
    8. export AIXTHREAD_MUTEX_DEBUG=OFF
    9. export AIXTHREAD_COND_DEBUG=OFF
    10. export AIXTHREAD_RWLOCK_DEBUG=OFF
    11. export AIXTHREAD_AFFINITY=strict
  2. Specjbb2015 configuration and run steps:

    1) In SPECjbb2015-1.00, use the below mentioned  script to run specjbb2015 in multi mode using IBMJDK: <run_multi.sh.ibmjdk.aix_withtuning_72D_SMT8>You can use the scripts for the other two modes or OpenJDK.
    2) Look over the run_multi.sh.ibmjdk.aix_withtuning_72D_SMT8 script to make sure things are correct like JAVA environment variables, numa bindings.

    Please remove this java option “-Xlp” in the script, if you are not planning to use large pages from vmo_options. You should find two places where that exists. The script does NUMA binding and sets to use the OS default pre-fetcher setting. The script is setting the options and then spawning the java JVMs.

    The number of JVMs are: 1 Controller (does very little CPU-wise), GROUPs * (1 Driver and 1 Backend). The group is a variable we have in the run script. We have set it to 4. We are binding each group to a NUMA node.

    The script run_multi.sh.ibmjdk.aix_withtuning_72D_SMT8 calls tune_aix.sh which has the following content:

    • ulimit -s unlimited; ulimit -d unlimited; ulimit -m unlimited; ulimit -f unlimited; ulimit -n unlimited
    • export LDR_CNTRL=STACKPSIZE=64K@DATAPSIZE=16M@TEXTPSIZE=64K@SHMPSIZE=16M
    • export MEMORY_AFFINITY=MCM
    • dscrctl -n -b -s 3
    • export AIXTHREAD_MUTEX_DEBUG=OFF
    • export AIXTHREAD_COND_DEBUG=OFF
    • export AIXTHREAD_RWLOCK_DEBUG=OFF
    • export AIXTHREAD_AFFINITY=strict

    3) The last configuration file is config/specjbb2015.props. This is where you can change specjbb2015 specific options. Current options are for running on a 24-core Tuleta S824 with 4 groups (one on each NUMA node)
    4) Start the run: ./kill.all.jvms.sh;run_multi.sh.ibmjdk.aix_withtuning_72D_SMT8. The kill all jvm script kills all java processes.in case there are other JVMs running from previous runs, but be careful if you have other java processes that you do not wish to kill.

    For OOB measurement use the script for reference:  run_multi_ibmjdk_aix_OOB.sh

    This script does not have any bindings.

    ##Run Results

    Monitor the run on controller.log or controller.out. At the end of the run, you can find the results in controller.out. Just grep… grep “RUN RESULT” 17-*/controller.out. and you will see the max-jops and crit-jops.

    Final metrics of the benchmark, namely the max-jOPS which is a pure throughput metric, and the critical-jOPSwhich is a throughput under response time constraint metric.

    Please refer the following document for detailed information on Specjbb2015 benchmark.

    https://www.spec.org/jbb2015/docs/userguide.pdf

    Design document at

    https://www.spec.org/jbb2015/docs/designdocument.pdf

     

Join The Discussion