Simplified program conversion with QBNCVTPGM and QBNCVTPLST

Introduction to program conversion

Taking advantage of new processor features may improve the overall performance of an application and using the latest integrity enhancements can make an application more secure. A way to include those improvements in an existing program is to convert it. Introduced in IBM i release V3R6, program conversion rebuilds the program to take advantage of the latest processor features and integrity enhancements. However, program conversion happens automatically only when it is needed. Therefore, programs may be running on an IBM i system that have not been converted to incorporate the latest processor features and integrity enhancements.

Program conversion happens automatically if a program is moved to another IBM i system that does not support all of the processor features used by the program. For example, to implement a sequence of instructions, a program may use hardware features available on the current processor. If this program is moved to an earlier version of the processor that does not support those specific hardware features, the program must be converted so those features can be implemented in a different way (but still correctly). Note that automatic conversion does not happen if moving a program in the other direction. If a program runs correctly on an earlier version of the processor, it will run correctly on a later processor due to forward compatibility. See Figure 1. Automatic conversion example.

Figure 1. Automatic conversion example

Conversion can also happen if it is manually forced using one of the following mechanisms:

1. Using the Change Program (CHGPGM) or Change Service Program (CHGSRVPGM) commands with the force create option (FRCCRT(*YES)) if the program contains sufficient creation data. To check if there is enough creation data, use the Display Program (DSPPGM) or Display Service Program (DSPSRVPGM) commands. For Integrated Language Environment (ILE) programs or service programs, a value of *YES for All creation data indicates that enough creation data is available. For Original Program Model (OPM) programs, a value of *ALL for Observable information indicates that enough creation data is available. To check modules, the Display Module (DSPMOD) command can be used. A value of *YES for Creation data indicates that enough creation data is available.

2. Saving the program or service program using the Save Object (SAVOBJ) command and restoring it using the Restore Object (RSTOBJ) command with the force object conversion option (FRCOBJCVN(*YES *ALL)).

Here are some conditions where manual conversion would be useful:

1. Programs are being moved to a new processor. Because processors are forward compatible, moving a program from an older processor will not cause automatic conversion. Yet, the new processor may have additional features that the optimizing translator can take advantage of to make the program more efficient.

2. A new release of the operating system has been installed. The optimizing translator may be able to improve the efficiency of the program due to enhancements that were made. Also, new integrity enhancements may have been made to the optimizing translator, making the application more secure.

3. Integrity PTFs have been installed which may affect the generated machine code of the optimizing translator. These code generation changes may affect the security of the application being converted.

Introduction to the QBNCVTPGM and QBNCVTPLST tools

Starting in IBM i release 7.1, new tools are available to easily convert existing programs. They are the QBNCVTPGM and QBNCVTPLST tools. These tools can be used to convert objects even if the program does not have the required creation data needed for the change commands (see the Restrictions section). QBNCVTPGM is included in the operating system through a program temporary fix (PTF).

QBNCVTPLST is available for download. See the How to get the tools section.

Note that use of these new tools is not required for the correct running of an application; however, using them can provide the advantages described previously.

QBNCVTPGM Tool

The QBNCVTPGM tool is installed with a PTF. It is used to perform conversion of a single program, service program, or module (type *PGM, *SRVPGM, or *MODULE).

Syntax

The syntax for invoking the tool is:

CALL PGM(QSYS/QBNCVTPGM) PARM(object library type)

Where

• object is the name of a single object to be converted.
• library is the name of a single library containing the object.
• type is one of the following: *PGM, *SRVPGM, or *MODULE.

For example, to convert program BLDCPS in library QGPL, use the following code:

CALL PGM(QSYS/QBNCVTPGM) PARM(BLDCPS QGPL *PGM)

Restrictions

In order for the QBNCVTPGM tool to convert the program, service program, or module, the following conditions must be true:

1. The object must be based on the IBM i machine interface (MI). This means any program that runs in the PASE environment cannot be converted with this tool.

2. The MI created object must contain creation data. To check individual programs, use the display commands described in the Introduction section. The object has enough creation data if *YES, *ALL, or *UNOBS is shown. If the object does not contain the appropriate level of creation data, message CPF9899 will be generated by the tool and the object will not be converted.

3. IBM objects included in the QSYS library do not contain creation data and cannot be converted.

4. The object cannot be in use or locked by another job.

5. The profile running this tool must have the *ALLOBJ and *SECADM special authorities, plus *EXECUTE authority to the library containing the object.

6. An OPM program that resides in the QTEMP library cannot be converted.

Messages returned

Any escape messages are signaled back to the caller, so they have an awareness of how the conversion progressed. The following table lists the escape messages and description. Viewing all previous messages in the job log will indicate the actual error that occurred.

Generally, the caller of this tool should perform the following tasks:

1. If MCH3405 or MCH5802 is received, the object could not be converted because it is currently in use or locked by another job. The conversion can be tried again once the object is no longer in use or the lock is released.

2. If CPF9899 is received, the conversion has failed. The job log will contain additional messages describing the details of the failure. There is little sense in trying the conversion again; it will likely fail again.

3. If any other errors occur, it is up to the client to decide what to do.

Possible side effects

When an object is converted, the processor features of the current system are used when building the program. Using these new features may improve the overall performance of the application. In addition, any of the latest integrity enhancements will be included in the converted program.

Converting a program can have the following effects:

1. Objects that are digitally signed will lose their digital signatures. An object is digitally signed if YES is displayed for Digitally Signed in the Display Object Description (DSPOBJD) command, when using the DETAIL(*FULL) option.

2. If a program was previously performance profiled, it may lose some performance benefits it previously obtained from profiling. A program is performance profiled if the DSPPGM or DSPSRVPGM commands show *APYALL, *APYBLKORD, or *APYPRCORD for Profiling data. This change can affect the performance of the object.

3. If a program contains debug breakpoints and undergoes conversion, any breakpoints are removed from the program. Therefore, it is possible for another job to set breakpoints and then have the breakpoints removed. When that happens, message CPD190A (breakpoints, step, or statement traces changed) is sent to the jobs that had set breakpoints in the program.

Thread safety

The QBNCVTPGM program is not thread safe; therefore, if multiple instances of this program are to be run simultaneously, each instance should be run in a different job, not in different threads in the same job. Running in separate jobs will ensure that each instance of the program will run independently.

Using the QBNCVTPLST tool

To provide an example of how the QBNCVTPGM tool can be used, the QBNCVTPLST CL program was developed. The source code for the program is provided below. It can be used as is or modified as needed. Also, see the How to get the tools section to download it.

Listing 1. Source for the QBNCVTPLST CL Program

LINE
0001          PGM        PARM(&LIB &FILE)
0002
0003 /* INPUT/OUTPUT FILE - MEMBERS ARE CREATED BY DSPOBJD */
0004 /*                     OUTPUT(*OUTFILE) */
0005              DCL        VAR(&LIB) TYPE(*CHAR) LEN(10)
0006              DCL        VAR(&FILE) TYPE(*CHAR) LEN(10)
0007
0008 /* MEMBER NAMES IN &LIB/&FILE */
0009 /*   PGMLIST   (INPUT/OUTPUT) - LIST OF PROGRAMS TO CONVERT */
0010 /*   RETRYLIST (TEMP) - IN-USE PROGRAMS FOR LATER RETRY */
0011 /*   ERRORLIST (OUTPUT) - FAILED PROGRAMS */
0012              DCL        VAR(&PGMS) TYPE(*CHAR) LEN(10) +
0013                         VALUE('PGMLIST')
0014              DCL        VAR(&RETRY) TYPE(*CHAR) LEN(10) +
0015                         VALUE('RETRYLIST')
0016              DCL        VAR(&ERRORS) TYPE(*CHAR) LEN(10) +
0017                         VALUE('ERRORLIST')
0018  
0019 /* WORK VARIABLE FOR CONSTRUCTING MESSAGES */
0020              DCL        VAR(&MSG) TYPE(*CHAR) LEN(300)
0021  
0022 /* DECLARE THE INPUT FILE (LIST OF PROGRAMS) AND OVERRIDE IT TO */
0023 /* USE THE SPECIFIED &LIB/&FILE */
0024              DCLF       FILE(QSYS/QADSPOBJ)
0025              OVRDBF     FILE(QADSPOBJ) TOFILE(&LIB/&FILE) +
0026                         MBR(&PGMS)
0027
0028 /* REMOVE ANY EXISTING MEMBER FOR THE RETRY LIST */
0029              RMVM       FILE(&LIB/&FILE) MBR(&RETRY)
0030              MONMSG     MSGID(CPF7310)
0031
0032 /* ERROR LIST WILL REMAIN UNCHANGED - IT ACCUMULATES ERRORS FROM */
0033 /* EACH INVOCATION OF QBNCVTPLST */
0034
0035  LOOP:       RCVF
0036              MONMSG     MSGID(CPF0864) EXEC(GOTO CMDLBL(LOOPEND))
0037
0038              CALL       PGM(QSYS/QBNCVTPGM) +
0039                         PARM(&ODOBNM &ODLBNM &ODOBTP)
0040
0041 /* MONITOR FOR "PROGRAM IN USE" EXCEPTIONS, ADD TO RETRY LIST, */
0042 /* CONTINUE */
0043              MONMSG     MSGID(MCH3405 MCH5802) +
0044                         EXEC(GOTO CMDLBL(ADDRETRY))
0045
0046 /* MONITOR FOR "PROGRAM CONVERSION FAILED" EXCEPTION, ADD TO */
0047 /* ERROR LIST, CONTINUE */
0048              MONMSG     MSGID(CPF9899) EXEC(GOTO CMDLBL(ADDERROR))
0049
0050 /* MONITOR FOR ANY EXCEPTIONS THAT SHOULD JUST BE LOGGED AND */
0051 /* IGNORED, CONTINUE */
0052              MONMSG     MSGID(CPFA0AE) /* IGNORE DELETED OBJECTS */
0053
0054              GOTO       CMDLBL(LOOP)
0055
0056  ADDRETRY:   DSPOBJD    OBJ(&ODLBNM/&ODOBNM) OBJTYPE(&ODOBTP) +
0057                         OUTPUT(*OUTFILE) OUTFILE(&LIB/&FILE) +
0058                         OUTMBR(&RETRY *ADD)
0059              MONMSG     MSGID(CPF2114)
0060              GOTO       CMDLBL(LOOP)
0061
0062  ADDERROR:   DSPOBJD    OBJ(&ODLBNM/&ODOBNM) OBJTYPE(&ODOBTP) +
0063                         OUTPUT(*OUTFILE) OUTFILE(&LIB/&FILE) +
0064                         OUTMBR(&ERRORS *ADD)
0065              MONMSG     MSGID(CPF2114)
0066             GOTO       CMDLBL(LOOP)
0067
0068  LOOPEND:
0069
0070 /* REPLACE MEMBER PGMLIST WITH MEMBER RETRYLIST FOR A LATER */
0071 /* ATTEMPT AT CONVERTING IN-USE PROGRAMS */
0072              RMVM       FILE(&LIB/&FILE) MBR(&PGMS)
0073              RNMM       FILE(&LIB/&FILE) MBR(&RETRY) NEWMBR(&PGMS)
0074              MONMSG     MSGID(CPF3178) EXEC(GOTO CMDLBL(CHKERR))
0075              CHGVAR     VAR(&MSG) VALUE('SOME PROGRAMS IN USE.')
0076              SNDPGMMSG  MSG(&MSG)
0077              CHGVAR     VAR(&MSG) VALUE('-- CALL QBNCVTPLST PARM(')
0078              CHGVAR     VAR(&MSG) VALUE(&MSG *TCAT &LIB *BCAT &FILE)
0079              CHGVAR     VAR(&MSG) VALUE(&MSG *TCAT ') WHEN PROGRAMS UNUSED.')
0080              SNDPGMMSG  MSG(&MSG)
0081
0082  CHKERR:
0083
0084 /* CHECK FOR ANY CONVERSION FAILURES */
0085              CHKOBJ     OBJ(&LIB/&FILE) OBJTYPE(*FILE) MBR(&ERRORS)
0086              MONMSG     MSGID(CPF9815) EXEC(GOTO CMDLBL(END))
0087              CHGVAR     VAR(&MSG) VALUE('CONVERSION ERRORS OCCURRED.')
0088              SNDPGMMSG  MSG(&MSG)
0089              CHGVAR     VAR(&MSG) VALUE('-- RUNQRY QRYFILE((')
0090              CHGVAR     VAR(&MSG) VALUE(&MSG *TCAT &LIB *TCAT '/')
0091              CHGVAR     VAR(&MSG) VALUE(&MSG *TCAT &FILE *BCAT &ERRORS)
0092              CHGVAR     VAR(&MSG) VALUE(&MSG *TCAT ')) TO DISPLAY LIST.')
0093              SNDPGMMSG  MSG(&MSG)
0094
0095  END:        ENDPGM

Input

Input to this program is the name of a file and library that contains the member PGMLIST, which has a list of objects to be converted. The member PGMLIST should be generated using the Display Object Description (DSPOBJD) command with the OUTPUT(*OUTFILE) parameter. Because the QBNCVTPGM tool converts only program, service program, and module objects, only *PGM, *SRVPGM, and *MODULE types should be included in the list. As an example, assume all the programs and modules in libraries ABC and DEF are to be converted. The following commands would create member PGMLIST in file MYFILE in library MYLIB and call this CL program.

/* Start PGMLIST with program and modules in ABC. */
DSPOBJD OBJ(ABC/*ALL) OBJTYPE(*PGM *MODULE) OUTPUT(*OUTFILE) OUTFILE(MYLIB/MYFILE)
        OUTMBR(PGMLIST)

/* Then add programs and modules in DEF. */
DSPOBJD OBJ(DEF/*ALL) OBJTYPE(*PGM *MODULE) OUTPUT(*OUTFILE) OUTFILE(MYLIB/MYFILE)
        OUTMBR(PGMLIST *ADD)

/* Verify the contents of the list. */
RUNQRY QRYFILE((MYLIB/MYFILE PGMLIST))

/* Call the QBNCVTPLST CL program. */
CALL PGM(QBNCVTPLST) PARM(MYLIB MYFILE)

How QBNCVTPLST works

The CL program defines the input file as having the same format as the output from the DSPOBJD command (lines 24-25). It then loops through each record of the file (each record is one object to convert) and calls the QBNCVTPGM tool to perform the conversion of the object (lines 35-39).

If message MCH3405 or message MCH5802 is generated, the object is considered to be in use. The object did not get converted. It is added to a RETRYLIST member in the input source file. That way it can be retried later when the object is no longer in use (lines 43-44 and 56-60).

If message CPF9899 is generated, the most likely reason for this failure is that the object did not have enough creation data available. If that is the case, it makes no sense to retry this object again because it will likely fail the conversion each time an attempt is made. Instead, it is added to an ERRORLIST member in the input source file (lines 48 and 62-66). Objects that are added to member ERRORLIST did not get converted.

On line 52, message CPFA0AE is monitored. This message is generated if an object cannot be found. Because the objects should be found in the input list (as the list was generated by the DSPOBJD command), these messages are just placed in the job log and processing continues.

Other escape messages generated by the QBNCVTPGM tool will not be monitored and will be presented as an inquiry message by the CL program. If another behavior is required, a new CL program based on this source can be created.

Output

If the QBNCVTPLST CL program is allowed to complete (and it is not cancelled), all objects in the PGMLIST member are processed. After that is done, the control branches to the label LOOPEND on line 68. When converting objects in the PGMLIST member, the program may have created a retry list in member RETRYLIST and a list of errors in member ERRORLIST. Because the retry list includes those objects that can still be converted, lines 70-80 copies the member RETRYLIST into member PGMLIST. Doing so allows the same call to QBNCVTPLST to be issued again after the objects are no longer in use.

The member ERRORLIST contains a list of objects that did not convert, likely due to missing creation data. If ERRORLIST is not empty, lines 82-93 print a command that can be used to easily view what is in the ERRORLIST member. Note that this list is cumulative. It is meant to be a list of all objects that did not convert from multiple runs of the CL program. If the ERRORLIST member needs to be cleared of previous entries, the following command can be issued:

RMVM FILE(MYLIB/MYFILE) MBR(ERRORLIST)

Sample usage of QBNCVTPLST

The following sequence of commands shows how QBNCVTPLST can be used. An attempt will be made to convert all programs starting with the letter B in library MHESSLER. First, create the list of programs in member PGMLIST in file MYFILE in library MYLIB:

DSPOBJD OBJ(MHESSLER/B*) OBJTYPE(*PGM) OUTPUT(*OUTFILE) OUTFILE(MYLIB/MYFILE)
        OUTMBR(PGMLIST)

Note: As this program runs, member PGMLIST will be changed. If the original list is to be kept around, make a copy of it before running the tool.

The list contains the following objects:

BLAZE
BLDCAROL
BLDCPS
BLDCVT
BLDLETTERS
BLD669385

In this example, program BLDCPS is currently in use (for example, another job is currently running the ILE program) and program BLAZE contains no creation data.

Now, invoke the QBNCVTPLST tool to start the conversion of objects listed in member PGMLIST:

CALL QBNCVTPLST PARM(MYLIB MYFILE)

When the tool completes, it generates the following messages:

SOME PROGRAMS IN USE.
-- CALL QBNCVTPLST PARM(MYLIB MYFILE) WHEN PROGRAMS UNUSED.
CONVERSION ERRORS OCCURRED.
-- RUNQRY QRYFILE((MYLIB/MYFILE ERRORLIST)) TO DISPLAY LIST.

As BLAZE contains no creation data, it is not converted and is placed into the ERRORLIST member. Because BLDCPS is currently in use, it is left in the member PGMLIST so that the command can be tried again when it is not in use. Later, after the BLDCPS program is no longer in use, the command can be run again:

CALL QBNCVTPLST PARM(MYLIB MYFILE)

This time, QBNCVTPLST generates the following message:

CONVERSION ERRORS OCCURRED.  
-- RUNQRY QRYFILE((MYLIB/MYFILE ERRORLIST)) TO DISPLAY LIST.

BLAZE is still in the member ERRORLIST from the first time the tool was run. It was not tried again because failing without creation data on the first invocation means it will fail again on the second invocation. Therefore, it is placed into member ERRORLIST rather than kept in PGMLIST. Because the second run did not generate the in use message for BLDCPS, there is nothing left to retry. Of all the programs that were initially listed to be converted (MHESSLER/B*), all were converted except those listed in the ERRORLIST member (MHESSLER/BLAZE).

How to get the tools

The QBNCVTPGM conversion tool can be obtained in releases IBM i 7.1 through 7.3 using PTFs. The PTF needed for each release is listed below. When the PTF is applied, the tool will be placed into the QSYS library.

Download a copy of the QBNCVTPLST CL program source, then place the code in a source physical file, for example, MYLIB/QCLSRC.QBNCVTPLST. If the source is to be used as is, a program object can be created in the MYLIB library that can be called as described in the Sample usage of QBNCVTPLST section by issuing the following command:

CRTCLPGM PGM(MYLIB/QBNCVTPLST) SRCFILE(MYLIB/QCLSRC)

Summary

QBNCVTPGM and QBNCVTPLST provide a simple way to force program conversion without using save/restore or change commands. One benefit of converting programs includes taking advantage of as many processor features as possible on the system on which the conversion takes place. Doing so may improve the overall performance of an application. Also, converting a program causes it to use the latest integrity enhancements available. This can make an application more secure. Program QBNCVTPGM is included with the operating system through a PTF. The source for the QBNCVTPLST CL program can be downloaded and customized as needed.

Learn more

• For information about IBM i, refer to the IBM i on Power Systems web page.
• For information about the following commands, see IBM Knowledge Center
  • CHGPGM
  • CHGSRVPGM
  • DSPMOD
  • DSPOBJD
  • DSPPGM
  • DSPSRVPGM
  • RSTOBJ
  • SAVOBJ
• The ILE Concepts manual contains a chapter on Advanced Optimization Techniques. This chapter discusses program conversion and performance profiling.
• Get the source code for the CL program QBNCVTPLST.