Simplified program conversion with QBNCVTPGM and QBNCVTPLST
Tools for converting programs and modules on IBM i
Introduction to program conversion
Taking advantage of new processor features may improve the overall performance of an application and using the latest integrity enhancements can make an application more secure. A way to include those improvements in an existing program is to convert it. Introduced in IBM i release V3R6, program conversion rebuilds the program to take advantage of the latest processor features and integrity enhancements. However, program conversion happens automatically only when it is needed. Therefore, programs may be running on an IBM i system that have not been converted to incorporate the latest processor features and integrity enhancements.
Program conversion happens automatically if a program is moved to another IBM i system that does not support all of the processor features used by the program. For example, to implement a sequence of instructions, a program may use hardware features available on the current processor. If this program is moved to an earlier version of the processor that does not support those specific hardware features, the program must be converted so those features can be implemented in a different way (but still correctly). Note that automatic conversion does not happen if moving a program in the other direction. If a program runs correctly on an earlier version of the processor, it will run correctly on a later processor due to forward compatibility. See Figure 1. Automatic conversion example.
Figure 1. Automatic conversion example
Conversion can also happen if it is manually forced using one of the following mechanisms:
Using the Change Program (
CHGPGM) or Change Service Program (
CHGSRVPGM) commands with the force create option (
FRCCRT(*YES)) if the program contains sufficient creation data. To check if there is enough creation data, use the Display Program (
DSPPGM) or Display Service Program (
DSPSRVPGM) commands. For Integrated Language Environment (ILE) programs or service programs, a value of *YES for All creation data indicates that enough creation data is available. For Original Program Model (
OPM) programs, a value of *ALL for Observable information indicates that enough creation data is available. To check modules, the Display Module (
DSPMOD) command can be used. A value of *YES for Creation data indicates that enough creation data is available.
Saving the program or service program using the Save Object (
SAVOBJ) command and restoring it using the Restore Object (
RSTOBJ) command with the force object conversion option (
Here are some conditions where manual conversion would be useful:
Programs are being moved to a new processor. Because processors are forward compatible, moving a program from an older processor will not cause automatic conversion. Yet, the new processor may have additional features that the optimizing translator can take advantage of to make the program more efficient.
A new release of the operating system has been installed. The optimizing translator may be able to improve the efficiency of the program due to enhancements that were made. Also, new integrity enhancements may have been made to the optimizing translator, making the application more secure.
Integrity PTFs have been installed which may affect the generated machine code of the optimizing translator. These code generation changes may affect the security of the application being converted.
Introduction to the QBNCVTPGM and QBNCVTPLST tools
Starting in IBM i release 7.1, new tools are available to easily convert existing programs. They are the QBNCVTPGM and QBNCVTPLST tools. These tools can be used to convert objects even if the program does not have the required creation data needed for the change commands (see the Restrictions section). QBNCVTPGM is included in the operating system through a program temporary fix (PTF).
|IBM i 7.1||5770SS1-SI69751|
|IBM i 7.2||5770SS1-SI69748|
|IBM i 7.3||5770SS1-SI69742|
QBNCVTPLST is available for download. See the How to get the tools section.
Note that use of these new tools is not required for the correct running of an application; however, using them can provide the advantages described previously.
The QBNCVTPGM tool is installed with a PTF. It is used to perform conversion of a single program, service program, or module (type *PGM, *SRVPGM, or *MODULE).
The syntax for invoking the tool is:
CALL PGM(QSYS/QBNCVTPGM) PARM(object library type)
- object is the name of a single object to be converted.
- library is the name of a single library containing the object.
- type is one of the following: *PGM, *SRVPGM, or *MODULE.
For example, to convert program
BLDCPS in library
QGPL, use the following code:
CALL PGM(QSYS/QBNCVTPGM) PARM(BLDCPS QGPL *PGM)
In order for the QBNCVTPGM tool to convert the program, service program, or module, the following conditions must be true:
The object must be based on the IBM i machine interface (MI). This means any program that runs in the PASE environment cannot be converted with this tool.
The MI created object must contain creation data. To check individual programs, use the display commands described in the Introduction section. The object has enough creation data if *YES, *ALL, or *UNOBS is shown. If the object does not contain the appropriate level of creation data, message CPF9899 will be generated by the tool and the object will not be converted.
IBM objects included in the QSYS library do not contain creation data and cannot be converted.
The object cannot be in use or locked by another job.
The profile running this tool must have the *ALLOBJ and *SECADM special authorities, plus *EXECUTE authority to the library containing the object.
An OPM program that resides in the QTEMP library cannot be converted.
Any escape messages are signaled back to the caller, so they have an awareness of how the conversion progressed. The following table lists the escape messages and description. Viewing all previous messages in the job log will indicate the actual error that occurred.
|CEE9901||A catch-all error returned when something unexpected happened.|
|CPF0001||The wrong number of parameters were passed to the tool or the wrong object type was passed to the tool.|
|CPF5CA4, CPF9872||Parameter validation errors occurred.|
|CPF9899||The conversion of the object failed, likely due to the object not having enough creation data.|
|CPFA0AE||The object or library cannot be found.|
|CPFB9B4||The OPM object could not be duplicated into the QTEMP library so it cannot be converted.|
|CPFBB38||The OPM object cannot reside in the QTEMP library.|
|MCH3405, MCH5802||The program or service program is currently in use or locked by another job.|
Generally, the caller of this tool should perform the following tasks:
If MCH3405 or MCH5802 is received, the object could not be converted because it is currently in use or locked by another job. The conversion can be tried again once the object is no longer in use or the lock is released.
If CPF9899 is received, the conversion has failed. The job log will contain additional messages describing the details of the failure. There is little sense in trying the conversion again; it will likely fail again.
If any other errors occur, it is up to the client to decide what to do.
Possible side effects
When an object is converted, the processor features of the current system are used when building the program. Using these new features may improve the overall performance of the application. In addition, any of the latest integrity enhancements will be included in the converted program.
Converting a program can have the following effects:
Objects that are digitally signed will lose their digital signatures. An object is digitally signed if YES is displayed for Digitally Signed in the Display Object Description (DSPOBJD) command, when using the DETAIL(*FULL) option.
If a program was previously performance profiled, it may lose some performance benefits it previously obtained from profiling. A program is performance profiled if the DSPPGM or DSPSRVPGM commands show *APYALL, *APYBLKORD, or *APYPRCORD for Profiling data. This change can affect the performance of the object.
If a program contains debug breakpoints and undergoes conversion, any breakpoints are removed from the program. Therefore, it is possible for another job to set breakpoints and then have the breakpoints removed. When that happens, message CPD190A (breakpoints, step, or statement traces changed) is sent to the jobs that had set breakpoints in the program.
The QBNCVTPGM program is not thread safe; therefore, if multiple instances of this program are to be run simultaneously, each instance should be run in a different job, not in different threads in the same job. Running in separate jobs will ensure that each instance of the program will run independently.
Using the QBNCVTPLST tool
To provide an example of how the QBNCVTPGM tool can be used, the QBNCVTPLST CL program was developed. The source code for the program is provided below. It can be used as is or modified as needed. Also, see the How to get the tools section to download it.
Listing 1. Source for the QBNCVTPLST CL Program
LINE 0001 PGM PARM(&LIB &FILE) 0002 0003 /* INPUT/OUTPUT FILE - MEMBERS ARE CREATED BY DSPOBJD */ 0004 /* OUTPUT(*OUTFILE) */ 0005 DCL VAR(&LIB) TYPE(*CHAR) LEN(10) 0006 DCL VAR(&FILE) TYPE(*CHAR) LEN(10) 0007 0008 /* MEMBER NAMES IN &LIB/&FILE */ 0009 /* PGMLIST (INPUT/OUTPUT) - LIST OF PROGRAMS TO CONVERT */ 0010 /* RETRYLIST (TEMP) - IN-USE PROGRAMS FOR LATER RETRY */ 0011 /* ERRORLIST (OUTPUT) - FAILED PROGRAMS */ 0012 DCL VAR(&PGMS) TYPE(*CHAR) LEN(10) + 0013 VALUE('PGMLIST') 0014 DCL VAR(&RETRY) TYPE(*CHAR) LEN(10) + 0015 VALUE('RETRYLIST') 0016 DCL VAR(&ERRORS) TYPE(*CHAR) LEN(10) + 0017 VALUE('ERRORLIST') 0018 0019 /* WORK VARIABLE FOR CONSTRUCTING MESSAGES */ 0020 DCL VAR(&MSG) TYPE(*CHAR) LEN(300) 0021 0022 /* DECLARE THE INPUT FILE (LIST OF PROGRAMS) AND OVERRIDE IT TO */ 0023 /* USE THE SPECIFIED &LIB/&FILE */ 0024 DCLF FILE(QSYS/QADSPOBJ) 0025 OVRDBF FILE(QADSPOBJ) TOFILE(&LIB/&FILE) + 0026 MBR(&PGMS) 0027 0028 /* REMOVE ANY EXISTING MEMBER FOR THE RETRY LIST */ 0029 RMVM FILE(&LIB/&FILE) MBR(&RETRY) 0030 MONMSG MSGID(CPF7310) 0031 0032 /* ERROR LIST WILL REMAIN UNCHANGED - IT ACCUMULATES ERRORS FROM */ 0033 /* EACH INVOCATION OF QBNCVTPLST */ 0034 0035 LOOP: RCVF 0036 MONMSG MSGID(CPF0864) EXEC(GOTO CMDLBL(LOOPEND)) 0037 0038 CALL PGM(QSYS/QBNCVTPGM) + 0039 PARM(&ODOBNM &ODLBNM &ODOBTP) 0040 0041 /* MONITOR FOR "PROGRAM IN USE" EXCEPTIONS, ADD TO RETRY LIST, */ 0042 /* CONTINUE */ 0043 MONMSG MSGID(MCH3405 MCH5802) + 0044 EXEC(GOTO CMDLBL(ADDRETRY)) 0045 0046 /* MONITOR FOR "PROGRAM CONVERSION FAILED" EXCEPTION, ADD TO */ 0047 /* ERROR LIST, CONTINUE */ 0048 MONMSG MSGID(CPF9899) EXEC(GOTO CMDLBL(ADDERROR)) 0049 0050 /* MONITOR FOR ANY EXCEPTIONS THAT SHOULD JUST BE LOGGED AND */ 0051 /* IGNORED, CONTINUE */ 0052 MONMSG MSGID(CPFA0AE) /* IGNORE DELETED OBJECTS */ 0053 0054 GOTO CMDLBL(LOOP) 0055 0056 ADDRETRY: DSPOBJD OBJ(&ODLBNM/&ODOBNM) OBJTYPE(&ODOBTP) + 0057 OUTPUT(*OUTFILE) OUTFILE(&LIB/&FILE) + 0058 OUTMBR(&RETRY *ADD) 0059 MONMSG MSGID(CPF2114) 0060 GOTO CMDLBL(LOOP) 0061 0062 ADDERROR: DSPOBJD OBJ(&ODLBNM/&ODOBNM) OBJTYPE(&ODOBTP) + 0063 OUTPUT(*OUTFILE) OUTFILE(&LIB/&FILE) + 0064 OUTMBR(&ERRORS *ADD) 0065 MONMSG MSGID(CPF2114) 0066 GOTO CMDLBL(LOOP) 0067 0068 LOOPEND: 0069 0070 /* REPLACE MEMBER PGMLIST WITH MEMBER RETRYLIST FOR A LATER */ 0071 /* ATTEMPT AT CONVERTING IN-USE PROGRAMS */ 0072 RMVM FILE(&LIB/&FILE) MBR(&PGMS) 0073 RNMM FILE(&LIB/&FILE) MBR(&RETRY) NEWMBR(&PGMS) 0074 MONMSG MSGID(CPF3178) EXEC(GOTO CMDLBL(CHKERR)) 0075 CHGVAR VAR(&MSG) VALUE('SOME PROGRAMS IN USE.') 0076 SNDPGMMSG MSG(&MSG) 0077 CHGVAR VAR(&MSG) VALUE('-- CALL QBNCVTPLST PARM(') 0078 CHGVAR VAR(&MSG) VALUE(&MSG *TCAT &LIB *BCAT &FILE) 0079 CHGVAR VAR(&MSG) VALUE(&MSG *TCAT ') WHEN PROGRAMS UNUSED.') 0080 SNDPGMMSG MSG(&MSG) 0081 0082 CHKERR: 0083 0084 /* CHECK FOR ANY CONVERSION FAILURES */ 0085 CHKOBJ OBJ(&LIB/&FILE) OBJTYPE(*FILE) MBR(&ERRORS) 0086 MONMSG MSGID(CPF9815) EXEC(GOTO CMDLBL(END)) 0087 CHGVAR VAR(&MSG) VALUE('CONVERSION ERRORS OCCURRED.') 0088 SNDPGMMSG MSG(&MSG) 0089 CHGVAR VAR(&MSG) VALUE('-- RUNQRY QRYFILE((') 0090 CHGVAR VAR(&MSG) VALUE(&MSG *TCAT &LIB *TCAT '/') 0091 CHGVAR VAR(&MSG) VALUE(&MSG *TCAT &FILE *BCAT &ERRORS) 0092 CHGVAR VAR(&MSG) VALUE(&MSG *TCAT ')) TO DISPLAY LIST.') 0093 SNDPGMMSG MSG(&MSG) 0094 0095 END: ENDPGM
Input to this program is the name of a file and library that contains the member PGMLIST, which has a list of objects to be converted. The member PGMLIST should be generated using the Display Object Description (DSPOBJD) command with the OUTPUT(*OUTFILE) parameter. Because the QBNCVTPGM tool converts only program, service program, and module objects, only *PGM, *SRVPGM, and *MODULE types should be included in the list. As an example, assume all the programs and modules in libraries ABC and DEF are to be converted. The following commands would create member PGMLIST in file MYFILE in library MYLIB and call this CL program.
/* Start PGMLIST with program and modules in ABC. */ DSPOBJD OBJ(ABC/*ALL) OBJTYPE(*PGM *MODULE) OUTPUT(*OUTFILE) OUTFILE(MYLIB/MYFILE) OUTMBR(PGMLIST) /* Then add programs and modules in DEF. */ DSPOBJD OBJ(DEF/*ALL) OBJTYPE(*PGM *MODULE) OUTPUT(*OUTFILE) OUTFILE(MYLIB/MYFILE) OUTMBR(PGMLIST *ADD) /* Verify the contents of the list. */ RUNQRY QRYFILE((MYLIB/MYFILE PGMLIST)) /* Call the QBNCVTPLST CL program. */ CALL PGM(QBNCVTPLST) PARM(MYLIB MYFILE)
How QBNCVTPLST works
The CL program defines the input file as having the same format as the output from the DSPOBJD command (lines 24-25). It then loops through each record of the file (each record is one object to convert) and calls the QBNCVTPGM tool to perform the conversion of the object (lines 35-39).
If message MCH3405 or message MCH5802 is generated, the object is considered to be in use. The object did not get converted. It is added to a RETRYLIST member in the input source file. That way it can be retried later when the object is no longer in use (lines 43-44 and 56-60).
If message CPF9899 is generated, the most likely reason for this failure is that the object did not have enough creation data available. If that is the case, it makes no sense to retry this object again because it will likely fail the conversion each time an attempt is made. Instead, it is added to an ERRORLIST member in the input source file (lines 48 and 62-66). Objects that are added to member ERRORLIST did not get converted.
On line 52, message CPFA0AE is monitored. This message is generated if an object cannot be found. Because the objects should be found in the input list (as the list was generated by the DSPOBJD command), these messages are just placed in the job log and processing continues.
Other escape messages generated by the QBNCVTPGM tool will not be monitored and will be presented as an inquiry message by the CL program. If another behavior is required, a new CL program based on this source can be created.
If the QBNCVTPLST CL program is allowed to complete (and it is not cancelled), all objects in the PGMLIST member are processed. After that is done, the control branches to the label LOOPEND on line 68. When converting objects in the PGMLIST member, the program may have created a retry list in member RETRYLIST and a list of errors in member ERRORLIST. Because the retry list includes those objects that can still be converted, lines 70-80 copies the member RETRYLIST into member PGMLIST. Doing so allows the same call to QBNCVTPLST to be issued again after the objects are no longer in use.
The member ERRORLIST contains a list of objects that did not convert, likely due to missing creation data. If ERRORLIST is not empty, lines 82-93 print a command that can be used to easily view what is in the ERRORLIST member. Note that this list is cumulative. It is meant to be a list of all objects that did not convert from multiple runs of the CL program. If the ERRORLIST member needs to be cleared of previous entries, the following command can be issued:
RMVM FILE(MYLIB/MYFILE) MBR(ERRORLIST)
Sample usage of QBNCVTPLST
The following sequence of commands shows how QBNCVTPLST can be used. An attempt will be made to convert all programs starting with the letter B in library MHESSLER. First, create the list of programs in member PGMLIST in file MYFILE in library MYLIB:
DSPOBJD OBJ(MHESSLER/B*) OBJTYPE(*PGM) OUTPUT(*OUTFILE) OUTFILE(MYLIB/MYFILE) OUTMBR(PGMLIST)
Note: As this program runs, member PGMLIST will be changed. If the original list is to be kept around, make a copy of it before running the tool.
The list contains the following objects:
In this example, program BLDCPS is currently in use (for example, another job is currently running the ILE program) and program BLAZE contains no creation data.
Now, invoke the QBNCVTPLST tool to start the conversion of objects listed in member PGMLIST:
CALL QBNCVTPLST PARM(MYLIB MYFILE)
When the tool completes, it generates the following messages:
SOME PROGRAMS IN USE. -- CALL QBNCVTPLST PARM(MYLIB MYFILE) WHEN PROGRAMS UNUSED. CONVERSION ERRORS OCCURRED. -- RUNQRY QRYFILE((MYLIB/MYFILE ERRORLIST)) TO DISPLAY LIST.
As BLAZE contains no creation data, it is not converted and is placed into the ERRORLIST member. Because BLDCPS is currently in use, it is left in the member PGMLIST so that the command can be tried again when it is not in use. Later, after the BLDCPS program is no longer in use, the command can be run again:
CALL QBNCVTPLST PARM(MYLIB MYFILE)
This time, QBNCVTPLST generates the following message:
CONVERSION ERRORS OCCURRED. -- RUNQRY QRYFILE((MYLIB/MYFILE ERRORLIST)) TO DISPLAY LIST.
BLAZE is still in the member ERRORLIST from the first time the tool was run. It was not tried again because failing without creation data on the first invocation means it will fail again on the second invocation. Therefore, it is placed into member ERRORLIST rather than kept in PGMLIST. Because the second run did not generate the in use message for BLDCPS, there is nothing left to retry. Of all the programs that were initially listed to be converted (MHESSLER/B*), all were converted except those listed in the ERRORLIST member (MHESSLER/BLAZE).
How to get the tools
The QBNCVTPGM conversion tool can be obtained in releases IBM i 7.1 through 7.3 using PTFs. The PTF needed for each release is listed below. When the PTF is applied, the tool will be placed into the QSYS library.
|IBM i 7.1||5770SS1-SI69751|
|IBM i 7.2||5770SS1-SI69748|
|IBM i 7.3||5770SS1-SI69742|
Download a copy of the QBNCVTPLST program source, then place the code in a source physical file, for example,
MYLIB/QCLSRC.QBNCVTPLST. If the source is to be used as is, a program object can be created in the MYLIB library that can be called as described in the Sample usage of QBNCVTPLST section by issuing the following command:
CRTCLPGM PGM(MYLIB/QBNCVTPLST) SRCFILE(MYLIB/QCLSRC)
QBNCVTPGM and QBNCVTPLST provide a simple way to force program conversion without using save/restore or change commands. One benefit of converting programs includes taking advantage of as many processor features as possible on the system on which the conversion takes place. Doing so may improve the overall performance of an application. Also, converting a program causes it to use the latest integrity enhancements available. This can make an application more secure. Program QBNCVTPGM is included with the operating system through a PTF. The source for the QBNCVTPLST CL program can be downloaded and customized as needed.
- For information about IBM i, refer to the IBM i on Power Systems web page.
- For information about the following commands, see IBM Knowledge Center
- The ILE Concepts manual contains a chapter on Advanced Optimization Techniques. This chapter discusses program conversion and performance profiling.
- Get the source code for the CL program QBNCVTPLST.