I work at IBM as a developer and I worked on the original project to put the enterprise capabilities of IBM Integration Bus v10 on the cloud as the IBM-hosted and managed service IBM® Integration Bus on Cloud. Recently, I have been working on putting the newly-released IBM App Connect Enterprise v11 Fix Pack 2 into IBM App Connect on IBM Cloud, the next-generation IBM-hosted and managed service, to provide the enterprise capabilities in the Lite plan and Custom Enterprise plan. Here are some of my reflections and thoughts about the experience I had using IBM App Connect Enterprise v11 compared to using IBM Integration Bus v10.
I have worked on and used many IBM integration products over the last 20 years, and it has been interesting to see how they have improved over time. In particular, the evolution of App Connect Enterprise(ACE) v11 from its preceding products (IIB, WMB, WMQI) has been an evolution from being large enterprise tools with complex installation and prereq requirements, like the need for a database and a queuing system to provide the most basic of function, to now a lightweight single executable that is as easy to run as nodejs.
Several years ago I used IBM Integration Bus(IIB) v10 to create a docker container that could run a single BAR file. IIB v10 had many enhancements over previous versions to drastically reduce the prereq requirements. At the time, this enabled me to create a container that only needed the IIB software and did not have any need for IBM MQ software or a connection to a database. There were two main challenges with creating this container:
- The first challenge was that IIB still had several running components that needed a working installation
- The second challenge was that it required quite a few configuration and deployment steps to get a single BAR file working.
These two challenges were never a big issue when running a large IIB node on-premises where many developers shared one installation on a very large and powerful machine. But trying to do the same thing where we wanted many separate small installations in containers was not a trivial thing to do. We used several techniques at the time to reduce the running IIB installation down to a single process, and also had quite sophisticated scripts to configure various parts of the installation to allow connectivity over different required transports.
I was able to achieve the hope that I had when moving to ACE v11, that I could remove most of this complexity and use proper built-in mechanisms to simplify running a BAR file with a specific environmental configuration. The first major improvement in ACE v11 was the ability to run it as a single standalone integration server, independent of an integration node. This fits perfectly with a docker container where it is normal for docker to contain and run only one major process. The steps to do this in a docker containerh are simple:
- Download tar file for ACE v11, untar it and run command to accept license
- Create a work directory to house the config for a running server:
- Start the integration server:
#>IntegrationServer --workdir /var/ace/workdir
I think that is about as easy as it is possible to make it! You can even miss out the step to create a work directory and let the directory get auto-created by the
IntegrationServer start command. This is very powerful as it allows ACE v11 to be run almost anywhere: in docker, on machines with other micro-services, or on a variety of cloud environments. It also allows ACE v11 to be used in whatever architecture that is required ranging from more traditional large installs to very fine-grained microservices architectures.
This is all great, but one of the hardest problems integration engines have over runtimes like nodejs or java is setting up the environment so that they can interact with many other systems over many different protocols. The very nature of being an integration engine means that it has to be configured for many different transports and protocols. In my case, I was using ACE v11 for the enterprise capabilities within IBM App Connect on IBM Cloud. In that case, it creates an integration server for each uploaded BAR file. It creates the container when starting the integration and destroys it when it stopped. It is treated very much like cattle rather than a pet. To achieve this means being able to quickly and easily set up the environment each time an integration server is started. I will not go into the details of what I had to do in v10, but it was not simple or clean. In ACE v11 it is very straightforward. The work directory contains all the environmental config files and all the integration flows that are required for the server to run. The environmental config files have two main types:
- The first type is a
server.conf.yamlfile in the root of the work directory. It contains details like the ports to use for HTTPS and the amount of memory to use for the JVM. In fact, it contains almost all the configuration required for running a server.
- The second type is the policy documents, which contain details about connecting to a particular end system. For example: there might be a policy document for connecting to a particular IBM MQ system queue manager.
The only commands that have to be run are to set up security identities like user name and passwords. Ideally, in the future, even this step can be replaced with a policy document or entries in the
--- # ACE Integration Server configuration file # # General notes : # - Integration Server will load server.conf.yaml from directory set via --work-dir # - To ensure valid YAML avoid any use of TAB characters # - File paths may be taken as absolute, or relative to the integration server's work directory # serverConfVersion: 1 defaultQueueManager: 'TEST1' # Set non-empty string to specify a default queue manager trace: none # choose 1 of : none|service|diagnostic traceSize: 1G # Set the service trace size traceNodeLevel: true # Enable or disable Trace Nodes forceServerHTTPS: true # force HTTPS on all HTTP/SOAP input nodes Log: consoleLog: true # Control writing BIP messages to standard out. Set to true or false, default is true. outputFormat: 'text' # Set the format for writing BIP messages ibmjson or text, default is text. RestAdminListener: port: 7600 # Set the Admin REST API Port for ACE Web UI and Toolkit or -1 to disable. Defaults to 7600. ResourceManagers: JVM: jvmMinHeapSize: 33554432 # minimum JVM heap size in bytes jvmMaxHeapSize: 268435456 # maximum JVM heap size in bytes jvmDebugPort: 0 # Set non-zero to activate JVM debug port for Toolkit debugging HTTPConnector: ListenerPort: 7800 # Set non-zero to set a specific port, defaults to 7800 HTTPSConnector: ListenerPort: 7843
The way the docker container I created for IBM App Connect Enterprise on IBM Cloud works is it:
- Creates the workdir.
- Makes REST calls to another microservice to get server.conf.yaml, a BAR file, and a few other config files.
- Puts the files in the workdir.
- Creates any security details.
- Starts the integration server.
The docker container also includes the client libraries for any transport that ACE v11 does not include the client for, like MQ or DB2.
The great thing with all of this is that the docker container is a completely blank canvas and just those few simple downloads are enough to completely configure it. It can be blown away at any point and recreated in seconds, which is ideal for a microservice running in the cloud. ACE v11 is so flexible and simple to use I am really excited to see how other customers use and embed it for their integration needs.
The enterprise capabilities of IBM App Connect on IBM Cloud now use ACE v11 as described above. This does not change too much how the product works but it has a few noticeable improvements:
- Everything is just quicker from uploading BAR files to starting integrations.
- Policies can be included in the BAR file rather than defined through the user interface.
- Any type of BAR file can be uploaded including BAR files without any applications or BAR files that include CMF flows (there is still a limitation on some of the node types supported).
To try this and other functions in IBM App Connect on IBM Cloud go to: Build on IBM Cloud for free with no time restrictions. The Lite plan is a no-charge plan to provide the essential functionality to connect cloud and on-premise applications, as automated event-action flows and RESTful APIs or enterprise integrations created in our most powerful Integrated Development Environment.
Here are some useful links on ACE v11: