Docker, Kubernetes and Helm work together to provide a platform for managing, packaging and orchestrating containerised workloads. For App Connect Enterprise this enables the packaging of an integration server into a standardised unit for deployment that can be promoted through a development pipeline then deployed, managed and scaled.
IBM Cloudâ„˘ Private is a private cloud platform for developing and running workloads locally. It builds on Kubernetes and Helm technologies as well as providing a catalog, private image repository, management console and monitoring framework.
There are a number of key new features to this release:
– Dashboard: Management of BAR files and integration servers, no need to wait for integration server to start before deploying a BAR file
– Monitoring: Message flow statistics and JVM statistics data provided to the IBM Cloudâ„˘ Private monitoring dashboard
– Logging: Improved integration with the IBM Cloudâ„˘ Private ELK stack
– Configuration: Much greater control of the integration server through configuration
Deploying IBM App Connect Enterprise to IBM Cloud Private
The IBM Cloudâ„˘ Private catalog provides two components for IBM App Connect Enterprise:
– ibm-ace-dashboard-dev: Dashboard providing BAR file management and a UI to view and create integration servers
– ibm-ace-server-dev: This chart deploys a single IBM App Connect Enterprise integration server for running integrations
The following video demonstrates installing the dashboard, uploading BAR files and creating integration servers for running those BAR files that can be viewed and managed through the dashboard and IBM Cloudâ„˘ Private console.
The dashboard component has several functions:
1. Provides BAR file management for upload and storage
2. Creation of servers through the dashboard to run a managed BAR file – which it then serves to the integration server containers
3. Discovery and display of integration servers installed to the same namespace
When the dashboard is scaled the running pods may be scheduled onto different nodes. As such, the backing storage for the BAR files needs to support ReadWriteMany Access Mode (such as GlusterFS or NFS). Through the “BAR files” page within the dashboard you can upload BAR files that the dashboard will store.
When an integration server is created using the dashboard you are prompted to select a BAR file you have uploaded previously, or to upload a BAR file that the dashboard will store at that time. A “Content URL” is then provided on the screen that needs to be copied so it can be supplied as a parameter that will be used to create the integration server. The value needs to be posted into the field named “Content server URL” in the configuration page for the integration server. At runtime, when the container has been instantiated, but before the integration server has started, the BAR file will be retrieved using the “Content URL” and unpacked to the run directory of the integration server. When the integration server is subsequently started it reads the BAR file automatically. This takes away the need to deploy the BAR file once the integration server is started. In an agile integration architecture it is highly recommeded to avoid post start customisation of an integration server. Note that there is a limit of one BAR file per integration server when using the dashboard to provision BAR files in this way.
Installed integration servers are discovered within the same namespace by using the Kubernetes API to find deployment and stateful set resources that have “appconnectenterprise” and “serverName” annotations and a port named “webui”. A tile is then displayed to represent each discovered integration server. To be able to drill down into the integration server to view deployed applications, libraries etc. the dashboard uses the integration server REST API. For this it requires a defined webuser that has READ access named “ibm-ace-dashboard-viewer” to be configured – this is created by default for integration servers created through the dashboard.
Bring Your Own Server
For CI/CD pipelines users may prefer to build and promote images that embed the BAR files and other prereqs, removing the requirement to retrieve the BAR file when the container starts but still enable the integration server to be viewed and mananaged through the dashboard and IBM Cloudâ„˘ Private console. To achieve this:
1. Build your image based on the sample in https://github.com/ot4i/ace-docker.
2. Push your image to a Docker registry.
3. Install a helm release to the same namespace the dashboard is installed to in IBM Cloudâ„˘ Private, the https://github.com/ot4i/ace-helm provides a example to achieve this which sets the annotations and creates an appropriate user with READ access.
4. Open the dashboard.
IBM Cloudâ„˘ Private deploys an ELK stack, referred to as the management logging service, to collect and store all Docker-captured logs. App Connect Enterprise integration servers can log directly to standard out in the command shell from which they are started. From App Connect Enterprise v18.104.22.168 we have added a new setting which lets you specify that these log entries should be made in a JSON format (as opposed to the human readable option). These log entries are scraped and used to populate the logging service. This enables much more powerful searching and visualisation of logging data in support of the IBM App Connect Enterprise deployments.
By default the dashboard and integration servers log to stdout in JSON format, a sample Kibana dashboard is also available on Github that uses the logstash-* index to illustrate potential visualisations of the data that can be imported through Kibana:
The monitoring dashboard within IBM Cloudâ„˘ Private uses Grafana and Prometheus to present detailed data about your cluster nodes and containers. In addition to the system level data, component level data is essential to enable effective operational and administrative oversight of workloads. The accounting and statistics feature in IBM App Connect Enterprise provides the component level data with detailed insight into the running message flows to enabled problem determination, profiling, capacity planning, situation alert monitoring and charge-back modelling.
The message flow accounting and statistics data is made available to the monitoring stack within IBM Cloudâ„˘ Private enabling dashboards and alerting to be created around the data to support operations, development and administration.
By default the monitoring data is enabled for the integration server and Queue Manager (if present), and a sample Grafana dashboard is available on Github to illustrate potential visualisations of the data which can be imported through Grafana: