Integration platform as a service: The next generation of ESB, Part 2
Taking enterprise integration to the next level
Integration platform as a service (IPaaS) is a new integration solution model that uses cloud computing technology. It is elastic and scalable and uses a consumption-based model to achieve major cost savings. An enterprise can achieve a significant return on investment by choosing this solution rather than using a corporate-owned data center. This consumption-based model is calculated on the actual usage, therefore it achieves further cost savings when the CPU, memory, storage, software, and integration services are idle.
The essence of the consumption-based model in IPaaS is that the service platform always knows who is using which services, for how long, and when. The primary external metric for the IPaaS consumption model is the use of IPaaS services. Since IPaaS offerings are pre-engineered to provide price smoothing across hardware variables, and also includes considerations for operational and application management support, all of the costs associated with IPaaS are rolled up to the IPaaS service level and associated with the consumption of those services.
IPaaS solutions can be architected to function in either a single-tenant or multi-tenant environment. In situations where a client is looking to deliver a multi-tenant SaaS-based solution to their customer, the underlying IPaaS integration architecture must support that. This is particularly true with inter-enterprise API designs. Even when the solution is clearly single-tenant, IPaaS services are typically enterprise level and must support a range of constituent lines of business and their trading partners. As a result, IPaaS supports multiple views to report on service consumption.
IPaaS service consumption is recorded and reported for two primary purposes:
- Monitoring overall capacity usage in business terms, thereby allowing proper forecasting of capacity
- Providing the ability of charge back or other cost recovery schemes based on the use of IPaaS services
To support this consumption-based model, IPaaS includes the technical capability to apply reporting analytics to business events, correlate participation in business events across multiple IPaaS components, and identify the complete business context around each business event.
Demand can vary for a number of reasons. For example:
- Seasonal or event-specific business patterns may drive spikes and valleys in production usage
- Test usage due to back-end changes, where major system of record enhancements require a full set of end-to-end acceptance and performance tests including IPaaS
IPaaS is specifically architected to expand and contract rapidly to changes in demand. This is attainable because the IPaaS environment builds on the features and functions of the IaaS and PaaS cloud-offerings, so there is no re-inventing these underlying services for IPaaS. An IPaaS service request for more capacity will automatically call on the underlying IaaS and PaaS cloud provisioning services. In addition, the IPaaS catalog has pre-engineered the provisioning request specifically to the IPaaS functional services. This is consistent with the overall philosophy of cloud architecture and open source initiatives like Cloud Foundry, Docker, and OpenStack. This IPaaS pre-engineering also provides for price smoothing across hardware variables, and also includes considerations for operational and application management support.
The services catalog is the key to understanding the richness of the IPaaS delivery model. In the past, the capability of a given ESB was defined as the set of transaction/messaging patterns that it implemented. Now with IPaaS, there is a more powerful approach. IPaaS is defined as a set of standardized and selectable business capabilities (i.e. services). It is helpful to think of these services in three classes:
- Functional: These services define the business capability domains that can be provided by IPaaS; for example, enterprise service bus (ESB), business process management, API management, managed file transfer, and electronic data interchange.
- Non-functional: These services enable the quality of service (QOS) attributes of the IPaaS functional services. For example, a customer may select an ESB service from the catalog, but may have different QOS characteristics in a production instance than in an instance that’s used for testing. Non-functional services allow the customer to select levels of services applicability to their specific QOS requirements (such as high availability, continuous availability, response time, and throughput rate).
- Lifecycle: These are the underlying create/read/update/delete (CRUD) services needed to provide on-demand scalability within IPaaS. For example, it is possible to add additional nodes under an IPaaS functional service and then later remove them or relocate them to another service center.
From this perspective, the IPaaS services catalog can be seen as a three-dimensional model of offerings.
Figure 1. Service classes within the IPaaS services catalog
Another important consideration of the IPaaS services catalog is that the necessary hardware, software, configurations, topologies, and support are standardized and predefined. This means that there is no new development effort needed to stand up a service from the IPaaS services catalog. It also means that there are no compatibility issues between services in the IPaaS services catalog that might lead to a runtime discovery of a mismatch of IPaaS services.
PureApplication System and IPaaS
On the IPaaS platform, using patterns is the key to easily deploying and moving both the infrastructure platform (such as business process management and ESB) and the enterprise applications. By capturing the many steps of platform environment configuration and deployment into a defined pattern, IPaaS can automate the complexity of the platform architecture and deployment model in an easily deployable form. This pre-built pattern based clone approach reduces time, costs, and risk by taking human tasks and errors out of the equation. Once a pattern is created, it can then be used across off-premises and on-premises cloud environments without having to readjust for each environment—providing seamless portability. Pattern-based approaches enable the service to then be added to the service catalog in IPaaS and to ensure compatibility with other components in IPaaS.
While patterns simplify application configuration and portability across hybrid cloud options, a pre-integrated and automated application environment is also required to simplify and reduce the application lifecycle. With IBM® PureApplication® solutions, the monitoring and lifecycle management of the deployed pattern is automated according to the various policies and service-level requirements contained in the pattern. The underlying infrastructure and platform requirements of the application are dynamically adjusted to meet those service levels.
In particular, IBM Business Process Manager Pattern V8.5.5 offers a pattern configuration of business process management and ESB that is optimized to work with IBM PureApplication Service on SoftLayer.
IBM Virtual Cloud Servers (previously announced as IBM Business Process Manager Application Pattern) are optimized to help accelerate setting up and managing complex, highly available business process environments with the following capabilities:
- Introducing elastic scaling policies to automatically detect when configuration capacity should be increased to meet increasing demands on your process solutions
- Facilitating streamlined setup and management of complex, highly available IPaaS configurations that are optimized for development, test, or production use
- Enabling critical resources that focus more on value-add activities and less on installation, configuration, and management
- Helping accelerate the expansion of business process management from a project to a program across an enterprise
IBM SoftLayer provides off-premises cloud infrastructure services for the other piece of a dynamic hybrid environment. PureApplication Service on SoftLayer lets clients automate and manage off-premises SoftLayer infrastructure services for client enterprise applications simply and easily. PureApplication Service on SoftLayer runs in an isolated infrastructure environment (compute, storage, and networking) within SoftLayer to keep IPaaS workloads safe.
As a result, PureApplication Service on SoftLayer was introduced for the IPaaS framework, with the ability to expand business possibilities. The cloud can be an incubator of ideas that help organizations continually test, adjust, and deploy new innovations more nimbly than ever before.
IBM PureApplication Service on SoftLayer provides tools and scripts to easily manage the IPaaS pattern and continuous delivery (provisioning, deployment, release control, versioning control, and integration underneath applications). The following three figures illustrate the capabilities of the PureApplication Service on SoftLayer.
Figure 2. Easy-to–deploy, pattern-based application
Figure 3. Elastic scaling service: Pattern-based workload control
Figure 4. Elastic scaling: Setting up dynamic auto-scaling
All IBM products in this IPaaS architecture (ESB, business process management, and connectors) are designed to support clusters of both application servers and database servers for increased scalability and availability. As the transaction volume and user population grows, additional server instances can be added dynamically to the IPaaS to support the increased load.
IPaaS solutions are architected to scale up to support the largest deployments in the world, providing enterprise integration solutions across all industries. IBM customers have successfully implemented enterprise integration and BPM solutions for tens of thousands of users and hundreds of process flows, based on this expertise. IBM developed a roadmap for BPM adoption, from a quick-win pilot to enterprise-wide use. In addition, governance and control over all processes, process artifacts, governance, and deployment can be maintained at all levels, facilitating compliance with internal or external requirements or mandates.
Challenges and business considerations
The challenges for implementing IPaaS require a strong commitment for support from C-suite executives, as well as the effort required to migrate the platform and the new services model. But there are numerous benefits for enterprises that adopt this new IPaaS model as their integration solution, such as cost saving, flexibility, elasticity, standardization, and higher quality of services.
The business consideration can be the security and compliance regulations: SOX, FDA, HIPPA, ICS410 and others need to be implemented either on premise or as a private/public cloud solution. However, cloud-based solutions will require more careful auditing and certification to ensure that all business requirements are met.
Security and networking
As with any cloud deployment, security and networking can be major concerns. IPaaS is built on an underlying IaaS framework, which provides basic protection against denial of service and penetration attacks. There is often a perception that private clouds offer increased security — this is not necessarily the case. Commercial cloud providers have a mission-critical interest in security and often have more resources to invest in both shaping and complying with the latest security standards than do on-premise implementations. Cloud security is far from being a wild frontier, and there are a number of international standards related to cloud security, including:
- ISO 27000 series of security and privacy standards
- ISO 27001 & ISO 27002 – the foundations for IT security cloud computing impact on security and privacy
- ISO 27017 – security for cloud services
- ISO 27018 – data protection for cloud services (i.e. privacy)
IPaaS implementations often involve a hybrid cloud with the systems of record typically being located in a customer premise location or private cloud while the systems of engagement and systems of interaction are distributed over some mix of public and private clouds. VPN-type connections between these components provide a level of security over the physical security provided by the network transport.
From an enterprise integration standpoint, it is important to ensure that connection data back to an application hosted in the organization is protected as well. SSL communication — especially two-way SSL communication done using certificates, even over a VPN — adds another layer of protection for application communication with and between IPaaS components. Certificates used should not have long lifetimes (months to a year at most). Self-signed certificates and custom trust stores should be used to limit the impact of compromised certificates. And make sure that the certificate authority is not also on the cloud.
In addition, standards-based, message-level security is built into the IPaaS components. These are based on the web service interoperability (WS-I) standards. The following significant standards are related to message-level security and integrity:
- WS-Secure Conversation
Taken together, these standards provide IPaaS solutions with three levels of security (infrastructure, component, and message) that are both standards-based and engineered into the fabric of the IPaaS components and infrastructure.
Monitoring and auditing
IBM Cloud Virtual Servers (formerly Business Monitor) provides real-time business activity monitoring for your business processes, regardless of where those processes execute. In IPaaS solutions, business activity monitoring (BAM) can visualize overall business performance generated by BPM applications, ESB services, and enterprise applications such as SAP or Oracle. Monitoring dashboards can display data at both the individual instance level and the aggregate level, and offer sophisticated dimensional analysis. This monitoring data can also be used to audit events, tracking who is requesting the service and when.
At its core, IBM Business Monitor processes XML representing business events, extracts metric data of business relevance, and stores it to a database. Dashboards query the database to display information that’s relevant to a business user. Regardless of how an application is implemented (whether in Java®, C++, COBOL, or otherwise) and regardless of where it runs (whether on a mobile device, a personal computer, or a mainframe), as long as it can provide such XML to IBM Business Monitor, the application can enjoy the benefits of business activity monitoring.
Figure 5. IBM Business Monitor in IPaaS
The following IPaaS components are enabled for automatic event emission; developers can just check a box and the events will flow without any coding whatsoever:
- IBM business process management
- IBM Integration Bus
- IBM application adaptors (SAP, Oracle, JDBC, IBM MQ)
Figure 6. IPaaS integrated business event reporting
For example, events from a SAP online catalog order application that processes orders placed by customers can now be monitored directly by IBM Business Monitor without having to generate or implement mediation modules. The adapter for SAP software listens for order placement activity in the SAP application and forwards events with business payloads to IBM Business Monitor. All of the relevant business activity by customers to place orders occurs in the IPaaS and is monitored by IBM Business Monitor. Here are some of the key performance indicators that you can track for your application running in the IPaaS:
- Total number of orders processed
- Total price of all orders placed
- Average price of all orders placed
Average order processing time
Figure 7. Integration of business application events into IPaaS
In addition, IBM Control Center (formerly Sterling Control Center) can emit events to IBM Business Monitor describing the file transfer activity occurring in many Sterling-branded products. This enables your business users to gain visibility into such activity and to manage Services Level Agreements (SLA) associated with such activity.
Performance and SLA
IPaaS is envisioned as a turn-key approach to enterprise integration. IPaaS comes with pre-engineered patterns and configurations. Specific capabilities and capacities can be called up on-demand from the IPaaS service catalog. As a result, the IPaaS user expects assurances with respect to overall performance and SLAs.
Intrinsic to IPaaS is the ability to process high transaction volumes with minimal latency. Under the covers, IPaaS has multiple components which can be specifically optimized for different kinds of processing. For example, EDI translation and content-based message routing can be processed by separate components, each one optimized for its own purpose.
Architecturally, IPaaS makes extensive use of clustering for both availability redundancy and workload distribution. Underneath IPaaS are the cloud technologies of IaaS and SaaS which also intrinsically support availability redundancy and workload distribution.
The net result is a high degree of probability that the IPaaS service listed in the service catalog has been carefully engineered to perform at or exceed the SLA listed in the catalog.
Potential market: Watson, predictive analytics, and IPaaS
In the IPaaS model, systems of engagement send transactions to systems of record via systems of interaction. IPaaS itself is a component of the systems of interaction. In older models of computing, systems of record processed data, but their use cases seldom included analysis patterns of transactions. This capability was left to business analytics (i.e. systems of insight) and was often performed in a post-processing mode through some form of data warehouse.
With the emergence of the Internet of Things, dynamic traffic pattern relationships become more critical. They must be correlated in real time and the traffic data may span the domains of multiple systems of record. This is where IPaaS adds value as the real-time source for consolidated business events. Unlike earlier models of enterprise integration, IPaaS is highly aware of the context around business events and can be the source for predictive analytics. Fraud detection is one area that is rapidly expanding its use of real-time transaction data from multiple sources. However, there are many more potential applications of real-time predictive analytics. For example, correlating and even re-targeting social media campaigns in real time based on observation of IPaaS traffic patterns is an area that contains many use cases.
Certainly one of the things that IPaaS does is open up the enterprise to using and offering API services. In this case, IPaaS itself can be a user of API services; for example, calling a Watson API to analyze metadata on its own traffic patterns.
This article explored the topic of how IPaaS goes beyond the original service-oriented architecture notion of the ESB to provide a full range of enterprise integration capabilities. It discussed how the demand for these additional enterprise integration capabilities is being driven by technology-enabled changes in the market (such as cloud, Internet of Things, API economy). Most significantly, it laid out the case that leveraging underlying cloud technology and taking a service approach to IPaaS delivery now makes these enhanced enterprise integration capabilities readily accessible from the perspective of cost, implementation, and ongoing support.