• APIs enable you to bring new digital services to market quickly
• A traditional Registry and Repository does not meet the need for API Management
• IBM API Connect enables management and creation of APIs
• You can move from a traditional Registry and Repository to API Connect for some scenarios
The business need for an API Management solution
Application programming interfaces (API) are a digital glue that links services, applications, sensors and mobile devices to create compelling customer experiences and help businesses tap into new market opportunities. They allow you to bring new digital services to market, open revenue channels and exceed customer expectations.
API Management is motivated by the adoption of new channels such as mobile apps and Internet of Things (IoT). API Management is consumer-centric. API Management tracks to the level of the individual end user of an API. An API Management solution provides a Portal where developers can discover and consume APIs. An API Management solution enables you to:
• Securely expose systems of record via a Gateway to Applications for Mobile, IoT, or any third party requirement.
• Publish APIs to expand brand reach – Tap into developer & partner ecosystems
• Enable new business – Monetize existing and new data & algorithms
Flavors of API
APIs can be split into two flavors of API: System APIs and Interaction APIs.
System APIs are APIs that pass through data from a system of record unchanged.
Interaction APIs invoke one or more System API’s or data sources, and manipulate the returned data with new logic. Interaction APIs promote reuse across new applications.
Companies already have web services, which can be exposed as APIs. Software vendors are adding API support to existing products to pass through data from a system of record unchanged (i.e., System APIs). System APIs should be managed and secured to protect enterprise systems. An API can be managed in terms of where they are published, and which developers can discover them. An API can be secured by controlling which applications can access them, how frequently and at what cost.
Why are Create & Run Important?
Digital Business relies on enterprise grade APIs. Existing system APIs are invoked and their results are manipulated to create new, reusable “Interaction APIs”.
APIs face web scale demands of the digital economy, with millions of requests per month; a proven runtime approach is needed to meet these demands.
Why Registry & Repository Solutions Don’t Meet the Need
A traditional Registry and Repository solution is a system for storing, accessing, and managing service metadata. This is used in selection, invocation, management, governance, and reuse of services in an SOA.
In other words, it is where you store information about services residing in your systems or other organizations’ systems that you already use, plan to use, or want to be aware of. A Registry and Repository is the single source of truth for services in your organization.
A registry should store information about services, such as their interfaces, operations, and parameters, and a metadata repository should provide a robust, extensible framework to accommodate the diverse nature of service usage.
A Registry and Repository solution should have management and governance capabilities that allow users to get the most business value from your SOA. A Registry and Repository may provide control over the design time and development time life cycle of a service, through control over when a service moves through a runtime life cycle and visibility of where in the life cycle a service is. These controls allow you to enforce requirements on the development of a service.
An API Management solution is consumer centric, treating APIs as products, enabling the consumer to discover and sign up for APIs. An API Management solution is designed for consumers. By contrast a Registry and Repository solution is designed for use by service providers and is provider-centric.
A traditional Registry and Repository solution may not offer:
• A runtime gateway for enforcement of consumer access and security.
• Self-service sign up to enable developers to quickly discover and consume APIs and securely access enterprise data.
• Security and governance controls for IT operations.
• Create and run for services and microservices.
• Analytics of invocations of services to monitor services to improve performance.
IBM API Connect
Designed for organizations looking to streamline and accelerate their journey into the API economy, API Connect is a comprehensive management solution that addresses all four aspects of the API lifecycle: create, run, manage and secure. This makes API Connect far more cost-effective than limited point solutions that focus on just a few lifecycle phases and can end up collectively costing more as organizations piece components together.
API Connect features:
• An end-to-end integrated experience across the API lifecycle, including API and microservice creation, that can reduce costs and simplify application development.
• Automated API creation and discovery from systems of record for fast time to market.
• Self-service access to APIs to speed discovery and consumption by developers, while providing security and governance controls for IT operations.
• Unified management and administration of Node.js and Java plus support for multiple development languages and frameworks to enhance developer productivity and reduce dependence on central IT.
• Hybrid cloud licensing for seamless platform flexibility and lower cost of ownership.
Can you run a Registry and Repository and API Connect side-by-side?
A Registry and Repository can track back end services of all types, for example MQ, CICS Applications, File Transfer, REST and SOAP. API Connect specializes in SOAP and REST over HTTP(s) and therefore you may need to continue cataloguing your other back end services in a Registry and Repository.
As the source of truth for services in your organization, a Registry and Repository tracks all services including those not exposed by APIs. For instance you may have back end services solely consumed by other back end service, or services exposed through other channels. In this case you may need to continue to use an R&R to track such services.
If you are unable to adopt agile development practises or require the control of a heavy weight service governance process, then a Registry and Repository can continue to track your services. You can still expose completed services as System APIs using API Connect, and adopt agile development for your end-user applications.
For more information, click here.
Can you move from a Registry and Repository to APIC?
Given the features of a Registry and Repository solution, there are a number of uses of a Registry and Repository mapping well onto API Connect, and a number of use cases which do not.
Comparison of the relevant features of a Registry and Repository and API Connect
A Registry and Repository provides heavy weight design time governance features that allow you to control development and deployment of your services. A Registry and Repository can check model a service through a set of life cycles, for example tracking a service has been proposed and the approval that it can be developed, or tracking that a service interface is being specified, or that a service has now been developed and is ready for testing. A Registry and Repository can enforce governance policies at any stage of the life cycle, for example that required metadata has been entered before a service can be proposed for development, that the metadata conforms to certain requirements, or that a service has an SLD and endpoint associated before it can be moved into the “Staged” life cycle state.
API Connect takes a more lightweight approach to API design time. An API can be in one of three states; Identified, Specified or Realized. This allows you to identify which stage of the development life cycle the API is in. The API can be published to the developer portal at any stage, so that others can gain visibility of the API. The states are documentation for the consumer to inform them about the state of the API.
API Connect can require an approval step before an API can be staged into a Catalog (the combination of an API Gateway and developer portal), before the API becomes visible and accessible on a gateway if the API is implemented. By using approvals the API can be checked by a centre of excellence or reviewer before it is made available.
A typical Registry and Repository has extensible data model for metadata, where extra attributes can be added to the various objects used to model a service.
API Connect can model extensions to the metadata normally provided by the Open API standard using an “extension schema”. These allow extra API metadata to be modelled using the JSON Schema standard, and such extensions are validated according to the schema you provide. The extensions are then rendered in the API designer UI making it easy to add content under an extension.
Limited design time governance
If your use of a Registry & Repository design time service life cycle is restricted to indicating that a service is being planned, it is being developed, and it has been developed and is available for use, you can consider using API Connect.
An API can have one of three states for its phase; Identified if the API is in the early conceptual phase and is neither fully designed nor implemented, Specified if the API has been fully designed and passed an internal milestone but has not yet been implemented, Realized if the API is in the implementation phase. Changing phase is not checked or enforced and therefore the API phase is for documentation use only.
You can publish a Product containing APIs in any state to a Catalog and they will be listed in the associated Portal. The Portal can be used to socialize details about the API during development.
A Registry and Repository provides a view onto your services. Some Registry and Repository solutions allow a read only view of services, to provide a Service Catalog. If you want a Catalog of your services so developers can browse, discover and use services, then the API Connect developer portal enables this use case.
It may also be that you do not need a centralized repository if your Line of Business teams are creating APIs and managing them.
API Connect allows you to represent your services as System APIs, package them as products and then publish these products into one or more Catalogs. Each Catalog has a developer portal in which the products and APIs are visible.
Full details of each API are provided, and code snippets showing how to invoke the API. A developer can download the WSDL or Swagger document describing the API.
The developer portal can have a custom theme applied to change the look and feel of the portal, to ensure it matches your company design.
The developer portal features forums where each API can be discussed and users can post helpful details on how to use the API, or feedback about the API. The developer portal also features a blog where you can make posts.
If you are doing runtime enforcement of your services using a gateway, setting rate limits and checking consumers have an agreement to invoke the service, then the gateway features of API Connect enable this scenario.
API Connect enables you to publish your SOAP over HTTPS and REST over HTTPS services as System APIs with a DataPower or Node.js Micro Gateway enforcing access and rate limits. API Connect is tightly integrated with the gateway, and requires no configuration on DataPower or the Micro Gateway to create new API gateway paths.
Out of the box API Connect provides for rate limits on a plan or individual operation level. A product contains multiple APIs and plans. Each plan includes one or more APIs and can set a rate limit for a consumer, or can override the rate limit for any specific operation. Consumers subscribe to a plan within the product.
API Connect allows custom policies to extend the behaviour and use the full power of DataPower. Once a custom policy has been defined in DataPower and loaded into API Connect, you can reuse it without having to configure on DataPower again.
One such custom policy enables API Connect to invoke an MQ service as the back end of the API. Using this you can expose your MQ services as SOAP/HTTPS or REST/HTTPS APIs. Click here for more details.
API Connect captures sophisticated API analytics, enabling you to manage service levels, set quotas, establish controls, set up security policies, manage communities and analyse trends. Such analytics are out of the box and require no configuration on DataPower and require no other software to work.
API Connect can enforce APIs that expose a SOAP over HTTPS endpoint on the API gateway. The API running in the gateway can invoke an interaction API or backend using SOAP over HTTPS, REST over HTTPS or MQ (using the aforementioned article). The interaction API can then invoke a set of backend services using various protocols.
If you want to expose a service on a gateway using other protocols such File Transfer using API Connect, this would not be simple or possible out of the box without extensive customization.
Traditional Registry and Repository products may be aimed at a waterfall development model, where services follow a strict step of life cycle states, with approval at many stages. It may be that as well as moving from a Registry and Repository to API Management, you want to transform your development processes to be more agile and responsive.
An example lifecycle is as follows and illustrates the design time portion of the default service lifecycle in WebSphere Service Registry and Repository, which is a simplified version of the original service lifecycle.
Services are initially designed by fully specifying their interfaces. Once the interface is specified, the interface is loaded into the Registry and Repository and the specification is reviewed to agree the interface. Once the review is passed the service proceeds to a development phase where the service is implemented (“realized”) according to the interface. Once development is complete the service is tested, moves to a review state, and once approved, the service is “realized”.
Once realized the service can move into a runtime state where it is deployed to a staging environment and re-tested. After another approval a service can be deployed to a production environment and re-tested, then finally reviewed and approved for production release. At this point the service is available for re-use.
While a service is moving through its life cycle, the various review steps may fail and the service can be returned to a previous step. For example, the service is rejected during the realization review step and moves back into the specified state for more implementation work.
However moving a service through a strict step of life cycle states does not allow for scenarios such as when the interface is discovered to be incomplete during the testing phase, while a consuming application is using the service to implement some logic. After all, the interface has been reviewed and approved before development even began. To change the interface would, according to the life cycle, require moving the service all the way back to the interface design state (“Identified”). Carefully managing change may be your desired aim, in which case a heavy weight governance process is suitable.
Another issue arises if a change to a service is needed that is backward compatible with the existing service. A strict service lifecycle may require an entirely new version of the service, with associated interface design and reviews. In an agile environment it may be sufficient to test and release a new minor version of the service, and move consumers onto it without their knowledge.
A more suitable approach may be to use agile development practises to create the service as an API. A minimal pass at an interface could be created and a skeleton API developed to implement the interface. The API can then be published to a test environment by the API developer for app or website developers to begin testing against the few methods available, the API can indicate that it is still under development, but feedback is welcome.
After discovering that an app needs more methods on the API, a new version of the API can be rapidly created that just adds the new methods, published to the test environment and re-tested with applications. Alternatively if the API Management solution supports it, the existing version of the API can be changed and re-published on top of the existing API.
When the API is ready to be published to the staging or production environments, the API Management system may require some checks in a review step. To publish to a development or test environment does not need such checks, to allow for rapid prototyping and development of an API.
You may wish to have a DevOps pipeline where an API is pushed to production after some automated tests are run, and automation is able to push your API out to a gateway and developer Portal. Such a pipeline could be triggered by a code check in, pushing a tag in Git or some other source control action.
API Connect is generally aimed at agile development where an API changes rapidly, with new versions being prototyped, created and tested with consumers on a frequent basis. It may be that as well as moving from a Registry and Repository to API Connect, you want to transform your development processes to be agile and responsive.
API Connect has a development mode for a Catalog where any API can be overwritten with a new revision, while keeping the current consumers. This enables API developers to rapidly change and publish APIs while they are being developed and tested by subscribed applications.
API Connect will also enable DevOps tooling such as Jenkins or UrbanCode Deploy to automate the push of an API to various environments, API gateways and developer Portals. API Connect provides a command line interface and “Portal” REST API to enable such automation.
Moving from a Registry & Repository to API Connect
Moving from a Registry and Repository to API Connect concerns both moving the technical artifacts and changing the development process you may be using.
Using WSDL, XSD and Open API documents
Documents can be stored as WSDL, XSD and Swagger for REST APIs. Metadata is generally stored in the registry as objects and can decorate the WSDL, XSD and Swagger documents or be stored alone.
If you have a WSDL and XSD then you can create SOAP APIs in API Connect using WSDL and XSDs from your Registry and Repository. These “System APIs” expose your SOAP services via the gateway and enable you to enforce consumer agreements, security and collect analytics.
You can use WSDL to invoke your SOAP services as part of a new API you create in API Connect.
If you are using WebSphere Service Registry and Repository you can directly search and load SOAP services from within the API Connect user interface, to create System APIs.
If you have Open API (Swagger) API definitions for REST APIs then these can be loaded into API Connect as the basis for creating an API.
WSDL and XSD files generally define the interface for a service but will not contain specific metadata about the service. Therefore when loading WSDL and XSD to create a System API you will then need to copy any metadata from your Registry and Repository into API Connect. Metadata is stored in the Open API definition for the API in the standard fields, or as Swagger extensions, or on the Product definition.
Any documents stored in your Registry and Repository such as charters or design documents will need to be stored. API Connect is not designed to be a document management system and therefore it is recommended to store documents elsewhere in a document management system or under source control. Metadata can then be added to APIs in API Connect to describe where the documentation is stored, for example in the external documentation section of the Open API (Swagger) definition.
Using the IBM Service Oriented Architecture Transfer Tool
You can use the IBM Service Oriented Architecture Transfer Tool to move services and metadata into API Connect (as System APIs) individually or in bulk.
The transfer tool is a command-line tool for transferring services and their metadata from IBM WebSphere Service Registry and Repository (WSRR) to API Connect. The tool can transfer SOAP and REST services from WSRR and take the service metadata from WSRR: properties, classifications, state. The tool takes a set of services belonging to an organization and transfers them into API Connect. The tool can also take a single version of a service and transfer it.
The tool can be customized to query for services and metadata in various ways. The tool can be customized to change which data from WSRR is used to create the API definition, and where in the API and Product definitions your WSRR metadata is put. The tool will transfer in production (“Operational”) services and applications, and can be customized to transfer services and applications in other states.
The tool can push APIs and Products to drafts and can optionally publish Products and APIs to Catalogs.
The tool can find consumers of a service in WSRR and create Applications in API Connect and subscribe these Applications to APIs, to represent the consumers of a WSRR service in API Connect.
Using these features the tool can take in-production services from WSRR and create System APIs for them, taking the metadata from services and adding it to the Open API and Product definitions.
If desired, the tool can then be customized to transfer in-development services into API Connect. However it is recommended that services be pushed through their development life cycle until they are in production, and then transferred into API Connect. New services should be developed in API Connect from the start.
Thank you to Christopher Phillips for reviewing this blog.