Most web applications support RESTful APIs, but — unlike SOAP APIs — REST APIs rely on HTTP methods and lack a Web Services Description Language (WSDL) equivalent to define request and response structures between consumers and providers. Without an adequate contract service, many REST API providers use Microsoft Word documents or wiki pages to document API usage. Those formats can make collaboration and document version control difficult, especially for applications that have many APIs or resources, or when APIs are under iterative development. Those types of documents are also harder to integrate into an automated testing application.
The open source Swagger framework helps remedy these issues for API consumers and developers. The framework provides the OpenAPI Specification (formerly known as the Swagger specification) for creating RESTful API documentation formatted in JSON or YAML, a human-friendly superset of JSON. Swagger documents can be processed by various programming languages and can be checked into source control for version management during the software development cycle.
But Swagger has shortcomings of its own. When we used the framework to document our own APIs, we found gaps between our documentation needs and Swagger’s basic capabilities. We’ll describe the challenges we encountered during our documentation process and show how we addressed them by:
- Implementing Swagger extensions
- Streamlining Swagger’s functions to aggregate documents
- Creating a tool to output Swagger documentation in an HTML page
The solutions that we developed are available for download (see ” Downloads“). You can adapt our examples to document your own RESTful APIs with Swagger and — using the techniques you learn in this article — create your own Swagger customizations.
Using Swagger
Several Swagger editing tools help you to create API documents easily and ensure that they conform to the OpenAPI spec. With Swagger Editor, for example, you can create or import API documentation and browse it in an interactive environment. The display pane on the right side shows the formatted document and reflects changes that you make in the code editor in the left pane. The code editor indicates any formatting errors. You can expand and collapse either pane.
Here’s how the Swagger Editor UI looks after you import a leads.yaml definition:
The red arrows overlaid on the screenshot indicate the correspondence between the post: and get: definitions in the OpenAPI Specification-based leads.yaml file, and the documentation for the POST and GET APIs in the preview document.
If you use Eclipse as an IDE, you can use YEdit, which checks and highlights YAML syntax and provides editing and formatting features.
Extending Swagger
Existing tools make it easy to edit Swagger API documents, but certain documentation scenarios present challenges. These are some practical issues we encountered while using Swagger to document our own APIs:
- The API consumers needed information that was specific to our APIs, but the OpenAPI Specification didn’t include a standard for it.
- The API consumers needed samples of requests and responses, but the existing editor couldn’t include them.
- We needed to provide a rich and human-readable document that included samples for API consumers, preferably in an online HTML document.
To address these problems, we created our own attributes, tools, and templates based on the OpenAPI Specification.
Extending properties
You can extend Swagger using the x- extension property. Here are some customized extensions that we tailored to our API, along with examples of their use:
The following properties are for the fields of API payload or response:
x-sc-crud: documents an API field’s valid create, read, update, and delete (CRUD) operation(s):x‑sc‑crud: [ read, update, create ]x-sc-required: indicates which CRUD operation(s) this field is required for:x‑sc‑required: [ create ]x-sc-fieldmap: documents the database table and the UI field associated with the specified API field:x‑sc‑fieldmap: table: TASKS_RELATED_TO uifieldname: Related tox-sc-enum: documents the API field’s valid values. Instead of a list of static values, you can specify an API that returns the current set of possible values:x‑sc‑enum: api: /leads/enum/alt_address_countryx-sc-comments: supplements thedescriptionproperty and is used to capture additional, transient information for a given API field:x‑sc‑comments: ‑ readonly in UI, aka Domestic Buying Group or DB
The following listing is a sample of the x-sc properties in the YAML definition of the lead_source API field in the Lead module:
lead_source:
type: string
maxLength: 100
externalDocs:
description: Lead Source // Current (0100) // Approved // op ‑ Opportunity
url: https://w3.ibm.com/standards/information/tmt/output/Approved/
ibmww/op/bds/Opportunity_Management/Lead_Source.html
#
#lead_source value is returned when retrieving a lead,
#and you can set its value when creating or updating a Lead.
#
x‑sc‑crud: read, update, create
#
#The lead_source is a required field when creating a Lead.
#
x‑sc‑required: create
#
#You can retrieve valid lead_source values from the
#/leads/enum/lead_source API.
#
x‑sc‑enum:
api: /leads/enum/lead_source
AVL:
dictionary_name: leads_lead_source_dom
example: LinkedIn
#
#The value of lead_source is saved in the LEADS table.
#In UI, you can find lead_source under the "Lead Source" label.
#
x‑sc‑fieldmap:
table: LEADS
uifieldname: Lead Source
The following properties extend the description of an API operation:
x-sc-samples: documents samples. This property consists of a list of JSON references to the samples section of the documentation: (see ” Including samples” for more information)x‑sc‑samples: ‑ $ref: '#/x‑sc‑samples/leads‑post‑create‑lead' ‑ $ref: '#/x‑sc‑samples/leads‑post‑create‑lead‑employeecnum'x-sc-APIm-plans: lists which IBM API Connect (formerly known as IBM API Management) plan the operation is in. We want to capture information specific to the API Manager:x‑sc‑APIm‑plans: ‑ salesconnect_leads_read
The following is a sample of the x-sc properties in the YAML resource for the HTTP POST method of the /leads API endpoint:
paths:
/leads:
parameters:
‑ $ref: 'MASTER#/parameters/OAuthToken'
‑ $ref: 'MASTER#/parameters/ContentType'
post:
summary: create lead
tags: leads
#
#Use the x‑sc‑APIm‑plans property to specify that this endpoint
#is in APIm's salesconnect_leads_create plan.
#
x‑sc‑APIm‑plans:
‑ salesconnect_leads_create
description: |
<p>API to create a lead.</p>
#
#Use the x‑sc‑samples property to refer to samples of the usage
#of the /leads API.
#
x‑sc‑samples:
‑ $ref: '#/x‑sc‑samples/leads‑post‑create‑lead'
‑ $ref: '#/x‑sc‑samples/leads‑post‑create‑lead‑employeecnum'
parameters:
‑ name: lead_definition
in: body
description: definition of lead to be created
schema:
$ref: '#/definitions/LeadObject'
responses:
200:
$ref: '#/responses/LeadObjectResponse'
422:
description: |
<p>scenarios</p>
<ul>
<li>missing required field</li>
<li>invalid values for optional fields</li>
<li>et cetera</li>
</ul>Including samples
Even though Swagger is a powerful tool for defining RESTful APIs, it doesn’t yet provide ways to include examples of HTTP requests and responses, or add written documentation for developers.
To include request and response samples, we extended the spec and once again used YAML to document the samples. We put our samples in a separate schema so as not to overcomplicate the main API document.
The following leads-post-create-lead sample tells the consumer how to call the HTTP POST method of the /leads API, describing the URL, method, headers, URL parameters, input (sample payload), and response:
x‑sc‑samples:
leads‑post‑create‑lead:
title: Create a New lead
description: |
This sample creates a new lead, and assigns it to a user specified via sugar user id.
method: POST
url: https://w3‑dev.api.ibm.com/sales/development/salesconnect/leads
headers: |
Content‑Type: application/json
Oauth‑Token: 111e567a‑7624‑35f7‑ed82‑540f5a954312
Parameters: ?client_id={client_id}&client_secret={client_secret}
Input: |
{
"created_by": "eve25@tst.ibm.com",
"assigned_user_id": "51449f9b‑a68f‑059c‑ad06‑5039325c53b2",
"description": "2015‑01‑27 leads test",
"first_name": "one",
"last_name": "smith",
"phone_mobile": "22‑222‑2222",
...
}
Response: |
{
"my_favorite": false,
"following": false,
"id": "a260acfb‑5b3e‑3f74‑2392‑54d92387fb80",
"name": "one smith"
...
"_module": "Leads"
}
In the HTTP POST method of the /leads API section, we used the extended propertyx-sc-samples with a JSON reference to the sample code leads-post-create-lead in the x-sc-samples schema:
paths:
/leads:
parameters:
‑ $ref: 'MASTER#/parameters/OAuthToken'
‑ $ref: 'MASTER#/parameters/ContentType'
post:
summary: create lead
tags: leads x‑sc‑APIm‑plans:
‑ salesconnect_leads_create
description: |
<p>API to create a lead.</p>
#
#Use the x‑sc‑samples property to refer to samples of the usage
#of the /leads API.
#
x‑sc‑samples:
‑ $ref: '#/x‑sc‑samples/leads‑post‑create‑lead'
‑ $ref: '#/x‑sc‑samples/leads‑post‑create‑lead‑employeecnum'
parameters:
‑ name: lead_definition
in: body
description: definition of lead to be created
schema:
$ref: '#/definitions/LeadObject'
responses:
200:
$ref: '#/responses/LeadObjectResponse'
422:
description: |
<p>scenarios</li>
<ul>
<li>missing required field</li>
<li> invalid values for optional fields</li>
<li> et cetera</li>
</ul>Tying definitions and samples together
To tie all the module definitions and samples together, we used a MASTER.yaml file. This file documents the logistical information, including the Swagger version, API version, overall information, and the relative base path that the API is served on.
swagger: '2.0'
info:
version: '3.4'
title: Sales Connect API
#
#
description: >
This is the SalesConnect API.
<p> There are several modules, each with different top level path.
Most of the modules follow the same general pattern for operations and results,
varying in the Sales Connect object that is manipulated.</p>
<p> The individual module descriptions show
the particular operations and any module specific details.</p>
#
#
basePath: /sales/test/salesconnect
host: w3‑dev.api.ibm.com
In addition, MASTER.yaml includes annotations for packages, shared objects, and templates.
Packages
By defining package and package sets properties in the MASTER.yaml file, you can generate the content dynamically. A package defines all the YAML files relating to a given module, while a package set is a collection of packages that give you fine-grained control over the content in the final document. In our example, the demo and default package sets each pull content from a different set of files. Using the combination of package and package set, we can easily separate released modules from those still under development.
x‑sc‑master:
packages:
bulk:
‑ modules/bulk
#
#The oauth package contains 2 files:
# ‑ the modules/oauth.yaml file
# ‑ the samples/oauth‑samples.yaml file
#
oauth:
‑ modules/oauth
‑ samples/oauth‑samples
calls:
‑ modules/calls
‑ samples/calls‑samples
collab:
‑ modules/collab
notes:
‑ modules/notes
‑ samples/notes‑samples
...
leads:
‑ modules/leads
‑ samples/leads‑samples
‑ samples/leadsTBD‑samples
...
package‑sets:
#
#When generating a "default" document, our tool pulls
#content from files specified under the "oauth", "bulk",
#"calls", "collab" and "notes" packages.
#
default:
‑ oauth
‑ bulk
‑ calls
‑ collab
‑ notes
#
#When generating a "demo" document, our tool pulls
#in a different set of files.
#
demo:
‑ oauth
‑ notes
‑ leads
Shared objects
During the implementation process, we noticed that some objects were mentioned repeatedly: The same filter URL parameters were used by many API back ends, the same set of basic fields were returned by many APIs’ GET methods, and the same error message format was returned by the API Manager for different calls. To reduce redundancy, we extracted these elements into the MASTER.yaml file by using a BasicObject property, which defines the basic fields returned by all of the HTTP method GET APIs, and an APImException object, which describes the error structure returned from the API manager.
Using templates
Many APIs’ responses follow a similar pattern, so we designed templates to reduce duplication and support variation. For example, when a module is linked to another module, the response of some APIs will contain objects from both. To accommodate this situation, we designed an (OBJECT)Link(otherOBJECT)Response template in the MASTER.yaml file:
(OBJECT)Link(otherOBJECT)Response:
description: |
Result of creating link from (OBJECT) to (otherOBJECT).
The record contains the (OBJECT) and the related_record the (otherOBJECT) objects.
schema:
properties:
record:
$ref: 'MASTER#/definitions/(OBJECT)'
related_record:
$ref: 'MASTER#/definitions/(otherOBJECT)' When documenting the response of an API linking the Note module (OBJECT=NoteObject) to the Account module (otherOBJECT=AccountObject), you can use the following format:
post:
summary: Establish an accounts link to note using ccms id.
description: Establish an accounts link to note using ccms id.
responses:
default:
$ref: 'MASTER?OBJECT=NoteObject&otherOBJECT=AccountObject#/responses/
(OBJECT)Link(otherOBJECT)Response'
In our example, $ref tells the tool to go to the MASTER.yaml file and look under the responses schema for the (OBJECT)Link(otherOBJECT)Response object. Before instantiating the JSON reference, the tool replaces the OBJECT variable with NoteObject and otherOBJECT with AccountObject. In the end, the template is expanded to:
NoteObjectLinkAccountObjectResponse:
description: |
Result of creating link from NoteObject to AccountObject.
The record contains the NoteObject and the related_record the AccountObject objects.
schema:
properties:
record:
$ref: 'MASTER#/definitions/NoteObject'
related_record:
$ref: 'MASTER#/definitions/AccountObject'
Likewise, you can use this template to document the response of the API linking the Note module to the Opportunity module by setting OBJECT and otherOBJECT to different values:
post:
summary: Link from note to opportunity.
description: Link from note to opportunity.
x‑sc‑APIm‑plans: salesconnect_notes_create responses:
default:
$ref:
'MASTER?OBJECT=NoteObject&otherOBJECT=OpportunityObject#/responses/
(OBJECT)Link(otherOBJECT)Response'
Converting a YAML document into an HTML page
To make the API document more user friendly, we implemented a tool (swagger-to-html.php) that converts the YAML document to static HTML. Although existing tools are available for building HTML documents from Swagger, we decided to create our own so we could add special handling for our x-sc-* extensions.
The tool reads in the MASTER.yaml file, merges all the necessary YAML files, resolves references, and outputs an HTML page. The tool accepts various parameters that you can use to customize the content of the HTML file. For example, you can generate an HTML file for specific modules, an HTML file for a specific package set, or an HTML file only for those APIs registered with the IBM API Management application.
The generated HTML is a single file that uses CSS for styling and JavaScript to automate the section collapse-and-expand and navigation features. The HTML generator tool renders general JSON objects, JSON schema definitions, and the Swagger descriptions for parameters, responses, operations, and so on.
Here’s the HTML page generated for the item lead_source in the Lead module (see the corresponding YAML documentation in the ” Extending properties” section):
Here’s the HTML page for the HTTP POST method from the /leads API endpoint (see the corresponding YAML documentation in the ” Including samples” section):
Under the hood
This API-swagger.zip file (see ” Downloads“) demonstrates a subset of our Swagger API document for three modules in the SalesConnect system: OAuth, Lead, and Note. The APIs are in the /modules directory, and the corresponding samples are included in the /samples directory.
To generate the HTML page:
- Install the prerequisite software (PHP and PHP YAML extension).
cdto the API-Swagger directory.- To generate HTML for all the APIs specified in the MASTER.yaml file’s demo package set, enter the following command (type it as a single line):
php tools/swagger-to-html.php``salesconnect.swagger/``MASTER.yaml --set demo >``c:/swagger/salesconnectAPI.html - To generate HTML for modules
OAuthandLead, enter the following command (type it as a single line):php tools/swagger-to-html.php salesconnect.swagger/MASTER.yaml --modules oauth,leads > c:/swagger/salesconnectAPI.html
Conclusion
Swagger is a powerful tool for documenting RESTful APIs, and by implementing custom extensions, tools, and templates, you can gain more options and more control over the format and content of your Swagger-generated documentation. Extending Swagger’s capabilities can allow you to include more API-specific details, specify HTTP requests and responses, and output your documentation as HTML so both developers and API consumers can read it.