Get started with Distributed AI APIs

The Distributed AI APIs is an early access offering from IBM Research to enable AI at distributed and edge environments. It packages APIs to support AI applications across distributed hybrid cloud and edge environments. These APIs are general purpose and support many modalities of data, for example, visual, acoustic, sensors, network logs, time series, or natural language. Using these APIs, you can create AI-based applications in various industries. The following image shows the Distributed AI high-level architecture.

Edge AI high-level architecture

Starting at the bottom in the figure, it’s assumed that at a minimum each site have a Linux®-based server with a container runtime like Docker. There could be additional software such as Red Hat OpenShift or IBM Cloud Pak for Data, but it is not mandatory. Also, it’s useful to automate the deployment and management of these containers with IBM Edge Application Manager (IEAM) or Red Hat OpenShift Advanced Cluster Management capabilities.

Above that are CoreSets and Federated DataOps, the distributed DATA layer capabilities. CoreSets are a set of algorithms to intelligently sub-sample raw data or compute compressed semantic representations of raw data to optimize data collection from the edge or distributed environments. Federated DataOps can automatically assess data quality, identify issues such as missing or noisy data, and substitute or discard such data.

On top of the data layer are the AI MODEL capabilities. The Model Management addresses all of the second pattern questions discussed previously about selecting the most suitable model for each distributed or edge site, optimizing the model for each site, and monitoring its performance. The Model Fusion is focused on enabling federated learning and federated inferencing in the context of the third pattern where all of the training data cannot be aggregated in one place.

Finally, at the top are Distributed AI applications created by using the APIs to solve specific business problems. A few examples have been created to illustrate how the APIs work and how you can use them to create applications.

Because some companies have little or no instrumentation in their operations, it can hinder any opportunities for automation. In a recent partnership with Boston Dynamics, IBM demonstrated how Distributed AI can help.

Deploying sensors to instrument legacy equipment can be cost prohibitive. However, the SPOT robot from Boston Dynamics is a roaming edge server and a sensor platform. SPOT can walk around and look or listen for anomalies by incorporating visual and acoustic inspection onboard and can interact with equipment where needed. The Distributed APIs running on Boston Dynamic’s SPOT robot can unlock tremendous opportunities for asset health insights and more timely action.

Getting started with Distributed AI APIs

Distributed AI APIs are a set of RESTful web services with data and AI algorithms to support AI applications across hybrid cloud, distributed, and edge environments. The algorithms address many challenges that arise in these distributed and heterogeneous environments, for example, semantically sub-sample raw data before transporting it over the network, assess and repair data quality issues, select the best model and optimize the model for each site, and monitor the model’s performance after deployment. These APIs are designed for general purpose use and support many modalities of data such as visual, acoustic, sensors, network logs, time series, and natural language data. Therefore, you can develop a variety of AI applications based on these APIs.

Distributed AI APIs

The Distributed AI APIs are comprised of several components, each addressing a part of the challenge in enabling AI at heterogeneous environments. The Distributed AI APIs do not focus on the basic requirements of creating and deploying AI pipelines, for example, model training and model serving. For that, you’d use your favorite open source packages such as TensorFlow and PyTorch. Then, you can containerize your application, including the AI pipeline, and deploy these containers at each site. In many cases, it’s useful to use a container orchestrator such as Kubernetes or OpenShift operators to automate the deployment process.

The Distributed AI APIs help you address non-trivial challenges that are associated with optimizing data management and model management across cloud and distributed frameworks. Specifically, it enables the following scenarios:

  • When collecting data from each site, semantically compress the data depending on the downstream AI task to minimize the cost and resources spent in moving data
  • Identify and repair quality issues including schema mismatches across each site
  • After training the models, optimize them for each unique distributed environment to ensure the site’s resource constraints are met and adapt them to site data distributions
  • After the models are deployed, check if a new site’s sample is an outlier and if the data distribution has shifted
  • If data can never leave each site, enable fusing of local models into a global model that can be deployed at each disperse site

High-level architecture

The Distrubuted AI APIs contain the following APIs:

  • CoreSets API: Algorithms to intelligently sub-sample data before transporting over the network
  • Federated DataOps API: Algorithms to assess the data quality and repair labels and missing values
  • Model Fusion API: Algorithms to train the models while data is resident across many sites
  • Model Management API: Algorithms to maintain the most suitable models for each site
  • Model Compression API: Algorithms to compress models based on a resource constrained site

High-level architecture

Summary

This article provided an overview of Distributed AI APIs, a set of RESTful web services with data and AI algorithms to support AI applications across hybrid cloud, distributed, and edge environments. By using these algorithms, you can address many challenges that arise in distributed and heterogeneous environments.