Enabling distributed AI for quality inspection in manufacturing with edge computing

We have updated this article to address product name changes and include updated links.

Implementing AI-assisted quality inspection systems can significantly improve inspection cycle time and inspection accuracy. However, managing and supporting hundreds of thousands of cameras, robotic arms, and robots can be a challenge. AI-assisted quality inspection systems in manufacturing environments call for the use of an edge computing solution.

Edge computing solutions bring AI and analytics workloads closer to where data is created and where actions are taken. When implementing and deploying an AI-assisted quality inspection system in an actual manufacturing production environment, an edge computing solution would efficiently and securely deploy trained models to edge devices, manage the lifecycle for the models and those edge devices, and also archive edge data when it is no longer needed.

In this article, we describe the technology stack (essentially Maximo Visual Inspection and IBM Edge Application Manager) that enterprises can use to deploy a trained model to the edge, enabling their teams to efficiently scale model run times and simplify inference process for quality inspection in manufacturing. The same edge computing approach can be applied to different use cases with minimal changes.

This is a real solution we are implementing in manufacturing to improve quality inspection using AI. By developing services that connect two of our leading AI and edge products we are able to showcase an important and practical solution which can be put in production along with running all services in IBM Cloud. We are using what we sell!

High level architecture of a quality inspection system

In 2018, the IBM Systems supply chain successfully adopted IBM Maximo Visual Inspection on premises for quality inspection in IBM system hardware manufacturing for mainframe, IBM Power Systems™, and IBM storage systems. Models that were trained by Maximo Visual Inspection (previously called IBM PowerAI Vision) have been used for production-level quality inspection, resulting in improvements in both efficiency and quality assurance. Therefore, Maximo Visual Inspection plays a central role in our quality inspection system.

The system context diagram in Figure 1 illustrates the intended functions of the final quality inspection system.

Figure 1. System context diagram of quality inspection system

System context diagram of quality inspection system

In the quality inspection system:

  • We have different roles in an inspection process for automation:

    • The modeling engineer import models, deploy models, and manage models
    • The edge manager initiate devices and manage devices that are used for inferencing
    • The quality inspector triggers the quality inspections where the actual visual inspection takes place
  • The quality inspection system integrates with manufacturing specific process:

    • Images and results are stored in a data warehouse
    • The Manufacturing ERP system makes orders and validates serial numbers
    • The robotic arm is triggered to complete the quality inspection

Technology stack for our edge computing solution

We based the architecture for our edge computing solution on the IBM Cloud Architecture for Edge Computing. In our edge computing solution, enterprises can deploy in the cloud or on-premises. All components can be scaled to support multiple edge nodes with different types of models. In our solution, we used Faster R-CNN models to show the object detection functionality.

Developing a quality inspection system in a highly complex computing hardware manufacturing environment requires that the following features be present in a practical edge deployment on manufacturing floors:

  • High availability: Because the quality inspection is the final gate before product shipment to clients, the availability of the system has a direct impact on shipment schedule and, hence, revenue. The system must have 24×7 availability with minimum maintenance intervals.
  • User authentication and authorization: An important feature of the manufacturing quality management system is that only authorized and trained operators are allowed to perform the quality inspection. Therefore, user authentication and authorization (for different user roles) are required. Moreover, user authentication has the advantage of linking to the enterprise user directory.
  • Scalability and performance: Because the manufacturing operation can span multiple geographies, the system needs to be scaled to manufacturing plants in many locations. The quality inspection system should have the capability to be scaled out easily, that is by adding an IBM Maximo Visual Inspection instance, either on-premises or on cloud, and also by adding edge devices. Performance (for example, speed, exception handling, and so on) must be considered to support users around the globe.
  • Model management and device management: After deployment, the lifecycle management of the AI models and devices become critical to operations. Users require an easy and efficient way to manage the AI model versions used in production. Edge device monitoring and recovery are also important to minimize disruption of manufacturing schedules.

Figure 2 illustrates the architecture we used to build the model management system.

Figure 2. IBM Edge Computing solution architecture

IBM Edge Computing reference architecture diagram

The architecture consists of the following three main parts:

  • Maximo Visual Inspection for model training
  • Cloud or on-premises application stack for central control and management, including IBM Edge Application Manager
  • An NVIDIA Jetson TX2 as an edge node device for quality inspection

The main process flow for model training that yields the best automated quality inspection is:

  1. Modeling engineer uses Maximo Visual Inspection to train the object detection model.
  2. Modeling engineer initiates model export functionality in the main dashboard (after authorization process).
  3. Dashboard invokes model exporting service with storage specified as a cache for models.
  4. Model exporting service communicates with REST API from Maximo Visual Inspection to download the model.
  5. Model exporting service stores model to the edge IBM Edge Application Manager object storage from the Open Horizon Model Management Service (MMS).
  6. Modeling engineer initiates model deployment to specified edge nodes using main dashboard.
  7. Main dashboard communicates with the edge connector which is responsible for working with the Open Horizon exchange REST API.
  8. Edge connector initiates deployment of patterns using IBM Edge Application Manager exchange REST API.
  9. IBM Edge Application Manager agent receives configuration from IBM Edge Application Manager exchange to deploy a new container with specific model
  10. IBM Edge Application Manager agent initiates Docker container start up with the IBM Edge Computing service that we developed.
  11. IBM Edge Application Manager service downloads model content from the IBM Edge Application Manager object storage
  12. Docker downloads the required Docker image which represents runtime for model.
  13. Docker starts model with Rest API.
  14. Quality inspector uses the edge dashboard to analyze photos of the product.
  15. In the Edge dashboard, user initiates quality inspection using REST API of deployed model and the results is displayed thereby automating the inspection process.

Why Maximo Visual Inspection?

Maximo Visual Inspection was used for model training because of its significant speed advantage. When installed on-premises, these benefits are realized:

  • Reduced manual eyeballing inspection time from 10 min to 1 min per product
  • Reduced field issues and improved customer satisfaction
  • Large-scale model life cycle management
  • Large-scale edge device life cycle management (including device set up, monitor, and recovery)
  • Edge data archive
  • GPU usage and segregation between model training and inferencing for better resource utilization
  • Separation of the responsibilities among personas for quality inspector, modeling engineer, and edge manager

Maximo Visual Inspection has integrated REST APIs to automate train, retrain, and model exporting. It makes computer vision with deep learning more accessible to business users by including an intuitive toolset for labelling, training, and deploying deep learning vision models, without coding or deep learning expertise. Maximo Visual Inspection includes the most popular deep learning frameworks and their dependencies. It is built for easy and rapid deployment which translates into increased productivity. By combining Maximo Visual Inspection software with IBM Power Systems, enterprises can rapidly deploy a fully optimized and supported platform with blazing performance. You can read more about Maximo Visual Inspection in its documentation in the IBM Knowledge Center.

Why IBM Edge Application Manager

The IBM Edge Computing service that we developed was used to manage distributed nodes, and also to deliver, update, and remove the model for quality inspection. IBM Edge Application Manager provides users with a new architecture for node management. It is designed specifically to minimize the risks that are inherent in the deployment of either a global or local fleet of edge nodes. Users can also use IBM Edge Application Manager to manage the service software lifecycle on edge nodes fully autonomously. More specifically, IBM Edge Application Manager supports model management through Sync Service. It facilitates the storage, delivery, and security of models and metadata packages. You can read more about IBM Edge Application Manager in its documentation in the IBM Knowledge Center.

Conclusion

In this article, we introduced the background and business challenges in implementing an AI-assisted quality inspection system in a manufacturing environment. We also described the overall architecture of an edge computing solution that was implemented using the IBM Edge Computing platform. We shared how to use the Open Horizon Model Management Service to enable the function of extracting the model from Maximo Visual Inspection and deploying the model to edge.

In our future articles, we plan to explain how to configure NVIDIA Jetson TX2 and how to work with Edge Fabric CLI and REST APIs. We might also share the work on model lifecycle management, edge device management, edge data achieve, and security.

Acknowledgements

The authors would like to acknowledge the contributions and reviews of this article by Ekaterina Krivtsova, Dmitry Gorbachev, Charisse Lu, and Thomas Cook.