Use an open source image segmentation deep learning model to detect different types of objects from within submitted images, then interact with them in a drag-and-drop web application interface to combine them or create new images.
Most images that are shared online depict one or many objects, usually in some setting or against some kind of backdrop. When editing images, it can take considerable time and effort to crop these individual objects out, whether they are to be processed further elsewhere or used in some new composition. This application uses a deep learning model from the Model Asset eXchange (MAX) to automate this process and spark creativity.
In this application, the MAX Image Segmenter model is used to identify the objects in a user-submitted image on a pixel-by-pixel level. These categorized pixels are then used to generate a version of the image with each unique type of object highlighted in a separate color, called a colormap. Each segment is then split into its own image file that can be downloaded for use elsewhere. As subsequent images are uploaded, they are added to the carousel in the lower portion of the screen and saved in the browser, using PouchDB. From this carousel, images can be reviewed, deleted, or loaded into the “Studio.”
In the Studio section of the app, two images can be loaded into an interface that allows for drag-and-drop combinations of any two objects within them. Any new images you create here can also be downloaded.
When you have completed this code pattern, you’ll understand how to:
- Build a docker image of the Image Segmenter MAX Model
- Deploy a deep learning model with a REST endpoint
- Recognize object segments in an image using the MAX Model’s REST API
- Run a web application that uses the model’s REST API
- Interact with processed object segments to create new images
- The user submits an image using the web app interface.
- MAX Deep Learning Microservice identifies object segments in the image and returns a JSON response.
- The web app server uses the JSON response to crop objects from the image and displays them to user.
- The user interacts with object segments from uploaded images using the web app interface.
Find the detailed steps for this pattern in the README file. The steps show you how to:
- Start the MAX model API.
- Start the “Magic Cropping Tool” web app.