Taxonomy Icon

Artificial Intelligence

Snap and translate text in images

Get the code

Summary

Built for developers who are looking to recognize text from images and translate it, this pattern shows how to capture an image, extract text, and translate that text using Tesseract OCR and Watson Language Translator.

Description

Visual content can be much more engaging than plain text. Add some interesting text and cool typography to that image and you’ve got a great way to create an infographic, run an advertisement, or share memes that you find funny. Adding text to your visual image can help you get your message across much more effectively. But, what happens when you need to share this across different geographies with different languages?

This code pattern explains how to create a hybrid mobile app using the Apache Cordova development platform and the Node.js server application running on the IBM Cloud Kubernetes service. The app uses Tesseract OCR to recognize text in images, Watson Language Translator to translate the recognized text, and Watson Natural Language Understanding to extract emotion and sentiment from the text. The mobile app translates the recognized text from the images captured or uploaded from the photo album.

When you have completed this code pattern, you will know how to:

  • Use the Cordova mobile framework to build and deploy a mobile app
  • Create Node.js apps that capture, recognize, and translate text using Watson services
  • Deploy Tesseract OCR on the IBM Cloud Kubernetes service to recognize text and bind Watson services to the cluster
  • Translate recognized text using Watson Language Translator
  • Extract sentiment and emotion from the text using Watson Natural Language Understanding

Flow

  1. The user interacts with the mobile app and captures an image or selects an image from the photo album.
  2. The image is passed to the Node.js server application that is running on the IBM Cloud Kubernetes service, which uses Tesseract OCR to recognize text in an image.
  3. The Node.js app uses the Watson language translator service to translate the recognized text and Watson Natural Language Understanding to return the sentiment and emotion of the translated text.
  4. Recognized text, the translated language, sentiment, and emotion results are returned to the mobile app for display.

Instructions

Find the detailed steps for this pattern in the README. Those steps will show you how to:

  1. Clone the GitHub repo.
  2. Create the language translation and natural language understanding services with IBM Cloud.
  3. Run the server application in a container on IBM Cloud with Kubernetes.
  4. Run the server application locally using Docker.
  5. Run the mobile application.