Designing Imaging Solution
Author: Mayur Jain, IBM GBS ( India)
Table of Contents
The imaging solutions are typically designed to convert the paper documents into digital images and use these images or the data retrieved from them for performing the business processes.
Here imaging refers to the techniques used to scan and index documents for storage in an image management or document management system. Imaging can also refer to the extraction of data from paper forms, such as invoices, tax returns, claims and subscription documents for processing by other systems.
Imaging eliminates manual data entry, either the indexing of images or the typing of data from forms into legacy systems. Capture uses many technologies, including scanning, image enhancement, recognition, database matching, and Graphical User Interface (GUI) to automatically apply indexes to images or extract data from documents.
The first important question that we need to answer before designing the imaging solution is that what we want to achieve from this? Do we need the Document Capture Solution or Data Capture Solution?
The difference between the two is following:
Document Capture: In document capture, the paper documents are scanned and permanently archived. No data is retrieved from the images. For later retrieval manual indexing is done.
Data Capture: In data capture, the paper documents are scanned and used as a form. Every field in the form is extracted and the resulting data is used to for business processes.
A standard Imaging solution has 5 phases in complete process (Document preparation, Scanning, Recognition, Validation and Export). To get desired output from any solution, it should be provided with the proper inputs. The input for the imaging solution is a paper documents. So the key to the success of any imaging solution is the document preparation. This is the manual process hence chances of error are more here. So the user who needs to do this should be properly trained for this. For example if we are using Separator sheets for documents or batches or if we are using fixed pages documents or if we are using patch sheets for separating the docs etc. The following diagram shows the different phases in the imaging solution.
Phases in imaging solutions
When paper documents come to the scanning user they are usually not ready to scan. Document preparation is the process of pulling out staples, removing paper clips and aligning documents in a single orientation for the scanner and putting the separator sheets if required within the pages.
To use any scanning solution we need the good scanner. Scanners come in many different shapes, sizes and speeds. We should not waste money on a high-end scanner if requirement is to scan only few pages. Conversely, we shouldn’t be getting a low-cost scanner if we want it to handle the high volume of work. To select an ideal scanner for your solution you need to keep following things in mind.
i. A good scanner device must have a paper feeder that saves the time by automatically taking the next available piece of paper and pushing it into the scanner. Scanners range from low-volume, usually 10 to 25 pages per minute, to medium volume capable of 25 to 75 pages per minute, to high volume at 75 to 150 pages a minute, and all the way to very high volume units that run above 150 pages a minute.
ii. We also need to look at the duty cycle of the scanner. Most scanner manufacturers publish the duty cycle for each scanner, i.e. the maximum page volume they can process per day or month.
iii. Bottom or top feeder – A bottom feeder takes paper from the bottom of the stack and allows you to add more while it is running. Top feeders are much more common and generally have simpler sets of rollers and feeders.
iv. If we are getting the documents via Fax then we don’t need a scanner. We can install the fax server (for ex. Right Fax), then inbound faxes will be stored as image files – the same as if they had been scanned. Then the capture software can read the images created by the fax server and start to process them just like scanned images.
Distributed and Browser-based Data Capture: IBM Data-Cap also provide the browser based scanning. This feature is also very handy in reducing the cost by lowering the shipping costs for documents and overnight mail, speed input of documents, and lower total cost of ownership as it will not be adding another desktop client for IT to manage and support.
The recognition has two functions to do. One is image cleanup and the second is Data recognition. For Image cleanup each Document Capture tools use different Image enhancement techniques like deskew image, border removal, rotation etc. If required one can also delete the blank pages scanned during scanning process. For simple forms processing, the user can enter information manually from the image, but in most capture applications the goal is to automate as much as possible using technologies that automatically read the information on the page. These are called recognition technologies. Different types of information require different data recognition technologies. A specific recognition technique from a vendor is often called a "recognition engine".Some engines, known as ICR, recognize hand printed characters, while others, called OCR, specialize in reading typed characters; still others read barcodes, and OMR engines read check boxes. Choose the right engine for the job and you’ll get optimal productivity.
i. Optical Character Recognition (OCR) engines are designed to read typed information, generically known as "machine printed" characters. But an OCR engine has its limitations. They usually struggle to read characters that are smaller than 8 points. Similarly, very large font, over 24 points will be ignored by the OCR engine. Everything is not printed on high-resolution laser printers. When the dots of the characters don’t quite touch or the type is very faint then expected OCR accuracy to decreases with it.
ii. Intelligent character recognition (ICR): There are two interrelated terms, "handwriting recognition" and intelligent character recognition (ICR). Before ICR, there was Optical Character Recognition (OCR). It was good at reading machine printed (i.e. typed) characters, but problematic when it came to reading handwriting, or even hand printed characters. To solve this issue artificial intelligence branch called neural network technology is used and a new technique to read handwritten characters is developed. As it was based on artificial intelligence hence it is called Intelligent Character Recognition. Many use the term "ICR" to refer exclusively to handwriting recognition. ICR works best on neat handwriting. Recognition is best when the characters are touching each other as little as possible.
iii. Optical Mark Recognition (OMR): Think about all the different kinds of forms that you have ever filled in and how you were asked to fill them in. In school you probably took standardized tests and filled in "bubbles" – small ovals next to the question. And you’ve probably gotten many opinion questionnaires with rectangular check boxes. Sometimes, of course, you are just given a circle to fill in. All of these can be read with OMR. The tricky part of OMR is to know whether the bubble or box or circle is actually filled in or not. Some form fillers may just draw a line across a series of boxes to indicate that they are selecting all boxes – but the line may wander in and out of individual boxes. Or form filler using bold check marks may draw the "tail" of the check right into an adjoining box. Good OMR recognition has to try to flag these situations as "uncertain" so that a user can decide what is happening.
iv. Barcode Recognition: There are a number of different types of bar codes, called "symbols". The most popular ones, like Code 39, code 128 or Interleaved 2 of 5 are s one dimensional symbols, means they consist few characters of data. There are also two dimensional like PDF 417 bar codes which can encode a more information. The disadvantage of using PDF 417 or other 2D bar codes is that normally there is no room to also print the human readable value that is encoded. Bar codes are a very sure way of passing information from a piece of paper into your computer. They are read fast and with amazing accuracy. Of course, bar codes have their drawbacks. For example, if a bar code is not read by the recognition engine, then what to do? It is important to always put a human readable version of the value on the same page as the bar code itself. Sometimes you can even use OCR as an automatic fall back if a bar code is not read.
Recognition engines alone are of no use. They convert images into characters, provide useful information about the success of the recognition, the recognition "confidence" on a character-by-character basis, but they generally don’t understand much about the meaning of the information. For example, the engine may be told to only recognize numbers, but to know the validity of the value a calculation or lookup may be required. That’s usually handled by the capture software that controls the recognition engine.
No matter how good recognition engines are, sometimes they make mistakes. There are times when a recognizer will generate results that, while having a "high confidence" level, are very wrong. To catch these instances before they become problems, you can apply validation rules to check the results of recognition. The important parts of validation process are following:
i. Validation Rules: Since the goal of the capture system is to accurately capture data about a document (indexing) or from a document (data entry) therefore the data should be properly validated. The rules can be applied to verification of the data for ex. the value should be between 1 and 9 or it should be non-numeric value or it should be from the available list of values etc.
ii. Indexing/ Verification User Interface: The most visible aspect of a capture system is the user interface for verifying recognition results or manually keying index information. A user interface needs to be efficient and flexible enough to handle all the different types of work that you need. A good verification system will automatically flag fields that need review. Fields may need it because a validation has failed, or because the recognition result is uncertain. When a character has been flagged, this is called a "reject." The character is rejected because the recognition engine has assigned a low confidence to its value: maybe it is an "8", but it could also be a "B," for example. The verification user interface must make it easy for a user to make the final determination – by looking at both the original image and the data in one glance – and then quickly move on to the next field.
iii. Key from Image: Some applications are not right for recognition. Many document-indexing applications involve assigning key index values to a document based on information about a document that may not be readily found with any recognition technology. In that case, displaying the document image on screen, with the fields to fill in, can be the best approach. A key-from-image screen can also show field snippets, when the location of the snippets is known. And since key from image is a fall back strategy when recognition fails, a good verification screen can also serve for key-from-image.
A capture system must export images appropriately and translate data into the proper formats for each class of system. Document management systems are among the internal computer systems to which organizations typically store data and images for the storage and retrieval of digitized documents. Other internal computer systems which process data from documents include Enterprise Resource Planning (ERP systems and standard databases). The following are brief summaries of the systems and their purpose:
i. Enterprise Document Storage and Retrieval: Organizations that receive thousands of documents a day use special systems for collecting, distributing and storing document images called Document Management or Image Management systems for example IBM FileNet CM or FileNet Image Services etc.
ii. Accounts Payable, Shipping: Accounts Payable, Shipping, Purchasing and other
back-office departments rely on Enterprise Resource Planning (ERP) systems, which help manage the inflow, payment and auditing of corporate transactions.
There are many tools available in market for document and data capture. Some of them are following:
Capture is tightly integrated with FileNet document management systems. IBM Filenet Capture is a thick client scanning product.
IBM Datacap provides both thick and browser based scanning capabilities. IBM Datacap is a rule based capture product and gives you capability of customizing the rules to achieve the desired functionality.
EMC Captiva InputAccel lets organizations capture data from paper, electronic files, and other sources, transforming it into digital content and delivering it into content management systems and business processes. By reducing manual paper handling, InputAccel minimizes processing errors, improves data accuracy, and accelerates business processes by making information instantly available in content repositories like Documentum, ApplicationXtender, and Microsoft SharePoint.
Ascent Capture delivers powerful, production-level document and data capture. For document capture, it boasts the fastest, most flexible scanning and indexing solution around. For data capture, Ascent Capture extracts important information such as machine-printed text, bar codes, hand-printed words and even checked boxes.
An imaging solution provides following benefits to the organization.
It reduces cost by reducing the need for manual data entry or infrastructure expenses since you won’t need as much space and cupboards for document storage.
With paper documents captured in the digital form, you can access the information you need quickly and easily.
When documents have been scanned and indexed, searching is quick.
It integrates the data with the organization’s ECM, ERP, Financial or corporate systems which saves time, reduces risk and faster the processing.
By converting paper documents in electronic content as they enter in the enterprise you eliminate pushing paper processes and are able to respond quickly.