We have all found ourselves lost and frustrated looking for an answer buried somewhere in vast amounts of information. It costs time, wears patience down to a frazzle, and is expensive for businesses. This is one area where IBM Watson Retrieve and Rank can help–an API announced in General Availability last week. IBM Watson Document Conversion, which is often used with Retrieve and Rank, was also released in experimental mode to help convert content in commonly used documents (e.g., PDF, Word) into formats that can be used by other Watson services.
Retrieve and Rank is designed to help developers build cloud-based cognitive apps that improve knowledge workers’ ability to find a needle in a haystack. We consider it to be a core part of our platform and a service around which many future capabilities will be built. This service applies a machine learning technique known as Learning to Rank to help users get better results from Apache Solr, a popular open source platform. It does this by finding “signals” in the data that are relevant to a user’s query.
For example, one early validation partner in a Contact Center used Retrieve and Rank to improve the ability of their agents to find highly relevant content to address incoming customer queries. Typical approaches to achieving this require painful and time-consuming manual tuning to get it right. Using Retrieve and Rank, and leveraging a combination of relevant information (for example, product name, short description, detailed problem description etc.) the partner was able to train a machine learning model to improve the information results generated from contact center agent queries. This resulted in a significant improvement in findability compared to conventional information retrieval techniques.
As we continue to expand our portfolio of Watson Developer Cloud services, you’ll notice how they start to fit together nicely. Document Conversion is a natural fit with Retrieve and Rank–and other services too. Document Conversion is designed to take documents (currently PDFs, Word, and HTML), break them into bite sized chunks, and convert them to an output format that can be used by another service. In conjunction with Retrieve and Rank, Document Conversion is a perfect fit for an organization that must wade through a stack of manuals or other types of unstructured documents to find critical information. You could use the Document Conversion service to produce textual units of the appropriate size, and feed those as documents into Retrieve and Rank. Then, by using our enhanced relevancy techniques, users will be able to quickly find the information they need.
Our research scientists continue to innovate in these areas. We are looking at ways to make the process of getting up and running easier–while striking the balance between making machine learning easy to apply while still giving sufficient control and flexibility to developers who want to tweak knobs. We are also investigating ways to provide developers additional Watson automated learning capabilities. This can include, for example, applying what can be learned from user clicks and other activity to continuously improve the models for a service in order to provide better results.
Please let us know the interesting things you do with these services by participating in our Watson Developer Community Forum.
IBM is placing the power of Watson in the hands of developers and an ecosystem of partners, entrepreneurs, tech enthusiasts and students with a growing platform of Watson services (APIs) to create an entirely new class of apps and businesses that make cognitive computing systems the new computing standard.