IoT devices can create a lot of data. In fact, sensor technology on smart devices already accounts for about 30 percent of the world’s data. This is the new natural resource we want to capture, refine, and distill into something useful. But with all this wonderful data comes a challenge. Where do I store it? The scale is massive. As IoT devices continue to grow in capability and sophistication, IoT businesses are trying to handle terabytes of data at an entirely new rate.
One thing is pretty obvious. A single specific storage solution is not going to work for every case. Needs around performance, latency, cost, and scalability vary widely. So what’s needed is an ability to pick and chose the right storage technology to suit your priorities.
Watson IoT Platform recognized this need for flexibility and earlier this year we started introducing some changes to the way we provision data storage capabilities. First of all we introduced a last value cache that makes it easy to access the most recent events from a device. Today I’m please to announce the release of the next component in our storage strategy in the form of a managed connector from Watson IoT Platform to the IBM Cloudant NoSQL DB service on Bluemix.
IBM Cloudant NoSQL Database for Bluemix is a fully managed NoSQL database as a service (DBaaS). It’s built from the ground up to scale globally, run non-stop, and handle a wide variety of data types like JSON, full-text, and geospatial. Cloudant NoSQL DB is an operational data store optimized to handle concurrent reads and writes, and provide high availability and data durability. This service is compatible with CouchDB, and accessible through a simple to use HTTP interface for mobile and web application models. You can work with self-describing JSON documents through a RESTful API that makes every document in your Cloudant database accessible as JSON via a URL. Documents can be retrieved, stored, or deleted individually or in bulk and can also have files attached. With extremely powerful indexing, real time MapReduce and Apache Lucene-based full-text search, Cloudant NoSQL DB makes it easy to add advanced data analytics and powerful data access. Data access can also extend to Cloudant Sync, enabling data access from mobile devices and client apps to run connected or off-line.
So connecting a Cloudant database service instance to Watson IoT Platform gives you a powerful and robust means to store and access your historical device data ready for further processing and analytics. All the power of the Cloudant APIs and UI are available. Device data can be stored in daily, weekly, or monthly databases depending on your selected “bucket” interval.
When you begin using a Cloudant NoSQL DB to store device data three databases are automatically created, one database is created for the current bucket interval, one for the upcoming interval, and one configuration database. Design documents can be added to the configuration database and will be copied to new databases as they are created. When the end of an interval is reached device data is stored in the bucket database for the new interval, and a new database is created for the following interval.
Stay tuned as we bring out more connectors to further increase the data storage flexibility of our Watson IoT platform.
Previous announcement of Last Value Cache
IBM Watson IoT Platform documentation
Cloudant API documentation
IBM Cloudant for Bluemix Public – New Pricing Model