IBM Developer Blog

Follow the latest happenings with IBM Developer and stay in the know.

Rapidly ingest and analyze streamed data for event driven applications with Db2 Event Store

This blog is part of the Db2 Event Store learning path.

Level Topic Type
100 Achieve real time analytics, IoT, and fast data to gather meaningful insights Blog
101 Understand customer interests with clickstream analysis Code pattern
102 Analyze IoT sensor data with machine learning and advanced analytics Code pattern
103 Stream and store retail order data for analysis Code pattern
104 Stream data with Apache Kafka into the IBM Db2 Event Store Tutorial

Data is fueling today’s digital transformation, but only 15% of organizations get what they need from their data. And 87% of them will reevaluate or adopt a fast data analytics strategy within the next two years to better achieve their goals. (Read the study conducted by Forrester).

On the path to transforming your organization to be AI enabled, you need to have a prescriptive approach to accelerating your journey. Read more about ‘How to Scale the AI Ladder’ in the Daniel Hernandez blog that talks about the following 5 steps to get to AI:

  1. Modernize all your data estates in a multicloud environment.
  2. Collect data to make it simple and accessible.
  3. Organize data to create a business.
  4. Analyze and scale AI everywhere with trust and transparency.
  5. Infuse and operationalize AI throughout your business.

There’s no AI without IA (Information Architecture), and drilling down into the collection piece of making data simple and accessible shows us that IBM has many options in terms of choosing the data repository that’s right for you.

Let’s start with the Hybrid Data Management Platform. This platform allows you to deploy and scale data when and where you want and even let’s you choose which deployment target you want–public or private cloud, on-premises and even appliance. All of these data stores are intertwined with one Common SQL Engine and one unified experience across all the different flavors.

Let’s describe the different types of workloads you may need to handle for your hybrid data management strategy, and which might be best for the job:

  • IBM Db2 Advanced Enterprise Server Edition: This is the most trusted database management system in the enterprise space and is used by some of the largest institutions in the world to run their most important transactional workloads and analytics.

  • IBM Db2 Warehouse: This is a highly flexible data warehouse, optimized for fast deployment into private or virtual private clouds via docker containers.

  • IBM Db2 Big SQL: This is an enterprise grade, hybrid ANSI-compliant, SQL on Hadoop engine, delivering massively parallel processing (MPP) and advanced data query–offering a single database connection or query for disparate sources like HDFS, RDMS, NoSQL databases, object stores, and WebHDFS.

Next, I’d like to hone in on a category of data workloads focused on fast data. Typically, you’ll use these types of data stores for IoT or real time analytics use cases. In this space, data gets generated in real time; therefore, we want to be able to act on that data in real time as well. For example, telecom companies generating large amounts of data from CDRs (Call Data Records) need to analyze data in real time so they can act on potential fraud or load balancing in the network. In addition, a manufacturing line, where you have millions of data points coming out from sensors that can detect when a part is going to fail in realtime–and be able to act on the failure without having to impact the whole production sequence.

These are examples where you need a database like IBM Db2 Event Store, so that you can rapidly ingest and analyze streamed data for event driven applications. IBM Db2 Event Store is capable of ingesting hundreds of billions of events per day and can analyze the ingested data immediately for real-time insights. The system also stores all the data it ingests in Apache Parquet format and is continuously available, meaning that hardware failures don’t impact the ability to ingest the data or derive insights.

IBM Db2 Event Store is optimized for machine learning and comes embedded with IBM Watson Studio. This means you can now use the data collected from your streaming sources and apply AI in real time on the ingested data. IBM Db2 Event Store is a premium add on top of IBM Cloud Pak for Data (a new kind of data and analytics platform that simplifies how you collect, organize, and analyze data to accelerate the value of data science and AI). It’s also optimized for IoT with features such as new time series libraries containing special SQL functions for this type of use case.


This blog is the first part of a learning path that guides you in quickly coming up to speed on what IBM Db2 Event Store offers and how it’s used. The learning path consists of a step-by-step tutorial, patterns, and complete examples of working code. As you proceed through the learning path, you’ll learn more features and different use cases for Db2 Event Store.

So let’s get started. The first step will be to learn how to ingest clickstream data and analyze your website customer activity with interactive visualizations.