Use MQTT to stream real-time data  

Create a real-time stream of carbon intensity for the NY State power grid with MQTT and Kubernetes

Last updated | By Sean Dague

Description

MQTT is a real-time publish subscribe protocol that’s well suited for efficient distribution of data. Using a series of microservices, you can convert existing open data about the NY State power grid into an open, real-time streaming service.

Overview

The ny-power project uses a set of microservices to consume open data about the performance of the NY State power grid, and computes an approximate level of carbon intensity. This computed data is served in a real-time MQTT stream, making it easy to consume in other applications. It also shows how to manage a set of data processing in Kubernetes with Helm.

When you have completed this code pattern, you will understand how to:

  • Use MQTT in applications where streaming access to data is needed
  • Use Helm to deploy a Kubernetes application
  • Host a non-cloud-native service, like MQTT, in Kubernetes
  • Use Kubernetes microservices to process data in a loosely coupled way

Flow

  1. The user accesses the website
  2. The web browser directly accesses the MQTT service
  3. The application’s data pump polls NY ISO every minute looking for new data
  4. Data is pushed to MQTT service
  5. The archiver is subscribed to the MQTT service and sends all new data to Influx DB
  6. On any new data it computes current CO2 / kWh, and publishes both it and the recent time series to the MQTT service

Related Blogs

Two “edgy” AI TensorFlow models for you!

The global Call for Code is well underway, we want to share some visual recognition models which could help you. These AI models can operate on the edge, which could be particularly useful for this years’ theme: disaster preparedness. How could visual recognition help in relief work? From satellite and drone imagery analysis, to classifying...

Continue reading Two “edgy” AI TensorFlow models for you!

Leveraging the power of AI at Unite Berlin

Last week, from June 19 – 21, we were at Unity’s premiere in Berlin: Unite 2018. This conference brought together Unity’s video game and development community. Unity touches 770 million gamers all over the world and is the market leader for consumer AR and VR use cases and is also rapidly emerging as the market...

Continue reading Leveraging the power of AI at Unite Berlin

Related Links