MQTT is a real-time publish subscribe protocol that’s well suited for efficient distribution of data. Using a series of microservices, you can convert existing open data about the NY State power grid into an open, real-time streaming service.
The ny-power project uses a set of microservices to consume open data about the performance of the NY State power grid, and computes an approximate level of carbon intensity. This computed data is served in a real-time MQTT stream, making it easy to consume in other applications. It also shows how to manage a set of data processing in Kubernetes with Helm.
When you have completed this code pattern, you will understand how to:
- Use MQTT in applications where streaming access to data is needed
- Use Helm to deploy a Kubernetes application
- Host a non-cloud-native service, like MQTT, in Kubernetes
- Use Kubernetes microservices to process data in a loosely coupled way
- The user accesses the website.
- The web browser directly accesses the MQTT service.
- The application’s data pump polls NY ISO every minute looking for new data.
- Data is pushed to MQTT service.
- The archiver is subscribed to the MQTT service and sends all new data to Influx DB.
- On any new data it computes current CO2 / kWh, and publishes both it and the recent time series to the MQTT service.