Welcome to this hands-on lab, *Viewing IoT Core Data Using BigQuery*.
Google Cloud IoT Core is a fully managed service that manages and ingests data from millions of globally dispersed devices.
You will gain experience by using IoT Core to collect data from a single device. However, you can use the same configuration steps to collect data from millions of devices.
In this lab, you will:
– Get hands-on with IoT Core to create a registry, add a device, and send simulated data from a compute engine VM.
– Configure a Cloud Dataflow template job to collect the data from a Pub/Sub topic, transform the data from JSON to table format, and store the data in BigQuery.
– Use BigQuery to query the data, sort it by the time collected to prove you have configured the IoT data pipeline to move data from Pub/Sub using DataFlow, and store it in BigQuery in the correct format.
– Export the data to Data Studio and display the heart rate data over time collected.
Learning Objectives
Successfully complete this lab by achieving the following learning objectives:
- Ingest the IoT Core Data
Create a Google Cloud IoT Core registry called us-iot-hr-trial, a Pub/Sub topic called us-iot-hr-queue, and a Pub/Sub subscription caled us-iot-hr-data.
Register the VM hrsensor007 in IoT Core and create a public/private key pair.
Send simulated data to the IoT Core device using the
heartrateSimulator.py
script on the VM.
- Build a Cloud Dataflow Pipeline
Create a BigQuery dataset called heartratedata and a table called heartratedatatable.
Create a Cloud Storage bucket and collect the endpoints for the Pub/Sub subscription and the Cloud Storage bucket.
Send the Dataflow template and Pub/Sub subscription to BigQuery using the Dataflow job name, BigQuery table information, Pub/Sub subscription link, and bucket location.
- View Our Data in BigQuery
Run a query to view the IoT data that was received in BigQuery, and query the data to return
timecollected
. Once the data has been viewed in BigQuery, export the results to Data Studio and observe the data in a graphical format.