Welcome to the hands-on lab, Viewing IoT Core Data Using BigQuery.
Google Cloud IoT Core is a fully managed service that manages and ingests data from millions of globally dispersed devices.
You will gain experience by using IoT Core to collect data from a single device. However, you can use the same configuration steps to collect data from millions of devices.
In this lab, we will:
1. Get hands-on with IoT Core to create a registry, add a device, and send simulated data from a compute engine VM.
2. Next, configure a Cloud DataFlow Template job to collect the data from a pub/sub topic, transform the data from JSON to table format, and store the data in BigQuery.
3. Finally, use BigQuery to query the data, sort it by the time collected, proving that you have configured the IoT Data Pipeline to move data from pub/sub using DataFlow, and store it in BigQuery in the correct format.
4. Optional: Export the data to Data Studio and display the heart rate data over time collected.
Learning Objectives
Successfully complete this lab by achieving the following learning objectives:
- Ingest the IoT Core data.
To complete this lab objective, you will need to perform the following tasks:
Create a Google Cloud IoT Core registry using following settings:
- Registry_ID:
us-iot-hr-trial - Pub/sub topic name:
us-iot-hr-queue - Pub/sub subscription name:
us-iot-hr-data
- Registry_ID:
Register a device in the IoT Core (VM instance). Use the following settings:
- Device_ID:
hrsensor007 - Create the public/private key pair.
- Device_ID:
Send simulated data to the IoT Core device using the python heart rate simulator. Use the following settings:
- Project_ID
<your project ID> - Device_ID
hrsensor007 - registry_Id
us-iot-hr-trial - private_key_file: =
../.ssh/ec_private.pem
- Project_ID
- Build a Cloud Dataflow pipeline.
Create a BigQuery dataset and table:
- Dataset name:
heartratedata - Table name:
heartratedatatable
- Dataset name:
Collect the endpoints for pub/sub subscription and Cloud Storage Bucket:
- BigQuery Table link
- Storage url
- Pub/sub subscription url
Send the Dataflow template and pub/sub subscription to Bigquery using the following endpoints:
- Dataflow job name
- BigQuery table information
- Pub/sub subscription link
- Storage Bucket location
- View our data in BigQuery.
Run a query to view the IoT data that was received in BigQuery.
Query the data to return the results sorted by
timecollected, nottimerecieved.Optional: Export the results to Data Studio to create a dashboard graph.
