Hands-On with Dataflow

By Joseph Lowery

This introduction to Cloud Dataflow for practitioners focuses on service fundamentals and the code used to create pipelines with Apache Beam.

8 hours
  • 29 Lessons
  • 2 Hands-On Labs

About the course

I’m pretty sure you’d agree with me if I said the world runs on data — especially the digital world — and the amount of data is ever-increasing. One way to really get your head around all that information is through data pipelines. Data pipelines pull in the data from a variety of sources, whip it into shape, and then toss it out to other services for storage, analysis, and/or visualization. As you’ll discover in this course, Google Cloud Dataflow is a best-in-class fully managed data processing service, ideal for all your data pipeline needs. Join me as we get hands-on with Dataflow.

  • Chapter 1 2 Lessons Introduction 4:28

    Course Introduction

    3:41

    Prerequisites

    0:47
  • Chapter 2 7 Lessons Understanding Dataflow 55:05

    Section Introduction

    1:46

    Grasping Dataflow Fundamentals

    10:17

    Examining Dataflow Concepts

    7:49

    Exploring Use Cases

    8:18

    Working with Streaming Data

    12:29

    Calculating Costs

    9:12

    Section Summary

    5:14
  • Chapter 3 6 Lessons Powering Up Your Dataflow Pipeline 37:41

    Section Introduction

    1:31

    Working with Apache Beam

    7:42

    Creating and Configuring a Pipeline

    11:31

    Spinning Up Templated Pipelines

    5:21

    Managing Your Pipeline

    7:13

    Section Summary

    4:23
  • Chapter 4 5 Lessons Employing Advanced Dataflow Features 24:54

    Section Introduction

    1:50

    Staging Custom Templates

    6:24

    Writing Cloud Dataflow SQL Statements

    3:46

    Utilizing AI Platform Notebooks

    9:39

    Section Summary

    3:15
  • Chapter 5 8 Lessons Integrating with Google Cloud Services 2:55:13

    Section Introduction

    1:38

    Streaming with Cloud Pub/Sub

    13:09

    Monitoring Dataflow Pipelines

    7:12

    Looking at Logging

    7:54

    Controlling Access

    5:53

    Create Streaming Data Pipeline on GCP with Cloud Pub/Sub, Dataflow, and BigQuery

    45:00 Hands-On Lab

    Viewing Cloud IoT Core Data Using BigQuery

    1:30:00 Hands-On Lab

    Section Summary

    4:27
  • Chapter 6 1 Lesson Conclusion 1:37

    Next Steps

    1:37

What are Hands-on Labs

What's the difference between theoretical knowledge and real skills? Practical real-world experience. That's where Hands-on Labs come in! Hands-on Labs are guided, interactive experiences that help you learn and practice real-world scenarios in real cloud environments. Hands-on Labs are seamlessly integrated in courses, so you can learn by doing.

Get Started
Who’s going to be learning?
Sign In
Welcome Back!

Psst…this one if you’ve been moved to ACG!