Archive Cosmos DB Data Using the Change Feed and an Azure Function

45 minutes
  • 2 Learning Objectives

About this Hands-on Lab

The Cosmos DB for NoSQL change feed is a powerful feature that can be leveraged for both event-driven applications as well as data management tasks. In this hands-on lab, you get a chance to practice working with an integration between Cosmos DB for NoSQL and Azure Function Apps to enable DIY data archiving in Azure Blob Storage.

Students with at least some experience in the Azure portal and scripting in C# — or any modern programming language — will be best positioned to complete this lab quickly and successfully, but even if you are a novice, you should be able to follow along and take advantage of a few hints dropped along the way.

Learning Objectives

Successfully complete this lab by achieving the following learning objectives:

Create a New Azure Function with Appropriate Bindings

In this objective, you will create a skeletal function in preparation to write the C# script using the Azure portal. Azure Functions are more typically coded in SDK code, but we wanted to make this a quick lab to help you get experience with the integration architecture, without the need to have a development environment spun up.

  1. Log into the Azure portal using the credentials provided in the lab UI. If you have any trouble, check the housekeeping video for tips.

  2. You should land in the resource group set up already. Take note of the location (region) for the resource group. You will need to use that same region when setting up the Azure Function App. Confirm that the following assets have already been deployed:

    • A Cosmos DB for NoSQL account: Check that the Cosmos DB account has a database called LabDB, and a container called LabContainer already deployed.
    • Azure Storage Account: Check that the Storage account already has a storage container called "lab-blobs" and confirm that the container is currently empty.
    • Log Analytics Workspace: Confirm that a Log Analytics Workspace was created.
  3. Create a new Azure Function App (Do not use an "integration" link from the Cosmos DB account; this may not bring you to the UI we want to use for this lab.). Helpful hints:

    • Use the resource group already created in the lab environment.
    • Provide any valid name you want.
    • Select the same region/location as the resource group already created in the lab environment.
    • Choose .NET as the runtime stack.
    • Confirm that the plan type is Consumption/Serverless.
    • Leave everything else defaulted as-is.
    • Choose the storage account deployed in the lab environment.
    • Enable Application Insights.
    • Leave all other categories of settings defaulted as-is.
  4. Once the Function App is complete, go to the Function App resource and, create a new Azure Function.

  5. For the input binding, choose a Cosmos DB trigger from the templated options.

  6. Create a new output binding.

  7. Test the Azure function.

  8. Run the sample function.

Update the Azure Function Code to Archive Data

In this objective, you will update the sample code with C# script that takes each updated item from the Cosmos DB change feed and writes it to the the Blob Storage container configured in the output binding. In the first step of this objective, we provide a natural language description of the code you need to author, purposely avoiding the use of syntax, in order to guide you without providing the actual code.

  1. Alter the sample code to write the inbound data to the Blob Storage container:

    • Add an out parameter to the function of type string and use the same name as the output binding reference: outputBlob
    • Replace the if condition with a try block, and put the two lines of code that write to the log in the try block.
    • Add a catch block that captures any exceptions and logs the exception message in the same manner as the two lines that write to the log in the try block.
    • The out parameter must be referenced in all paths (both successful and unsuccessful) , so add it to the catch block and set it to null to trivially fulfill this requirment for the unsuccessful path.
    • Back in the try block, cast the input item as a string and assign it to the out parameter.
  2. Save the code and test with a simple input, such as [{"id": "abc123"}].

  3. Check if your tests were fully successful by confirming that at least one or more documents (depending on how many tests you run) has been archived in the container.

  4. Check if the trigger on the change feed is working by adding a few items manually.

  5. Check the Blob Storage container to confirm that more items have been added.

Hints and Tips

  • If you do outside research in Microsoft documentation, be aware that when you create Functions in the portal, you are using C# as a scripting language. Microsoft provides samples in several forms and .Net versions. Make sure you are looking at the scripting examples.
  • Even though the Function takes a list, there will always only be one document in the list, so the code can be written to work with the first — and only — document in the list.
  • Do not over-think this task. You only need a few lines of code to successfully code the function. The purpose of the lab is to get practice with the integration architecture, not a test of your awesome coding skills.
  • If you get stuck, the full solution is in the lab guide and is reviewed in the solution video.
  • When testing the integration, you can put in any valid JSON you want. The only property that is required is id, and it must be unique within the container.

Additional Resources

Imagine your team maintains a Cosmos DB for NoSQL solution that uses the periodic backup mode that is the default for new Cosmos DB databases. For a number of reasons, you have decided not to switch to continuous backup mode, but you would still like to have an archive that captures data as it is inserted and updated in your Cosmos DB container and stores it in an Azure storage account. You have seen a simple architecture that leverages the Cosmos DB change feed and an Azure Function, and it seems to fit the bill for your purposes. You have taken it upon yourself to build a quick proof-of-concept. Follow the objectives outlined in the lab to implement the described architecture.

What are Hands-on Labs

Hands-on Labs are real environments created by industry experts to help you learn. These environments help you gain knowledge and experience, practice without compromising your system, test without risk, destroy without fear, and let you learn from your mistakes. Hands-on Labs: practice your skills before delivering in the real world.

Sign In
Welcome Back!

Psst…this one if you’ve been moved to ACG!

Get Started
Who’s going to be learning?