Lab
A Cloud Guru

Azure Data Lake Gen2 From the Command Line

Azure Data Lake Gen2 is built on Azure Blob Storage but offers additional features. With Data Lake Gen2, you can store unstructured Blob data hierarchically, providing greater flexibility in how your data is organized. In this lab, you will have the opportunity to work with Azure Data Lake Gen2 storage from a Linux command line. You will retrieve, edit, and upload some Azure Data Lake Gen2 data from within the Bash Azure Cloud Shell.

Try for free Contact sales

Path Info

Level

Intermediate

Duration

30m

Published

Feb 14, 2020

Challenge

Download the configuration file.
Log in to the Azure portal. In a separate tab, log in to the Azure cloud shell (bash) at shell.azure.com.

There is an existing storage account and file share that you can use for the bash cloud shell. After selecting Bash for the Azure Cloud Shell, go to the Advanced Settings.
- For Cloud Shell region, select West US.
- For Storage Account, select Use existing, then choose the storage account with the name that begins with cloudshell.
- For File Share, select UJse existing and enter cloudshell.
Authenticate with the Azure Storage service.
```
azcopy login
```
The command will provide a URL and an authentication code. Open the URL and enter the code to authenticate your azcopy cli tool.

Set an environment variable containing the name of the storage account so that you can easily refer to it. You can find the storage account name in Azure Portal. Its name begins with sattconfigs.
```
storage_account=<storage account name>
```
Download the configuration file from Azure Data Lake.
```
azcopy copy "https://${storage_account}.dfs.core.windows.net/configuration/inventory/processor/invprocessor.conf" invprocessor.conf
```
You can verify the file downloaded successfully by viewing the contents. You should see some configuration data.
```
cat invprocessor.conf
```
Challenge

Make the requested changes and upload the edited configuration file.
Edit the configuration file:
```
vi invprocessor.conf
```
Change the numThreads configuration value to 100:
```
...
numTreads=100
...
```
Upload the edited file to Azure Data Lake, replacing the existing file:
```
azcopy copy invprocessor.conf "https://${storage_account}.dfs.core.windows.net/configuration/inventory/processor/invprocessor.conf"
```

Author

A Cloud Guru

The Cloud Content team comprises subject matter experts hyper focused on services offered by the leading cloud vendors (AWS, GCP, and Azure), as well as cloud-related technologies such as Linux and DevOps. The team is thrilled to share their knowledge to help you build modern tech solutions from the ground up, secure and optimize your environments, and so much more!

What's a lab?

Hands-on Labs are real environments created by industry experts to help you learn. These environments help you gain knowledge and experience, practice without compromising your system, test without risk, destroy without fear, and let you learn from your mistakes. Hands-on Labs: practice your skills before delivering in the real world.