Google Certified Associate Cloud Engineer 2020

Sign Up Free or Log In to participate!

GCS & GCE Challenge Lab

Hello Folks,

The challenge is a good start to understanding well how Google Cloud thought about the components of a small project. If we progress to the next lesson and watch Mattias speaking the small project, explaining it using the Data Flows to build a mental model of what happened behind the scenes, it gets more clear how things are integrated and how data flows.

Following my notes; I performed the GCS & GCE Challenge Lab using the Console and I would like to put together here some points of attention. I would like to also confess that I wouldn’t able to execute it myself without cheating a little bit, watching Mattias doing it as the "scopes" was the point which makes the total difference.

* GCE instance that runs provided the script (work on scopes here is VERY important):  
    * SCOPES: attention to the Identity and API access having Storage to Write Only, and the Stackdriver Logging API, Stackdriver Monitoring API, and Stackdriver Trace to Write Only   
    * expand the Management, security, disks, networking, sole tenancy and add the script to the Automation >> Startup script  
    * attention toe the metadata as you need to map out the lab-logs-bucket=gs://mybucketname/

After creating the project some times using the Console, as said before, I started gathering the needed commands using gsutil and gcloud clients so I could repeat the same using only the Google Cloud Shell. I don’t want to share here the whole solution for that, but, to contextualize what I’m doing the problem I’m facing, I need to at least share the initial commands.

The whole idea is like below:

wagnerbianchijr@cloudshell:~ (challenge-lab-01-248121)$ gcloud projects create mytestproject03097557 --enable-cloud-apis  
Create in progress for [https://cloudresourcemanager.googleapis.com/v1/projects/mytestproject03097557].  
Waiting for [operations/cp.7602138930524904622] to finish...done.  
Enabling service [cloudapis.googleapis.com] on project [mytestproject03097557]...  
Operation "operations/acf.59e7adb0-9367-4217-adee-4d2f9ec675c9" finished successfully.  

wagnerbianchijr@cloudshell:~ (challenge-lab-01-248121)$ gcloud projects list  
PROJECT_ID               NAME                       PROJECT_NUMBER  
challenge-lab-01-248121  Challenge Lab 01           226890137415  
directed-cove-247421     Services Exploration Labs  881680634241  
mytestproject03097548    mytestproject03097548      41810333757  
mytestproject03097557    mytestproject03097557      592725485970  
neon-research-247220     Admin Project              465501537145  
omega-presence-234823    My Very First Project      523600421491  

wagnerbianchijr@cloudshell:~ (challenge-lab-01-248121)$ gcloud config set project mytestproject03097557  
Updated property [core/project].  

wagnerbianchijr@cloudshell:~ (mytestproject03097557)$ gcloud config set compute/region us-west2-b  
Updated property [compute/region].  

wagnerbianchijr@cloudshell:~ (mytestproject03097557)$ gcloud config set compute/zone us-west2-b  
Updated property [compute/zone].  

wagnerbianchijr@cloudshell:~ (mytestproject03097557)$ gcloud config configurations list  
NAME             IS_ACTIVE  ACCOUNT  PROJECT                DEFAULT_ZONE  DEFAULT_REGION  
cloudshell-5021  True                mytestproject03097557  us-west2-b    us-west2-b  

wagnerbianchijr@cloudshell:~ (mytestproject03097557)$ gsutil mb gs://mybuckwb01  
Creating gs://mybuckwb01/...  
AccessDeniedException: 403 The project to be billed is associated with an absent billing account.

After receiving the above message "AccessDeniedException: 403 The project to be billed is associated with an absent billing account.", I set the core/account and even with that, I don’t see yet a solution. Maybe you can share something with me to fix that:

wagnerbianchijr@cloudshell:~ (mytestproject03097557)$ gcloud config set core/account myemail@gmail.com  
Updated property [core/account].  

wagnerbianchijr@cloudshell:~ (mytestproject03097557)$ gcloud config configurations list  
NAME             IS_ACTIVE  ACCOUNT                    PROJECT                DEFAULT_ZONE  DEFAULT_REGION  
cloudshell-5021  True       myemail@gmail.com  mytestproject03097557  us-west2-b    us-west2-b  

wagnerbianchijr@cloudshell:~ (mytestproject03097557)$ gsutil mb gs://mybuckwb01  
Creating gs://mybuckwb01/...  
AccessDeniedException: 403 The project to be billed is associated with an absent billing account.

I wonder if is there something I’m missing on the command line to connect the project with a billing account.

Any thoughts? Thanks!!

Wagner Bianchi

I found another thread here on the forum where the discussion is almost the same: https://tinyurl.com/y5s4p4up

Wagner Bianchi

I don’t really know the reason for this yet but I will carry on hunting the answer. When I change the Google Shell project to the default one, the bucket creation works well. It seems something related to the project creation and my next step will be to look into the command to create the new project. It’s may be missing something else.

Wagner Bianchi

The solution is, you need to list your billing accounts and link the project with an existing billing account, See more here: The solution is here: https://cloud.google.com/sdk/gcloud/reference/alpha/billing/

Steve L Prescott

Wagner, thanks for listing these details. I struggled with the same issue. I did see some postings to the alpha command, but I ignored that because I figured there MUST be a solution that didn’t rely on alpha software. I guess not! Your solution worked for me.

Naga Sai Siva Sankar

Thanks Wagner, I too struggled with the same issue.

3 Answers

Alright! Nicely done, Wagner! And don’t think that what you’ve done is “cheating”, as I’ve intentionally tried to design the course to support you through the challenge at whatever your current level is. 😛 And it looks to me like you’ve used this challenge to very good effect! You got through the console lab with just a few hints, and then you also did the CLI lab. So 👍 to you!

A small thing I noticed, as I read through your post, is that you set “compute/region” to a zone instead of a region. 🙂 Just something you can try to remember for the future.

And then big kudos to you for debugging your problem with the billing account not being associated by default to projects created via gcloud! I guarantee you that struggling a bit with it like you did will now have locked that into your mind. I hope you’ve been enjoying this process, too.  What you’ve been doing is a lot like what a Cloud Engineer does, day to day.

Thanks for taking on the challenge and for posting your experience with it. 👏

Mattias

Steve L Prescott

Mattias, is there any CLI solution to this lab that doesn’t rely on alpha software? It seems that linking a billing account to a project would be a common requirement, and I’m surprised that a CLI command doesn’t already exist for this.

Mattias Andersson

Well, this has been in beta–not just alpha–for quite some time, now (years, I think?). And although Google’s "beta" bar is actually quite solid, it does make me sad that they haven’t gotten around to promoting it to full GA, yet. https://cloud.google.com/sdk/gcloud/reference/beta/billing/accounts/

Wagner Bianchi

Hey Mattias, I’m getting back to it now, after some time, and looks like that! The region, in this case, should be us-west2, and the zone should be the us-west2-b. I will post soon the complete CLI commands I used to get this challenge done using the CloudSDK. Thanks once again!

Swaminathan G

yes, beta is more stable. So I have tried like gcloud beta billing projects link <project-name> --billing-account=XXXXXX-XXXXXX-XXXXXX

Hello folks, how are you?

I’m getting back to the ACG right now and recapping on things here.

So, I executed the GCE & GCS Challenge Lab today again, and the commands for that would be like below:

#: create the new project, you probably need to change the below one as it should be unique
gcloud projects create mytestproject03097551 --enable-cloud-apis  

#: enable the billing account for the created project  
#: list billing accounts  
gcloud alpha billing accounts list  

#: link the project and the billing acocunt 
#: (fix the "AccessDeniedException: 403 The project to be billed is associated with an absent billing account")  
gcloud alpha billing projects link mytestproject03097551 --billing-account XXXXXXXXXX  

#: list projects
gcloud projects list  

#: set the current session project's
gcloud config set project [PROJECT_ID]

#: set the default region and the default zone on the cloud shell
gcloud config set compute/region us-west2  
gcloud config set compute/zone us-west2-b  

#: list configurations you have currently
gcloud config configurations list

#: create the bucket - the name should be unique, you probably need to change the below one**  
gsutil mb gs://mybuckwb027564/  

#: create the instance with the userdata script, the metadata for the bucket and the scopes for 
#: monitoring, storage and logging. I placed the Mattias script at the home of my user in the 
#: Cloud Shell, where I executed the below
gcloud compute instances create myvm01 --metadata lab-logs-bucket=gs://mybuckwb027564/ 
--metadata-from-file startup-script=script.sh --scopes storage-rw,monitoring-write,logging-write 
--machine-type f1-micro  

#: here you'd be argued by the process if you want to add the compute.googleapis.com or not. 
#: I pressed "y" and let it to enable the API. Some APIs need to be anabled for new projects.
#: Give it some time to run so you will see the file created at the bucket you created.  

#: list the bucket to verify if the file was already moved from the instance to the bucket
gsutil ls gs://mybuckwb01/machine-myvm01-finished.txt

* Corrected the set region as Mattias called the attention before.

I ran the above some times, and over the last three times, I got the final expected outcome:

wagnerbianchijr@cloudshell:~ (omega-presence-234823)$ gsutil ls gs://mybuckwb01/machine-myvm01-finished.txt  
gs://mybuckwb01/machine-myvm01-finished.txt

wagnerbianchijr@cloudshell:~ (mytestproject03097550)$ gsutil ls gs://mybuckwb02/machine-myvm01-finished.txt  
gs://mybuckwb02/machine-myvm01-finished.txt  

wagnerbianchijr@cloudshell:~ (mytestproject03097551)$ gsutil ls gs://mybuckwb027564/machine-myvm01-finished.txt  
gs://mybuckwb027564/machine-myvm01-finished.txt

So, you need to avoid spending money and then, follow what Mattias says, clean up your stuff after working on labs:

#: list projects
wagnerbianchijr@cloudshell:~ (mytestproject03097551)$ gcloud projects list  
PROJECT_ID             NAME                   PROJECT_NUMBER  
mytestproject03097551  mytestproject03097551  57519217666  

#: remove your recently created project
wagnerbianchijr@cloudshell:~ (mytestproject03097551)$ gcloud projects delete mytestproject03097551  
Your project will be deleted.  
Do you want to continue (Y/n)?  Y  
Deleted [https://cloudresourcemanager.googleapis.com/v1/projects/mytestproject03097551].  
You can undo this operation for a limited period by running '  
        commands below. See https://cloud.google.com/resource-manager/docs/creating-managing-projects for information on shutting down projects  
         $ gcloud projects undelete mytestproject03097551

Question: removing a project, so we remove everything created on or for that project. That is a good practice when we need to remove everything we created to avoid incurring charges?

Folks, let me know if it’s OK or not to publish answers for the challenges. I understand if not and won’t post more of this as I’m going through the training.

I hope it helps. Thanks!

Wagner Bianchi

It’s interesting to notice that, if you remove a project like I did above, you will end up receiving an email notification about recovering it (see the undelete command above). The documentation is clear on that, but, come of the features like the billing account connected to that deleted or shut down a project cannot be restored (https://cloud.google.com/resource-manager/docs/creating-managing-projects?visit_id=637122217442574083-2980679257&rd=1#shutting_down_projects). I also found an interesting video that does an overview of the recovery process and how Google delete data (https://youtu.be/ejBncCrlAqc). I also found threads over the internet discussing having a way to delete a project permanently. I’m not sure if it’s possible to delete a project permanently, it seems not.

Wagner Bianchi

By the way, after shutting down or delete a project, how long you have to execute the undelete? 30 days.

Swaminathan G

Instead of providing ‘y’ option, while creating instance. We hack it by using –quiet.

Swaminathan G

gcloud compute instances create myvm01 –metadata lab-logs-bucket=gs://mybuckwb027564/ –metadata-from-file startup-script=script.sh –scopes storage-rw,monitoring-write,logging-write –machine-type f1-micro –quiet

Wagner Bianchi

Uhuu! That’s a good trick, Swaminathan! Thanks for sharing!!

Thomas Scarpati

is this command needed when creating the project using cloud shell? –enable-cloud-apis

Cleanup Steps:

Delete Instance

gcloud compute instances delete myvm --quiet 

Delete Bucket

gsutil rm -r gs://bucket-name/

unlink the project from the billing account

gcloud beta billing projects unlink projectname

Verify,

gcloud beta billing accounts list

Copy the billing account number and use below command

gcloud beta billing projects list --billing-account=XXXXXX-XXXXXX-XXXXXX
gcloud projects delete projectname --quiet

Verify project deletion,

gcloud projects list
Wagner Bianchi

Awesome!!

Sign In
Welcome Back!

Psst…this one if you’ve been moved to ACG!

Get Started
Who’s going to be learning?