The challenge is a good start to understanding well how Google Cloud thought about the components of a small project. If we progress to the next lesson and watch Mattias speaking the small project, explaining it using the Data Flows to build a mental model of what happened behind the scenes, it gets more clear how things are integrated and how data flows.
Following my notes; I performed the GCS & GCE Challenge Lab using the Console and I would like to put together here some points of attention. I would like to also confess that I wouldn’t able to execute it myself without cheating a little bit, watching Mattias doing it as the "scopes" was the point which makes the total difference.
* GCE instance that runs provided the script (work on scopes here is VERY important): * SCOPES: attention to the Identity and API access having Storage to Write Only, and the Stackdriver Logging API, Stackdriver Monitoring API, and Stackdriver Trace to Write Only * expand the Management, security, disks, networking, sole tenancy and add the script to the Automation >> Startup script * attention toe the metadata as you need to map out the lab-logs-bucket=gs://mybucketname/
After creating the project some times using the Console, as said before, I started gathering the needed commands using gsutil and gcloud clients so I could repeat the same using only the Google Cloud Shell. I don’t want to share here the whole solution for that, but, to contextualize what I’m doing the problem I’m facing, I need to at least share the initial commands.
The whole idea is like below:
[email protected]:~ (challenge-lab-01-248121)$ gcloud projects create mytestproject03097557 --enable-cloud-apis Create in progress for [https://cloudresourcemanager.googleapis.com/v1/projects/mytestproject03097557]. Waiting for [operations/cp.7602138930524904622] to finish...done. Enabling service [cloudapis.googleapis.com] on project [mytestproject03097557]... Operation "operations/acf.59e7adb0-9367-4217-adee-4d2f9ec675c9" finished successfully. [email protected]:~ (challenge-lab-01-248121)$ gcloud projects list PROJECT_ID NAME PROJECT_NUMBER challenge-lab-01-248121 Challenge Lab 01 226890137415 directed-cove-247421 Services Exploration Labs 881680634241 mytestproject03097548 mytestproject03097548 41810333757 mytestproject03097557 mytestproject03097557 592725485970 neon-research-247220 Admin Project 465501537145 omega-presence-234823 My Very First Project 523600421491 [email protected]:~ (challenge-lab-01-248121)$ gcloud config set project mytestproject03097557 Updated property [core/project]. [email protected]:~ (mytestproject03097557)$ gcloud config set compute/region us-west2-b Updated property [compute/region]. [email protected]:~ (mytestproject03097557)$ gcloud config set compute/zone us-west2-b Updated property [compute/zone]. [email protected]:~ (mytestproject03097557)$ gcloud config configurations list NAME IS_ACTIVE ACCOUNT PROJECT DEFAULT_ZONE DEFAULT_REGION cloudshell-5021 True mytestproject03097557 us-west2-b us-west2-b [email protected]:~ (mytestproject03097557)$ gsutil mb gs://mybuckwb01 Creating gs://mybuckwb01/... AccessDeniedException: 403 The project to be billed is associated with an absent billing account.
After receiving the above message "AccessDeniedException: 403 The project to be billed is associated with an absent billing account.", I set the core/account and even with that, I don’t see yet a solution. Maybe you can share something with me to fix that:
[email protected]:~ (mytestproject03097557)$ gcloud config set core/account [email protected] Updated property [core/account]. [email protected]:~ (mytestproject03097557)$ gcloud config configurations list NAME IS_ACTIVE ACCOUNT PROJECT DEFAULT_ZONE DEFAULT_REGION cloudshell-5021 True [email protected] mytestproject03097557 us-west2-b us-west2-b [email protected]:~ (mytestproject03097557)$ gsutil mb gs://mybuckwb01 Creating gs://mybuckwb01/... AccessDeniedException: 403 The project to be billed is associated with an absent billing account.
I wonder if is there something I’m missing on the command line to connect the project with a billing account.
Any thoughts? Thanks!!
Alright! Nicely done, Wagner! And don’t think that what you’ve done is “cheating”, as I’ve intentionally tried to design the course to support you through the challenge at whatever your current level is. 😛 And it looks to me like you’ve used this challenge to very good effect! You got through the console lab with just a few hints, and then you also did the CLI lab. So 👍 to you!
A small thing I noticed, as I read through your post, is that you set “compute/region” to a zone instead of a region. 🙂 Just something you can try to remember for the future.
And then big kudos to you for debugging your problem with the billing account not being associated by default to projects created via gcloud! I guarantee you that struggling a bit with it like you did will now have locked that into your mind. I hope you’ve been enjoying this process, too. What you’ve been doing is a lot like what a Cloud Engineer does, day to day.
Thanks for taking on the challenge and for posting your experience with it. 👏
Mattias, is there any CLI solution to this lab that doesn’t rely on alpha software? It seems that linking a billing account to a project would be a common requirement, and I’m surprised that a CLI command doesn’t already exist for this.
Well, this has been in beta–not just alpha–for quite some time, now (years, I think?). And although Google’s "beta" bar is actually quite solid, it does make me sad that they haven’t gotten around to promoting it to full GA, yet. https://cloud.google.com/sdk/gcloud/reference/beta/billing/accounts/
Hey Mattias, I’m getting back to it now, after some time, and looks like that! The region, in this case, should be us-west2, and the zone should be the us-west2-b. I will post soon the complete CLI commands I used to get this challenge done using the CloudSDK. Thanks once again!
yes, beta is more stable. So I have tried like
gcloud beta billing projects link <project-name> --billing-account=XXXXXX-XXXXXX-XXXXXX
Hello folks, how are you?
I’m getting back to the ACG right now and recapping on things here.
So, I executed the GCE & GCS Challenge Lab today again, and the commands for that would be like below:
#: create the new project, you probably need to change the below one as it should be unique gcloud projects create mytestproject03097551 --enable-cloud-apis #: enable the billing account for the created project #: list billing accounts gcloud alpha billing accounts list #: link the project and the billing acocunt #: (fix the "AccessDeniedException: 403 The project to be billed is associated with an absent billing account") gcloud alpha billing projects link mytestproject03097551 --billing-account XXXXXXXXXX #: list projects gcloud projects list #: set the current session project's gcloud config set project [PROJECT_ID] #: set the default region and the default zone on the cloud shell gcloud config set compute/region us-west2 gcloud config set compute/zone us-west2-b #: list configurations you have currently gcloud config configurations list #: create the bucket - the name should be unique, you probably need to change the below one** gsutil mb gs://mybuckwb027564/ #: create the instance with the userdata script, the metadata for the bucket and the scopes for #: monitoring, storage and logging. I placed the Mattias script at the home of my user in the #: Cloud Shell, where I executed the below gcloud compute instances create myvm01 --metadata lab-logs-bucket=gs://mybuckwb027564/ --metadata-from-file startup-script=script.sh --scopes storage-rw,monitoring-write,logging-write --machine-type f1-micro #: here you'd be argued by the process if you want to add the compute.googleapis.com or not. #: I pressed "y" and let it to enable the API. Some APIs need to be anabled for new projects. #: Give it some time to run so you will see the file created at the bucket you created. #: list the bucket to verify if the file was already moved from the instance to the bucket gsutil ls gs://mybuckwb01/machine-myvm01-finished.txt
* Corrected the set region as Mattias called the attention before.
I ran the above some times, and over the last three times, I got the final expected outcome:
wagne[email protected]:~ (omega-presence-234823)$ gsutil ls gs://mybuckwb01/machine-myvm01-finished.txt gs://mybuckwb01/machine-myvm01-finished.txt [email protected]:~ (mytestproject03097550)$ gsutil ls gs://mybuckwb02/machine-myvm01-finished.txt gs://mybuckwb02/machine-myvm01-finished.txt [email protected]:~ (mytestproject03097551)$ gsutil ls gs://mybuckwb027564/machine-myvm01-finished.txt gs://mybuckwb027564/machine-myvm01-finished.txt
So, you need to avoid spending money and then, follow what Mattias says, clean up your stuff after working on labs:
#: list projects [email protected]:~ (mytestproject03097551)$ gcloud projects list PROJECT_ID NAME PROJECT_NUMBER mytestproject03097551 mytestproject03097551 57519217666 #: remove your recently created project [email protected]:~ (mytestproject03097551)$ gcloud projects delete mytestproject03097551 Your project will be deleted. Do you want to continue (Y/n)? Y Deleted [https://cloudresourcemanager.googleapis.com/v1/projects/mytestproject03097551]. You can undo this operation for a limited period by running ' commands below. See https://cloud.google.com/resource-manager/docs/creating-managing-projects for information on shutting down projects $ gcloud projects undelete mytestproject03097551
Question: removing a project, so we remove everything created on or for that project. That is a good practice when we need to remove everything we created to avoid incurring charges?
Folks, let me know if it’s OK or not to publish answers for the challenges. I understand if not and won’t post more of this as I’m going through the training.
I hope it helps. Thanks!
It’s interesting to notice that, if you remove a project like I did above, you will end up receiving an email notification about recovering it (see the undelete command above). The documentation is clear on that, but, come of the features like the billing account connected to that deleted or shut down a project cannot be restored (https://cloud.google.com/resource-manager/docs/creating-managing-projects?visit_id=637122217442574083-2980679257&rd=1#shutting_down_projects). I also found an interesting video that does an overview of the recovery process and how Google delete data (https://youtu.be/ejBncCrlAqc). I also found threads over the internet discussing having a way to delete a project permanently. I’m not sure if it’s possible to delete a project permanently, it seems not.
By the way, after shutting down or delete a project, how long you have to execute the
undelete? 30 days.
Instead of providing ‘y’ option, while creating instance. We hack it by using –quiet.
gcloud compute instances create myvm01 –metadata lab-logs-bucket=gs://mybuckwb027564/ –metadata-from-file startup-script=script.sh –scopes storage-rw,monitoring-write,logging-write –machine-type f1-micro –quiet
Uhuu! That’s a good trick, Swaminathan! Thanks for sharing!!
is this command needed when creating the project using cloud shell? –enable-cloud-apis
gcloud compute instances delete myvm --quiet
gsutil rm -r gs://bucket-name/
unlink the project from the billing account
gcloud beta billing projects unlink projectname
gcloud beta billing accounts list
Copy the billing account number and use below command
gcloud beta billing projects list --billing-account=XXXXXX-XXXXXX-XXXXXX
gcloud projects delete projectname --quiet
Verify project deletion,
gcloud projects list
I found another thread here on the forum where the discussion is almost the same: https://tinyurl.com/y5s4p4up
I don’t really know the reason for this yet but I will carry on hunting the answer. When I change the Google Shell project to the default one, the bucket creation works well. It seems something related to the project creation and my next step will be to look into the command to create the new project. It’s may be missing something else.
The solution is, you need to list your billing accounts and link the project with an existing billing account, See more here: The solution is here: https://cloud.google.com/sdk/gcloud/reference/alpha/billing/
Wagner, thanks for listing these details. I struggled with the same issue. I did see some postings to the alpha command, but I ignored that because I figured there MUST be a solution that didn’t rely on alpha software. I guess not! Your solution worked for me.
Thanks Wagner, I too struggled with the same issue.