hello all
I am probably missing something simple, but cannot see what.
8.1’s Challenge Lab uses a startup script, which calls the metadata API endpoint to determine the contents of a label, which is then used to choose a storage bucket. But it keeps 404’ing at that point in the logs and when I manually do it from within the VM in question.
The startup script lines that generates the 404 were:
log_bucket_metadata_name=lab-logs-bucket
log_bucket_metadata_url="http://metadata.google.internal/computeMetadata/v1/instance/attributes/${log_bucket_metadata_name}"
worker_log_bucket=$(curl -H "Metadata-Flavor: Google" "${log_bucket_metadata_url}")which results in this being run:
curl -H 'Metadata-Flavor: Google' http://metadata.google.internal/computeMetadata/v1/instance/attributes/lab-logs-bucket"
and the 404.
The ‘lab-logs-bucket’ label I set is showing as a label with a name and value in the webconsole, but if I explore by manually running:
curl -H 'Metadata-Flavor: Google' http://metadata.google.internal/computeMetadata/v1/instanc e/attributes/
…all I see is one attribute, that contains the "startup-script". It’s as if labels are not stored there. They are not in tags/, which seems to be network tags only.
I haven’t edited the startup script, but even if I had, I should still be able to find the label that I set somewhere via curl. The documentation is also really not great on this, as everything seems to focus on looking up or adding tags, but only the addition of labels or using labels for filtering (unless I am being dumb).
I feel that I understand enough to skip this, but it is annoying me, I have lost quite a few hours on something that should have been simple :(( There is clearly something that I am missing…..
Does this make sense to anyone? is anyone else getting this?
6 Answers
ah, ok. Labels are not classed as "meta-data". Meta-data are meta-data….
The issue is with this line : worker_log_bucket=$(curl -H "Metadata-Flavor: Google" "${log_bucket_metadata_url}"), when you parse this variable to this line : gsutil cp "${worker_log_file}" "${worker_log_bucket}" it screws up…….. I created a random bucket and used that instead of using the "${worker_log_bucket}" variable in the above line, as such : gsutil cp "${worker_log_file}" gs://lab-logs-bucket-2020/ and it worked! Give that a go……
Hi there 🙂 The script does work ok though. The problem turned out to be me assuming that labels are classed as "meta-data" that are stored under that /attributes api endpoint. But they are not, VM ‘labels’ were not used in the lab, just ‘metadata.’ It’s a bit of silly contradicting mess from GCP, probably legacy debt though, I guess
got what issue is. I got the same error because I didn’t read carefully this line
Metadata should be set in the "lab-logs-bucket" attribute using the "gs://mybucketname/" format.
I edited script and put my name of bucket, but I should have to put my name backet into metadata before create instance.
Metadata: key is lab-logs-bucket and value is gs://lab-logs-bucket-ap.
I did exactly the same thing. I spent an hour thinking I was making a typo 🙂 I think that guidance is not worded well.
Yeah this tripped me up as well. I initially edited the init script to "gs://${worker_log_bucket}" even though it said not to, then went back and reformatted my metadata and used the original script. Second that the comment might be worded clearer for future folks.
I modified two sections in startup script and it worked:
1) Modified "log_bucket_metadata_url" to point to gs://bucket-name + removed "worker_log_bucket"
# Metadata should be set in the "lab-logs-bucket" attribute using the "gs://mybucketname/" format.
log_bucket_metadata_name=lab-logs-bucket
log_bucket_metadata_url="gs://${log_bucket_metadata_name}"2) Altered gsutil command to copy file using
# And we copy that file to the bucket specified in the metadata.
echo "Copying the log file to the bucket..."log_bucket_metadata_url" instead of "worker_log_bucket"
gsutil cp "${worker_log_file}" "${log_bucket_metadata_url}"After contemplating on this line in the startup script
# Metadata should be set in the "lab-logs-bucket" attribute using the "gs://mybucketname/" format.
and a bit of reading here –> Link
I was able to solve the mystery!!!
We need to add a command to add metadata : => lab-logs-bucket=gs://mybucketname, like below
gcloud compute instances add-metadata $(hostname) --metadata lab-logs-bucket=gs://mybucketname
log_bucket_metadata_name=lab-logs-bucket
log_bucket_metadata_url="http://metadata.google.internal/computeMetadata/v1/instance/attributes/${log_bucket_metadata_name}"
worker_log_bucket=$(curl -H "Metadata-Flavor: Google" "${log_bucket_metadata_url}")
Yes , the startup script is so confusing . How do they expect us to be ready for the exam in time if we have to troubleshoot their own examples. Again , this is not how you get us ready. My 2c, If I want to spend my time googling, I wouldn’t have payed for ACG subscription.
labels are not same as metadata. The GCP documentation says for metadata "~/attributes/":
"A directory of custom metadata values passed to the instance during startup or shutdown. See Specifying Custom Metadata below."
Custom metadata can be set in gcloud compute command with additional flag like this:
–metadata lab-logs-bucket=gs://your-bucket-name
Ref: https://cloud.google.com/compute/docs/storing-retrieving-metadata#custom
heavily edited, just in case this arrived anywhere by email first!