Google Certified Professional Data Engineer

Sign Up Free or Log In to participate!

Can’t create cluster due to bucket not existing

I completed the first dataproc lab fine, but after deleting the cluster it left two temp and staging storage buckets behind. I deleted them both thinking just to clean up, however now I can’t create a cluster for the second lab as I get the error below. It makes me think the console is trying to reuse the previously created buckets but I deleted them. Does anyone have any advice please?  I obscured the GUID on the end of the bucket name.

INVALID_ARGUMENT: Google Cloud Storage bucket does not exist ‘dataproc-staging-europe-west2-random ID removed’

1 Answers

I managed to get around this and create the cluster by manually creating a staging bucket and selecting this in the advanced options on cluster creation.   It would be good to know if there is a better way I should have cleaned up the environment after the first lab though please if anyone has ideas?

Sign In
Welcome Back!

Psst…this one if you’ve been moved to ACG!

Get Started
Who’s going to be learning?