I’m trying to copy 4,000,000 objects from a multi-region bucket to a dual region bucket. I ran gsutil -m cp -p gs://nasunifiler5d346257-888e-4824-84ef-8629e79a5c56-13-0/ gs://temp_dual_region_test/ but after copying about 500,00 objects, the cloud shell timed out and the process died. Then tried nohup gsutil -m cp -np gs://nasunifiler5d346257-888e-4824-84ef-8629e79a5c56-13-0/ gs://temp_dual_region_test/ & it copied about another 500,000 objects and timed out again. Is there anything I can do to keep this cloud shell from existing after 500,000 objects, so that it runs in complete process without having to restart it over & over?
1 Answers
You can spin up micro-sized compute instance with proper IAM service account permissions, and execute it from there (send process to background with & etc), it is fully in your control in terms of compute lifecycle, compared to GCloud Console.