How to Transfer data from Google Drive to Google Cloud Bucket?

Today I’m writing a very important topic that is how you can transfer data from google drive https://drive.google.com/drive/my-drive to Google Cloud bucket storage https://cloud.google.com/. I know it's very time-consuming and frustrating when Google Colab resources (Ram, Disk Drive, GPU) are not enough to train your deep learning models or even you uploaded your data on Google Drive, and then you wanted to move to Google Cloud services with your data.

Well, there is no direct way to transfer the data from google drive to google cloud but we can sue a mediator which is Google Colab https://colab.research.google.com/. We need to follow these steps to transfer the data to google cloud buckets:

Mount Google Drive using Google Colab

from google.colab import drive
drive.mount(‘/content/gdrive’, force_remount=True)

Authenticating and Connecting with Google Cloud

from google.colab import auth

auth.authenticate_user()

ID_of_Project_at_google_cloud= ‘nifty-depth-12345’

!gcloud config set project {ID_of_Project_at_google_cloud}

!gsutil ls

Copying data from Google Drive to Google Cloud Bucket

gc_bucket_name= ‘ImageNet_bucket’

!gsutil -m cp -r /content/gdrive/My\Drive/Data/* gs://{gc_bucket_name}/

That’s it ✔️

If you want to see my other stories please visit https://imranzaman-5202.medium.com/.