Loading data from Bigquery into Google storage bucket in CSV format
I am running a dataset in bigquery on a daily basis that I need to export to my google storage bucket. The dataset is larger than 10 MB, which means I cannot use application scripts.
Basically, I would like to automate the data loading with my bigquery script that exports the dataset as a CSV file to google storage.
Can anyone point me in the right direction in terms of which program / method to use. Also tell us about your experience.
thank
Here you can find some information on how to export data from BigQuery to Cloud Storage along with a sample written in Python.
https://cloud.google.com/bigquery/exporting-data-from-bigquery
You can implement a simple application running on App Engine that will contain a cron job that will run once a day and follow the steps outlined in the tutorial above.
https://cloud.google.com/appengine/docs/python/config/cron