automation - Automating Google BigQuery query without local access to APIs -
i'm trying automatically run bigquery query every week , save data onto computer.
ideally done through apis due issues proxies , firewalls in work cannot access apis on computer.
are there other options available me? have tried looking google compute engine i'm not quite sure if me
it necessary data bigquery well, standard gap reporting through webui not suffice. , query result in ~ 2,000,000 rows of data each week
as might know, bigquery doesn’t provide scheduling mechanism.
should use 3rd party solution or develop own.
example, can use app engine cron service automate bq query execution , exporting result gcs.
https://cloud.google.com/appengine/docs/java/config/cron https://cloud.google.com/appengine/features/#cron https://cloud.google.com/bigquery/bigquery-api-quickstart
to result gcs local computer still need sort of access cloud , use respective tool (see link below)
https://cloud.google.com/storage/docs/gsutil
Comments
Post a Comment