Google Cloud Platform

Ingesting data using Google Cloud Platform (GCP) requires configuring Google Cloud Storage (GCS) bucket access. You may configure SQreamDB to separate source and destination by granting read access to one bucket and write access to a different bucket. Such separation requires that each bucket be individually configured.

GCP Bucket File Location

GCP syntax to be used for specifying a single or multiple file location within a GCP bucket:

gs://<gcs path>/<gcs_bucket>/

GCP Access

Before You Begin

It is essential that you have a GCP service account string.

String example:

Granting GCP Access

  1. In your Google Cloud console, go to Select a project and select the desired project.

  2. From the PRODUCTS menu, select Cloud Storage > Buckets.

  3. Select the bucket you wish to configure; or create a new bucket by selecting CREATE and following the Create a bucket procedure, and select the newly created bucket.

  4. Select UPLOAD FILES and upload the data files you wish SQreamDB to ingest.

  5. Go to PERMISSIONS and select GRANT ACCESS.

  6. Under Add principals, in the New principals box, paste your service account string.

  7. Under Assign roles, in the Select a role box, select Storage Admin.

  8. Select ADD ANOTHER ROLE and in the newly created Select a role box, select Storage Object Admin.

  9. Select SAVE.


Optimize access time to your data by configuring the location of your bucket according to Google Cloud location considerations.


Using the COPY FROM command:

    name     TEXT,
    team     TEXT,
    number   TEXT,
    position TEXT,
    age      TEXT,
    height   TEXT,
    weight   TEXT,
    college  TEXT,
    salary   TEXT
WRAPPER csv_fdw
OPTIONS(location = 'gs://blue_docs/nba.csv');

Using the CREATE FOREIGN TABLE command:

  Name       TEXT,
  Team       TEXT,
  Number     TEXT,
  Position   TEXT,
  Age        TEXT,
  Height     TEXT,
  Weight     TEXT,
  College    TEXT,
  Salary     TEXT
 WRAPPER csv_fdw
   LOCATION =  'gs://blue_docs/nba.csv'