Create Pipeline to GCS

Hi,
What is the exact syntax needed to create a pipeline to GCS?
I understand it should be supported (for what exact version?)

I guess it is based on the following:
CREATE PIPELINE mypipeline AS
LOAD DATA S3 ‘my-bucket-name’
CONFIG ‘{“region”: “us-west-1”}’
CREDENTIALS ‘{“aws_access_key_id”: “your_access_key_id”, “aws_secret_access_key”: “your_secret_access_key”, [“role_arn”:“replace_with_your_role_arn”]}’
INTO TABLE my_table

Thanks!

Hello oerner,

Could you please clarify the meaning of GCS?

Thanks,

Hi,
meaning is Google Cloud Storage.
Using a pipeline to upload files from bucket.
For backup memsql is using the S3 api.
Can it be done the same with pipelines?
if yes - in what version with what exact syntax?
if no - are you planing on providing it? about when?

Thanks much.

Hi oerner,

I have reached out to one of our engineers. Expect an answer soon.

Thanks for your patience.

Ramesh Narayanan

Hi @oerner!

This feature is going to be released in 7.1 (mid-Spring, I believe).

Is this still relevant to you? If yes, what version of MemSQL are you currently running and what flavor: cloud (Helios) or self-managed? If there still is an interest, I’ll ask around to see if we have spare cycles to backport it or maybe even do a custom test-build.

Hi,
Yes, still relevant to us.
We’re running v6.8 in Kubernetes.
Soon we’ll move to v7.

Thanks,

Fresh off the press. Ingest from GSP is now supported

1 Like