I’m trying to make transformation via Java because currently our topics contains messages are gzipped and serialized using Kryo,
Created simple application which return tab separated three fields in std.out
Tried several ways to but without success.
[2020-09-08 13:20:58] [HY000] Leaf Error (127.0.0.1:3306): Leaf Error (node-memsql-cluster-leaf-ag1-0.svc-memsql-cluster:3306): Transform for pipeline exited with nonzero exit code. Stderr: [2020-09-08 13:20:58] 2020-09-08 10:20:57.906 ERROR: Subprocess exited with failure result (2 : No such file or directory). Check that your transform has a proper shebang and is executable in this environment.
I have found folder with unpacked pipeline tarball and tried to run the same command from console - it works.
###### Table DROP TABLE messages; CREATE TABLE messages (v1 varchar(50), v2 varchar(50), v3 varchar(50)); ###### Pipeline DROP PIPELINE `quickstart_kafka`; CREATE PIPELINE `quickstart_kafka` AS LOAD DATA KAFKA 'kafka_topic' WITH TRANSFORM('http://184.108.40.206/pipeline7.tar.gz','run.sh','') INTO TABLE `messages` FIELDS TERMINATED BY '\t' ENCLOSED BY '' ESCAPED BY '\\' LINES TERMINATED BY '\n' STARTING BY '' ( v1, v2, v3 )
#!/bin/bash /var/lib/memsql/java/java-se-8u41-ri/bin/java -jar kafka_transformation-1.0-SNAPSHOT.jar
Cluster is running latest MemSQL on OpenShift,
You can examine tarball which is currently public available.