Support for DATETIME(3) for Kafka Connect

Help Connectors and Pipelines
1

Hi—continuing my experimentation with MemSQL, I’ve noticed that Kafka Connect doesn’t work well with MemSQL as source with schema evolution. The reason is that Kafka Connect, when using the MySQL dialect, tries to use the DATETIME(3) type for new timestamp columns, but MemSQL only supports DATETIME and DATETIME(6). [1] [2]

Wondering if there are any plans to support to DATETIME(3) or any workarounds to this other than forking the Kafka Connect JDBC code?

FWIW, I was able to get schema evolution working by setting the dialect to VerticaDatabaseDialect which mostly mirrors the MySQL types.

However, this prevents upsert behaviour because upserts on Vertica dialect are not supported (not sure, but I’d imagine syntax would be different as well).

I know I can use MemSQL pipelines for Kafka ingestion, but this does not support schema evolution.

2

Hey Scott,

Thanks for your feedback. We don’t currently support datetime(3), as noted. However, we are planning on building scheme registry integration in a not-too-distant future version of memSQL, stay tuned!

2 Likes
3

@JoYo — sounds fascinating! Please let us know if you need a beta tester.