I have a MemSQL kafka pipeline that has been stuck in a failed state for over a day now.
Here’s some debug info:
- All of the Global Variables are still set to their defaults.
- Error message from PIPELINES_ERRORS table:
Cannot get source metadata for pipeline
<pipeline>. Stderr: 2019-09-06 08:52:08.970 Waiting for next batch, 255 batches left until expiration. 2019-09-06 08:52:08.971 Batch starting with new consumer. 2019-09-06 08:52:09.019 Failed to get watermark offsets from Kafka with error Broker: Not leader for partition
- Confirmed that the Kafka topic itself is healthy, server responsive and all of that aka I’m certain that if I drop & recreate the memsql pipeline, things will work just fine.
- Looking into the PIPELINES_ERRORS & PIPELINES_BATCHES_SUMMARY tables, it seems to be still retrying & failing constantly. Except that the NUM_PARTITIONS value is now 0 for every try.
- MemSQL Version: 6.8.7, Kafka Server Version: 2.2.0
- Why is the memsql pipeline not able to reconcile the kafka topic metadata?
- What (setting?) do I need to change to make sure it auto-corrects when there is a leader change? I really don’t want to drop-recreate pipelines.
Thanks in advance.