Azure pipelines - "1970, 'Leaf Error: Subprocess timed out'"

Help Connectors and Pipelines
1

Hi Team,

I get the following error in 2 out of 8 leaves when trying to start Azure pipeline:

(1970, 'Leaf Error (172.16.0.11:3306): Leaf Error (172.16.0.10:3306): Subprocess timed out.  Truncated stderr:\n')

I looked for this error in this forum and found out that it might be related to the global variable pipelines_extractor_get_offsets_timeout_ms
so i changed this value from 10000 to 600000 (10 minutes) and the error still occurs 4 minutes after starting the pipeline.

What is the best way to debug it? If the problem caused by Azure how can i view the error message generated by Azure?

2

Hi chen, thanks for trying Azure pipelines.

Could you confirm some more information about your cluster and pipeline?

  1. MemSQL version

  2. Is your cluster running on Azure or elsewhere?

  3. You able to create pipeline, but the error occurs when trying to start it, is that correct? Could you try to capture and show the pipeline debug logs?

    SET GLOBAL pipelines_extractor_debug_logging = ON;
    START PIPELINE ‘…’ FOREGROUND LIMIT 1 BATCHES;

    when done you can set the global variable back.

    SET GLOBAL pipelines_extractor_debug_logging = OFF;

3
  1. Memsql version 6.7.14
  2. Yes, the cluster is running on Azure virtual machines.
  3. Yes it occurs when trying to start the pipeline. Right now I can’t reproduce it but once it occurs again I’ll try it and share the logs.

Thanks for your assistance,
Chen.