By default the value of pipelines_max_offsets_per_batch_partition is 1,000,000. Which is the best value for it?
What is the impact in case if I increase or reduce the value of pipelines_max_offsets_per_batch_partition value ?
How to check which is the best value for my environment?
This likely depends a lot on your cluster setup and workload. Are these wide rows? How many leaves in the cluster? How much memory per leaf? How urgently do you need the data after loading?