I have created a pipeline that as load data Kafka replace into table option and mapped JSON fields to table columns and one of the Field in JSON into a variable @cust_type. I want to load data only when @cust_type = ‘A’ and skip other messages.
I tried it however it’s failing and when I looked at the record that’s failing @cust_type is not ‘A’
Does pipeline first parse JSON fields and map to user defined variables and table columns before where condition is evaluated? I think my issue is certain fields do not exist in the JSON message when customer type is not equal to ‘A’