Odd issue with ETL and missing data

NathanDorsch

I have an ETL I'm working in which data from one of the input datasets is not being pulled through when I run the ETL. Oddly however, I can see the data in preview. In particular, the "Test 1" output dataset will show data when I view it in preview (see screenshot below), but the resulting dataset does not contain any data.

I am absolutely perplexed and growing increasingly frustrated with this. Has anyone come across a similar issue to this?

NathanDorsch

@ColemenWilson I finally discovered the issue from Domo support. Within the ETL, I had set up a filter by batch coming in from the input dataset…. not sure why the preview would still show the data, so apparently this is only applied when actually running the ETL.

Thanks again for jumping in a call with me the other day!

ColemenWilson

My first thought would be that the column is being removed in the Append Rows 1 step. You could switch it to "include all columns" if it isn't already and see if it comes through.

EDIT: re-read your post and to clarify, you are saying that for the test 1 output `Month` is not in the output?

NathanDorsch

thanks for the suggestion @ColemenWilson , but I do have (all) the append rows steps set to "include all columns".

ColemenWilson

If it is in the output dataset preview it should be in the output. I'd be happy to jump on a call with you and try and troubleshoot: https://calendly.com/cwilsondomo/15min

NathanDorsch

Thank you @ColemenWilson for willing to jump on a call with me! I scheduled something on Monday - appreciate you offering to help!

NathanDorsch

@ColemenWilson I finally discovered the issue from Domo support. Within the ETL, I had set up a filter by batch coming in from the input dataset…. not sure why the preview would still show the data, so apparently this is only applied when actually running the ETL.

Thanks again for jumping in a call with me the other day!

ColemenWilson

No problem! Glad support got it figured out and this is good to know. I assumed since the data was showing in the preview this wouldn't be applied. I will check this troubleshooting in the future!!