Magic ETL

Magic ETL

Data Processing using Partition

Why I am having data loss issue when using data partition in Domo ETL?

Welcome!

It looks like you're new here. Members get access to exclusive content, events, rewards, and more. Sign in or register to get started.
Sign In

Answers

  • @scpradhan Could you please share some details of how your ETL is set up?

  • You need to make sure the data partition is configured correctly. And make sure the partition key accurately segments your data without omitting relevant data.

    One of the things I watch closely in Domo ETL processing is duplicate records affecting joins. Number of rows can easily go whack. Normally expanding not loss. With less records I would recheck things like filters and formulas. And possibly calculations affected by NULLs. If values come out as null, the calculations could vacate. Review the use of COALESCE.

    ** Was this post helpful? Click Agree or Like below. **
    ** Did this solve your problem? Accept it as a solution! **

  • @MichelleH basically, in the ETL, we are pulling only the new data being appended in the input dataset and the partition is done by the date.

Welcome!

It looks like you're new here. Members get access to exclusive content, events, rewards, and more. Sign in or register to get started.
Sign In