Data Processing using Partition


Why I am having data loss issue when using data partition in Domo ETL?


  • MichelleH

    @scpradhan Could you please share some details of how your ETL is set up?

  • ArborRose

    You need to make sure the data partition is configured correctly. And make sure the partition key accurately segments your data without omitting relevant data.

    One of the things I watch closely in Domo ETL processing is duplicate records affecting joins. Number of rows can easily go whack. Normally expanding not loss. With less records I would recheck things like filters and formulas. And possibly calculations affected by NULLs. If values come out as null, the calculations could vacate. Review the use of COALESCE.

    ** Was this post helpful? Click Agree or Like below. **
    ** Did this solve your problem? Accept it as a solution! **

  • scpradhan

    @MichelleH basically, in the ETL, we are pulling only the new data being appended in the input dataset and the partition is done by the date.