Snapshot Fields Accuracy

helpme12345
helpme12345 Member
in Magic ETL

I'd like to create a new dataflow, preferably not MySQL, to show that the fields I intend to snapshot daily are actually being snapshotted. For more context, I have a list of companies with subscription updates that are currently being snapshotted, but I want to ensure that everyday, every company is included. Anyone have advice on how to tackle this?

Best Answer

  • GrantSmith
    GrantSmith Coach
    Answer ✓

    You can use a recursive dataflow and use a formula tile to calculate a TIMESTAMP on when the ETL runs using CURDATE() as the formula

    Ideally if you can calculate a run TIMESTAMP within your input dataset it would be more accurate as to when the data was actually pulled

    **Was this post helpful? Click Agree or Like below**
    **Did this solve your problem? Accept it as a solution!**

Answers

  • DashboardDude
    DashboardDude Coach

    Hi @renee12345 ,

    What's the datasource you're pulling data from? I ask because if there's something like a Last Run Date or something like that, you can leverage that.

    Circle.png

    John Le

    Are you on my newsletter? If not, signup here so you don't miss out on my Domo tricks, alerts about my webinars, cooking tips and more

    Signup here:

    https://www.dashboarddudes.com/newsletter

  • GrantSmith
    GrantSmith Coach
    Answer ✓

    You can use a recursive dataflow and use a formula tile to calculate a TIMESTAMP on when the ETL runs using CURDATE() as the formula

    Ideally if you can calculate a run TIMESTAMP within your input dataset it would be more accurate as to when the data was actually pulled

    **Was this post helpful? Click Agree or Like below**
    **Did this solve your problem? Accept it as a solution!**
  • helpme12345
    helpme12345 Member

    Thank you.

    I have the timestamp value, but how would I use that value to see if the number of companies, for example are accurately being snapshotted daily?

Categories