Skip to content

    • Magic ETL

    Magic ETL

    Snapshot Fields Accuracy

    Member
    in Magic ETL

    I'd like to create a new dataflow, preferably not MySQL, to show that the fields I intend to snapshot daily are actually being snapshotted. For more context, I have a list of companies with subscription updates that are currently being snapshotted, but I want to ensure that everyday, every company is included. Anyone have advice on how to tackle this?

    Welcome!

    It looks like you're new here. Members get access to exclusive content, events, rewards, and more. Sign in or register to get started.
    Sign In

    Best Answer

    • Coach
      Answer ✓

      You can use a recursive dataflow and use a formula tile to calculate a TIMESTAMP on when the ETL runs using CURDATE() as the formula

      Ideally if you can calculate a run TIMESTAMP within your input dataset it would be more accurate as to when the data was actually pulled

      **Was this post helpful? Click Agree or Like below**
      **Did this solve your problem? Accept it as a solution!**

    Answers

    • Coach

      Hi @renee12345 ,

      What's the datasource you're pulling data from? I ask because if there's something like a Last Run Date or something like that, you can leverage that.

      Circle.png

      John Le

      Are you on my newsletter? If not, signup here so you don't miss out on my Domo tricks, alerts about my webinars, cooking tips and more

      Signup here:

      https://www.dashboarddudes.com/newsletter

    • Coach
      Answer ✓

      You can use a recursive dataflow and use a formula tile to calculate a TIMESTAMP on when the ETL runs using CURDATE() as the formula

      Ideally if you can calculate a run TIMESTAMP within your input dataset it would be more accurate as to when the data was actually pulled

      **Was this post helpful? Click Agree or Like below**
      **Did this solve your problem? Accept it as a solution!**
    • Member

      Thank you.

      I have the timestamp value, but how would I use that value to see if the number of companies, for example are accurately being snapshotted daily?

    Welcome!

    It looks like you're new here. Members get access to exclusive content, events, rewards, and more. Sign in or register to get started.
    Sign In

    Welcome!

    It looks like you're new here. Members get access to exclusive content, events, rewards, and more. Sign in or register to get started.
    Sign In

    Quick Links

    Categories