Can you use _Batch_Last_Run_ as the date for a recursive dataflow?

renee12345
renee12345 Member
edited April 2023 in Magic ETL

Have a dataset that doesn't have a timestamp that is being imported via connector. Can I use the batch_last_run as the date to create the recursive dataflow?

Best Answer

  • GrantSmith
    GrantSmith Coach
    Answer ✓

    The batch_last_run will be the value when your raw data is imported into the system, the currentdate in an ETL would be populated when the ETL actually runs. Depends on what you're wanting to track with your date.

    **Was this post helpful? Click Agree or Like below**
    **Did this solve your problem? Accept it as a solution!**

Answers

  • GrantSmith
    GrantSmith Coach

    You could use it as your partition key but if you're only wanting a single entry you'll want to convert it to a date first before using it as your partition key.

    **Was this post helpful? Click Agree or Like below**
    **Did this solve your problem? Accept it as a solution!**
  • renee12345
    renee12345 Member

    Or would it be helpful to create a tile with currentdate and run that through, and then proceed making it recursive?


    Or maybe I am asking if that would work similarly?

  • GrantSmith
    GrantSmith Coach
    Answer ✓

    The batch_last_run will be the value when your raw data is imported into the system, the currentdate in an ETL would be populated when the ETL actually runs. Depends on what you're wanting to track with your date.

    **Was this post helpful? Click Agree or Like below**
    **Did this solve your problem? Accept it as a solution!**

Categories