Has anyone started to switch their recursive dataflows to use the new Upsert function? Are you still planning to build recursive dataflows? I have a few but going back and fourth on if switching is worth it/is it best practice?
Converting to the new upsert will allow your dataflow to run quicker, as you'll only pull in the data that you're updating instead of the entire dataflow. With a recursive dataflow, it starts to run slower over time as it needs to pull in the ever-increasing dataset.
It also simplifies the logic of your data flow.
I'd recommend switching over.
I feel that there is still a use case for recursive dataflows when your data does not have a unique identifier. In order to perform and UPSERT operation you need to have a unique key, but a recursive doesn't have this limitation.
