Input Dataset Sorting for Magic ETL Previews

MichaelB

When running a preview I often find that the relevant data to my dataflow is in the new data at the bottom of the input dataset and thus when running a preview to test the effects of my dataflow or troubleshoot it none of the applicable data shows up. Often, that defeats the purpose of the preview and leaves me with no other option but to temporarily alter the input dataset's sorting in an earlier dataflow and plug that in instead for the sake of testing.

 

While such temporary measures are certainly possible they are far from efficient. I propose that an option for input sorting when running a preview would be both extremely helpful and fairly simple, particularly if the option to "read from the bottom" were the only one added. I'm sure many other options would be helpful but the tendency of SQL to put new data at the bottom of a table makes that one seem particularly valuable.

 

Please contact me for any questions or clarifications,

Michael Clark Bacon

jaeW_at_Onyx

you can add a filter to your dataset for just the data you're interested in.

OR you can build a dataset view which applies the filter and then feed that you your Magic 2.0 dataflow as a 'temp dataset'.

 

the upside of the second approach is you guarantee JUST the data you want (which means faster preivews and more control over debugging