prioritize recent data for input datasets in dataflows for preview

WHM
WHM Contributor
in SQL Dataflows Ideas

We all run into the situation where we have two tables with hundreds of million of rows and we drop them in the dataflow and know that our join transforms will never match up. Could we get the most recent 10k rows from the input datasets rather than something seemingly random? That would really boost our ability to validate that SQL as we go along.

12
Up
12 votes

Active · Last Updated

Comments

  • DomoDork
    DomoDork Contributor

    I really need this. It's so hard to validate data without being able to control what records come through in a preview. I also really think a sort tile would be super useful. I realize you can do this with the Rank and Window tile, but dropping in a sort tile would be much simpler.

  • WHM
    WHM Contributor

    I would like to amend this request. Rather than simply prioritizing recent data, as I requested at Domopalooza 2024, I would like the ability to filter the datasets being brought in. Let me hit that 100k row limit based on attributes in the data. This would make dataflows an invaluable tool for tracking down data anomalies and simply understanding the datasets better without having to run the dataflow and check results.

  • DomoDork
    DomoDork Contributor

    @WHM - I've requested the same. I'd just like to see 'WHERE' clause features on input tiles so I can filter directly from the source. This would also reduce ETL processing time and load on downstream systems trying to feed all the data to Domo before it passes through a filter tile.

Categories