row count at each step

ydigilov
ydigilov Member
in Magic ETL Ideas

You should be able to see the # of rows output after each step of the etl flow.

10
Up
10 votes

In Review · Last Updated

I love this idea, and it's something I've been thinking about as well. I will review with the team.

Comments

  • AndreaHenderson
    AndreaHenderson Domo Product Manager

    I love this idea, and it's something I've been thinking about as well. I will review with the team. I had a follow-up question - would you want to see these row counts while you're authoring the DataFlow (as part of the Preview process), or after the DataFlow runs in the DataFlow Execution Details?

    Domo Product Manager for Data Transformation (MagicETL)

  • jessdoe
    jessdoe Contributor

    Hi @AndreaLovesData, I'm not sure which option @ydigilov would prefer but for me, having this ability while authoring the dataflow as part of the previous process would be awesome.

  • jessdoe
    jessdoe Contributor

    Being able to see row counts without having to run the ETL would allow you to more quickly ad easily see errors in your etl build-out.

  • ydigilov
    ydigilov Member

    @AndreaLovesData Just seeing now that you asked that question. I don't care if I have to hit run preview or if it automatically shows - if that's what you're asking. I just hate to have to output a new dataset at each step to see how many rows there are. I would hope that at least after hitting run preview, it would give us how many rows are in each step.

  • Recluz
    Recluz Member

    I just came on to add this. It would be so helpful to make sure filters, remove duplicates, and joins are behaving as expected. Especially in consumption models where you don't want to run it and then find out there is a problem.

Categories