ETL sometimes does bring information and in others it does not

Options
Rsandoval
Rsandoval Member
in Connectors 
I have a connection with QuickBase, sometimes when I execute my ETL it does bring the information correctly, but if it is executed more than once the result is zero records, does this happen to anyone else?


Comments

  • MichelleH
    MichelleH Coach
    Options

    @Rsandoval Does your output dataset actually contain 0 rows or does the dataflow history just show 0 rows updated? Depending on the schedule settings of your dataflow and Quickbase connector, it's possible that no rows were updated but the existing rows still exist in the output dataset.

    If a dataflow is triggered and the input datasets have not updated since the last run, then MagicETL v2 will not reprocess the data because the results would be the same. In this case, the dataflow history will show the number of rows changed rather than the total number of rows in your output dataset.

  • McSQL
    McSQL Contributor
    Options

    @Rsandoval It's hard to know exactly what is happening, but I did come up with one possible explanatino. There are some advanced settings that can be applied in Magic ETL inputs. It can be set to only process new rows (see 2 below in screenshot or in documentation https://domo-support.domo.com/s/article/360045402273?language=en_US#3.1.)

    If the input is set to only new rows, it can change the rows that are pulled in. Not sure if this is what you are running into, but I thought I would post.


    Screen Shot 2022-12-07 at 3.11.18 PM.png


    **Say “Thanks" by clicking the thumbs up in the post that helped you.
    **Please mark the post that solves your problem by clicking on "Accept as Solution"

Categories