Why is my input data source from an existing dataflow not updating?

a_mathew

First off, I'm a Domo newbie, only been working with the program for a couple of months now.

I discovered an issue with an ETL I've been working on that uses a colleague's data set as an initial input.

The source table itself is updating just fine, showing recent data. But when I look at the input data source, I only show up to about 4 days prior to the original version.

Some Steps I've done:

-Unlink and re-create the connection to the table

-Close all windows and launch brand new sessions

-Checked the original dataflow and confirmed the output table shows the up to date inputs.

Has anyone else dealt with something like this in the past? Any help would be appreciated!

I'm hoping to get my company's Domo primes to help, but this also seemed a good place to check. My leader, who is a Domo pro, has no clue why this is happening.

ArborRose

Have you checked the dataflow schedule? Make sure the dataflow is scheduled to run regularly. Sometimes, the dataflow may not be set to refresh automatically, which would cause the input data to lag behind the source table. (Let us know if you need some assistance understanding how to check or setup the schedule.)

Domo uses versioning for dataflows. Sometimes, an older version might be running, and despite updates to the source table, the dataflow might still be using cached or outdated data. Check if the dataflow is using the latest version, and consider forcing a manual run of the dataflow.

ETL Caching. Sometimes you need to clear cache.

Check you transformations or filters in your ETL. If any transformations are based on date fields or other that may have changed in the source date, it could be filtering out newer data. Also, make sure the ETLs are treating dates as dates. Sometimes you may have a date field that the dataset thinks is text or some other date type. This will change the behavior.

Sometimes, a dataflow might not properly detect updates in the source data. You may need to delete and re-add the input dataset in the dataflow to make sure it gets the right dataset. Or even recreate the dataflow if you think its glitched up somehow (which I haven't seen myself).

If all else fails….check with Domo Support. But, I think you'll get it fixed quicker through responses from us (Forum Community).

ArborRose

Have you checked the dataflow schedule? Make sure the dataflow is scheduled to run regularly. Sometimes, the dataflow may not be set to refresh automatically, which would cause the input data to lag behind the source table. (Let us know if you need some assistance understanding how to check or setup the schedule.)

Domo uses versioning for dataflows. Sometimes, an older version might be running, and despite updates to the source table, the dataflow might still be using cached or outdated data. Check if the dataflow is using the latest version, and consider forcing a manual run of the dataflow.

ETL Caching. Sometimes you need to clear cache.

Check you transformations or filters in your ETL. If any transformations are based on date fields or other that may have changed in the source date, it could be filtering out newer data. Also, make sure the ETLs are treating dates as dates. Sometimes you may have a date field that the dataset thinks is text or some other date type. This will change the behavior.

Sometimes, a dataflow might not properly detect updates in the source data. You may need to delete and re-add the input dataset in the dataflow to make sure it gets the right dataset. Or even recreate the dataflow if you think its glitched up somehow (which I haven't seen myself).

If all else fails….check with Domo Support. But, I think you'll get it fixed quicker through responses from us (Forum Community).