Datasets stuck refreshing

1. So I have a SQL Postgres database that I am uploading to domo via Workbench via 6 ODBS different datasets. (lets call them 1, 2, 3, 4, 5, 6) Each dataset is updated every 30mins from the Workbench - these work like clockwork without a problem.

2. Each dataset is then modified in ETL with some calculations and form in turn 1a, 2a, 3a, 4a, 5a, 6a. These Dataflows are automatically triggered each 30mins by the ODBS refreshes. Normally an update takes 8-12 secs. These datasets regularly get stuck on "running" and go on "running" for an hour or so, then cancel and restart. I would like to be able to troubleshoot and/or understand what causes it to get stuck like that because in turn it causes following dataflows to get stuck as well...

3. 1a, 2a, 3a, 4a, 5a, 6a are then combined into two separate datasets with the "join" command and other calculation (lets call them X and Y). X and Y can be triggered by an update in any of the 1a, 2a, 3a, 4a, 5a, 6a datasets. But the same thing happens they get stuck for an hour without refreshing - Most likely it is caused by the 1a, 2a, 3a, 4a, 5a, 6a getting stuck but just to confirm I am asking here....

anyone with any suggestions will be very welcome!

Thank you!

Comments

It sounds like an action in your ETL is just getting hung up, after which the process resets.

Have you viewed the dataflow history tab that shows what happened at each step? This screen is helpful to view where breakdowns happen and why. On the last column, status, the green or red Success or Failed icon is a link. If you click on that it will take you to the status of each step in the dataflow. Hover over each step status to find the tooltip error message. My guess is it's on a join step.

failure status.png

Let us know how it goes.

IVG

The problem is I don't get fails - what happens is that the the etl runs for an hour or so (50min to 1hr 20min) and then it cancels and autorestarts and then in most cases it goes right through automatically - but sometimes it can again hang for an hour. If I click on details - nothing shows because it did not"fail" it autocancelled...

Screen Shot 2018-02-23 at 12.15.42 PM.png

That's frustrating. And running dataflows don't show that detail.

One of my dataflows just now had the same problem and I can't tell either. Usually it runs in a few minutes, but this one autocancelled at 45 minutes

This almost sounds like a processing engine problem. Like a system hiccup, especially if those dataflows are all the same version. I don't think users have the details available to diagnose system problems like that.

IVG

well my problem is I am using a free account for now and the support is refusing to talk to me anylonger because I have already asked too many questions... so hence I am here trying to get community help... so not sure what I should try next...

Domo stores a bunch of metadata about each dataflow run.

How long it takes to load each dataset (ms)

How long it takes to run each transformation

How long it takes to load the output to permanent storage

How many rows are loaded

Timestamps at each step

How many rows are written

Etc.

I did some more research on the similar error that I had this morning, and my delays were related to dataset load times. You get a hint of this when you can see how many rows were loaded compared to the other successful runs.

I can't say that my failure is the same as yours, but I'm starting to think it has to do with failures to finish loading data into the dataflow for processing. There's likely nothing you or I can do about the system not loading your data properly for processing, especially when it's intermittent like that, except to lobby for improved error handling and debugging.

Maybe someone out there in the Dojo has some other ideas.

IVG

so far my datasets are dealing with a minimal number of rows that are added each time... i would say at most 10 rows would be added to a dataset on each refresh... as my SQL updated every 10 mins and the workbench every 30 - meaning if everything goes according to plan 3 rows of data is added every 30 mins but if the dataflow gets stuck and only updated every hour or hour and a half then additional rows would be added on the next successful refresh... ultimately my SQL DB is very small so far - I am not dealing with refreshing millions of rows... so it is very strange behaviour.... in any event I appreciate all the help you have provided so far!

Quick Links

Find more posts tagged with

Other Categories

Product Ideas
Have a Domo product enhancement idea? Submit or upvote on ideas in the Ideas Exchange.
Ideas Exchange
Suggest & vote on new features you would like to see implemented in the Domo Product.
Data Connections
Ask questions about Connectors, Workbench, Cloud Amplifier and get best practices from Domo peers
Connectors
Connectors, Custom Connectors, Writeback
Workbench
Ask questions about Workbench, a secure, client-side solution for uploading your on-premise data to Domo.
Cloud Integrations
Ask questions about Cloud Integrations and Federated Data connection to your data warehouse or lake.
Data & ETL
Ask questions about Magic ETL, SQL DataFlows, DataFusion, Dataset Views and get best practices from Domo peers
Magic ETL
Ask Magic ETL questions and get answers from Domo peers
SQL DataFlows
Ask SQL DataFlow questions and get answers from Domo peers
Datasets
Ask DataFusion and Dataset Views questions and get answers from Domo peers
Visualize & Apps
Ask questions about Beast Mode, Cards, Charting, Dashboards, Stories, Variables and get best practices from Domo peers
Dashboards
Ask Cards, Dashboards, and Stories questions and get answers from Domo peers
App Studio
Ask questions about building apps in App Studio.
Pro-code Components
Ask questions about pro-code components and Domo Bricks and get answers from Domo peers.
Charting & Analyzer
Ask questions about charting and Analyzer and get answers from Domo peers.
Calculations & Variables (Beast Mode)
Ask questions about using calculated fields and Variables (Beast Modes) in Analyzer.
AI & Data science
Ask questions about DomoAI and get answers from Domo peers.
Domo AI & AI Chat
Ask questions about AI Chat and AI assistants.
Managing AI
Ask questions about managing AI with AI Playground, AI projects, AI models, and more.
Jupyter Workspaces
Ask questions about Jupyter Workspaces, Notebooks, and file share.
Automate
Ask questions about App Framework, Workflows, Domo Bricks, Domo Developer, API and get best practices from Domo peers
Workflows
Ask questions about Task Center, building automations with Domo Workflows, and executing JavaScript or Python code with Code Engine.
Alerts
Ask questions about managing alerts in Domo and get answers from Domo peers.
Distribute
Ask questions about Domo Everywhere, Scheduled Reports, Mobile and get best practices from Domo peers
Domo Everywhere
Ask questions about embedded analytics with Domo Everywhere.
Reporting
Ask questions about Scheduled Reports, Report Builder, and Slideshow Publications.
Manage
Ask questions about Governance Administration, Approvals, Teams, Alerts, and Buzz and get best practices from Domo peers
Governance & Security
Ask questions about People, Groups, Roles, Sandbox, Activity log, Buzz, Teams, Approvals and PDP and get best practices from Domo peers
Navigation & Productivity
Ask questions about navigation, Projects & Tasks, Goals, and Buzz chat.
APIs
Ask APIs and Developer.domo.com questions and get answers from Domo peers
Add-ins & Plugins
Ask questions about plugins, Microsoft add-ins, and other third-party software integrations.
Domo Community Gallery
Watch how our Customers are using Domo to solve their complex problems.
Product Releases
Domo support and product teams are here to live-answer questions about the most recent product releases. Please post questions in this Forum board for all users to benefit (rather than submitting a support ticket).
Domo University
Questions or discussions related to Domo University, trainings and certifications
Community Forums
Getting Started
Welcome to Domo's Community Forums! You'll find everything you need to get started in this category.
Community Announcements
Get the latest from Domo's Community Team.
Social Groups
Archive
Old or outdated content that could still be found helpful.