Importing Data into Domo from our Connectors

Mike_Shrift
Mike_Shrift Member
edited March 2023 in Scheduled Reports

A lot of our data sources are becoming fairly large over a few million rows, and we have noticed refresh speeds getting much slower.

I think the solution is to have data appended as of last import rather than replace the whole thing? But I'm curious to know if anyone else has a better solution?

Tagged:

Comments

  • MarkSnodgrass
    MarkSnodgrass Coach

    If you are using Domo Workbench 5.1, they have the option of partition support, which can help when dealing with large datasets. Here's the KB article about it.

    https://domohelp.domo.com/hc/en-us/articles/360062446514-Workbench-5-1-Partition-Support

    If you are using another type of connector, appending is the most common way to minimize refresh speeds.

    **Check out my Domo Tips & Tricks Videos

    **Make sure to <3 any users posts that helped you.
    **Please mark as accepted the ones who solved your issue.
  • GrantSmith
    GrantSmith Coach

    You can also utilize UPSERT within workbench as well to update / create records so you only need to pull in the changes assuming your data doesn't get deleted out of your tables. If they do you'd need to use partitioning.

    **Was this post helpful? Click Agree or Like below**
    **Did this solve your problem? Accept it as a solution!**

Categories