Community Forums

Discover best practices and find answers to your questions with advice from fellow Domo users.

Recent Discussions

How to Use Partitioning to Retain Data in a Monthly Rolling Window Dataset
Hi Domo Community, I’m currently working with a dataset (via MongoDB Connector) that operates on a 12-month rolling window, where data older than 12 months is deleted on the 1st of each month. Long story short, I cannot change this configuration. But, I would like to retain some of this older data using partitioning before…
Partitioning for Dummies?
Can someone point me to a good step by step guide to setting up partitioning in Workbench 5.2? I have a very large dataset (currently 5.8 million rows) that gets updated daily with a full replace. As you can imagine it takes a HOT MINUTE to run. On average there are 12.5k new records and 100 records updated or deleted…
What's the best practice way to remove data from a partitioned dataset?
We have an ETL set up to a partitioned output. Now that we are far enough into 2024 I'm being told that we can reduce the size of this dataset by removing all 2022 data from it. What would be the best way to go about doing this?
Trying to Count Instances within certain date range where 1 field is the same
Hello! I am trying to replicate an excel formula to determine the number of times a caller calls within 4 days, 3 days, 2 days, and 1 day. I would like to create a field for each of these time windows. If someone calls 6 times within 4 days, I would like the 6th call to have the number 5 in it in the "4 day window" flag.…
for loop in Large amount of Data in Workbench.
Hello all, I'm tiring to pull large amount of data via Workbench, previously this was done with SSIS for loop container pulling only fraction of the time frame data at a time, and appending the results. is there way to achieve this in workbench as I'm not able to find the proper documentation.
Is there a way to rebuild a connector to allow for partitioning?
I use several connectors that do not currently have partitioning capabilities. Is there a way to rebuild them to allow this capability? For example, Facebook, LinkedIn, etc do not have the merge option.
Paritoning beta. Where to start?
I've got access to the partitioning beta and trying to figure out where to start. I have a number of legacy datasets (marketing platforms like Facebook, LinkedIn, Google Ads and Google Analytics) and dataflows that are clunky at best and go the long route to handle recursive data. I'm looking to simplify and make these…
Need help mimicking recursion with partitioned datasets
I need help implementing a particular use case for partitions: importing selected partitions from a dataset, modifying the records, then outputting the data back into it's original partition (or a new partition) in the original dataset. There are two uses for this I can think of off the bat: A dataset can have a maximum of…
Changing dataset update method from Replace to Partition doubles data
We've noticed that if we change the update method of an existing dataset to Partition from replace, we end up with two records in the dataset for every new one (and both of them look identical, down to the field we're using for partitioning): it was quite the shock to see a dataset I expected to have 91 million rows…
Using partitioning in ETL or Workbench
Good afternoon. Typically, to not do full replaces of data pulls into Domo from our databases, we have used recursive dataflows with ETL. I am trying to think of a way of accomplishing this using the partitioning features in ETL and/or workbench to avoid the ETL still having to process the whole set of updated+historical.…

Welcome!

It looks like you're new here. Members get access to exclusive content, events and more. Sign in or register to get started.