Is there an advantage to splitting a DataSet with a DataFlow over using a page/card filter?

We're using and learning Domo, and I'm checking out some alternatives as they come up. The Magic ETL DataFlow editor is slick. I'm not normall one for visual programming tools but, dang, that window is impressive.

Here's my test setup. I've got a DataSet with about 220K rows. It lumps together data from a variety of facilities into the one DataSource. This is very convenient for the import and seems fine. Once inside Domo most, if not all, cards and pages will only pull data from one facility out of the full DataSet. So, imagine out of the 220K rows that any one card/page only needs 5-10K of the records, max.

I just tried out a simple DataFlow that produces a new DataSet for one facility by using a simple filter rule. Here's the question, is there any point? I don't know enough (anything) about Domo's internals. Will I see any speed advantage to having pre-created a smaller DataSet for individual facilities? For all I know, the DataFlow is functionally a mask and I'll see no benefit. Or, it could be making a duplicate set with its own indexes and I will. And, for all I know, page filters may already do that sort of work behind the scenes automatically.

I know that my example row counts are small - but they'll grow. I figured I'd ask about alternatives now before we've sunk a lot of time into any particular approach.

Thanks very much! I've been getting great answers here, much appreciated.

Comments

Echelon

Here's an example that might help you think it through: for our org, we have about 3 products. But 90% of our focus and business is on product #1.

To have a data output that focuses by product, is helpful so that every time a new card is created from scratch, you don't have to add in that 1st layer filter of Product = 1. Especially if the Products would be siloed from each other anyway, there would be no reason to have them intermingle for cross comparison.

And if you're saving the card builders from a few extra steps, it will keep human error out of the equation as much as possible, and save time on the card building process.

ST_Superman

I feel that there are more advantages to keeping the dataset together as one big dataset. We have some pretty large data sets and I have not seen any noticeable delay in performance when using a large data set. The biggest advantage is that when you create a new card, even if it is for a specific facility, you could easily edit the card for the other facilities. Any beastmodes that were created on the dataset could be applied to all of the facilities as well. If you want to limit a user's access to data from other facilities, I would look into PDP solutions.

DataSquirrel

Thanks for the answers!

I built a pair of test charts drawn from the two different data sources (~200K with a fitler and ~5K with no filter) and they both render pretty much instantly. Domo is pretty mind-blowingly fast.

I like the idea of a single data set because it's less hassle in so many ways. However, we need to be sure not to leak data inappropriately, so the idea of a pre-filtered data set is also appealing. Building out that many data sets over time and adding new ones for new customers and locations would be a big pain. I'd figure we would need to look at the Publications Group features. That sounds like a very nice way to do, but it seems that the Pub Group's security interactions with PDP are...a bit confusing. Also, Pub Groups don't support SumoCards and some other interesting-sounding features.

Is there any sort of best practice or common pattern/strategy for dealing with data where the format is the same amongst customers but the content is private to each customer? (Conceputally, filters/views seem like a sensible approach.)

I'm new here and just trying to come to grips with the alternatives and best stategies before we get too locked into an approach that we latter find insecure or too restrictive. So thanks for any and all points of views and suggestions.

ST_Superman

I would recommend learning how to implement PDP in your instance. Contact your Domo CSM and see if they can put you in touch with someone to set this up. We have a fairly large instance and we are pushing our metrics out to our global salesforce. We don't want sales reps to have access to information for any account that is not theirs, and we only wanted our manager and area directors to have access to the customers in their area.

PDP is working beautifully to accomplish this. It also allows us to share any PDP enabled data set to our team and we don't have to worry about who sees what. If they have access to the account in our Salesforce instance, they can see it in Domo.

Quick Links

Find more posts tagged with

Other Categories

Product Ideas
Have a Domo product enhancement idea? Submit or upvote on ideas in the Ideas Exchange.
Ideas Exchange
Suggest & vote on new features you would like to see implemented in the Domo Product.
Data Connections
Ask questions about Connectors, Workbench, Cloud Amplifier and get best practices from Domo peers
Connectors
Connectors, Custom Connectors, Writeback
Workbench
Ask questions about Workbench, a secure, client-side solution for uploading your on-premise data to Domo.
Cloud Integrations
Ask questions about Cloud Integrations and Federated Data connection to your data warehouse or lake.
Data & ETL
Ask questions about Magic ETL, SQL DataFlows, DataFusion, Dataset Views and get best practices from Domo peers
Magic ETL
Ask Magic ETL questions and get answers from Domo peers
SQL DataFlows
Ask SQL DataFlow questions and get answers from Domo peers
Datasets
Ask DataFusion and Dataset Views questions and get answers from Domo peers
Visualize & Apps
Ask questions about Beast Mode, Cards, Charting, Dashboards, Stories, Variables and get best practices from Domo peers
Dashboards
Ask Cards, Dashboards, and Stories questions and get answers from Domo peers
App Studio
Ask questions about building apps in App Studio.
Pro-code Components
Ask questions about pro-code components and Domo Bricks and get answers from Domo peers.
Charting & Analyzer
Ask questions about charting and Analyzer and get answers from Domo peers.
Calculations & Variables (Beast Mode)
Ask questions about using calculated fields and Variables (Beast Modes) in Analyzer.
AI & Data science
Ask questions about DomoAI and get answers from Domo peers.
Domo AI & AI Chat
Ask questions about AI Chat and AI assistants.
Managing AI
Ask questions about managing AI with AI Playground, AI projects, AI models, and more.
Jupyter Workspaces
Ask questions about Jupyter Workspaces, Notebooks, and file share.
Automate
Ask questions about App Framework, Workflows, Domo Bricks, Domo Developer, API and get best practices from Domo peers
Workflows
Ask questions about Task Center, building automations with Domo Workflows, and executing JavaScript or Python code with Code Engine.
Alerts
Ask questions about managing alerts in Domo and get answers from Domo peers.
Distribute
Ask questions about Domo Everywhere, Scheduled Reports, Mobile and get best practices from Domo peers
Domo Everywhere
Ask questions about embedded analytics with Domo Everywhere.
Reporting
Ask questions about Scheduled Reports, Report Builder, and Slideshow Publications.
Manage
Ask questions about Governance Administration, Approvals, Teams, Alerts, and Buzz and get best practices from Domo peers
Governance & Security
Ask questions about People, Groups, Roles, Sandbox, Activity log, Buzz, Teams, Approvals and PDP and get best practices from Domo peers
Navigation & Productivity
Ask questions about navigation, Projects & Tasks, Goals, and Buzz chat.
APIs
Ask APIs and Developer.domo.com questions and get answers from Domo peers
Add-ins & Plugins
Ask questions about plugins, Microsoft add-ins, and other third-party software integrations.
Domo Community Gallery
Watch how our Customers are using Domo to solve their complex problems.
Product Releases
Domo support and product teams are here to live-answer questions about the most recent product releases. Please post questions in this Forum board for all users to benefit (rather than submitting a support ticket).
Domo University
Questions or discussions related to Domo University, trainings and certifications
Community Forums
Getting Started
Welcome to Domo's Community Forums! You'll find everything you need to get started in this category.
Community Announcements
Get the latest from Domo's Community Team.
Social Groups
Archive
Old or outdated content that could still be found helpful.