Change Certification from Datasets to Dataflows


Certification is currently requested and managed via a dataset, however we've found that in some cases changes to the dataflow don't cause the certification to expire. This introduces potential risk in cases where changes could be made that affect the accuracy of the data, but the dataset could still be flagged as Certified.
Also, although it's a minor issue, there's no indication on a dataflow that an output dataset is certified or not. So a developer could mistakenly update a dataflow breaking certification.
Additional rules could be put in place from a DOMO technical perspective, however I believe a better option would be to move the certification to the dataflow.
By certifying the dataflow:
- The actual object that governs the data accuracy/quality in the dataset/s is certified
- This could enable a visual reminder within the ETL to a developer they are editing a certified object
- The output dataset/s could inherit the certification from the dataflow
- If certification is broken/expired, this could then result in the certification of the output dataset/s expiring
- Better management of the certification would be enabled through the versions of the dataflow, as soon as a new version is created, certification should break/expire
- It could also enable quick rectification of accidental certification expiration by reverting to the prior certified version of the dataflow
This next part is possibly a stretch, and would undoubtedly require some expansion to the metadata of datasets/dataflows, but this could also enable the connection of certification between multiple datasets within a lineage.
For example:
- dataflow B is certified using certified dataset A as a source
- someone breaks the certification on dataflow A
- dataflow B certification then breaks, triggering an alert to that dataflow owner/s and subsequently removing certification from dataset B
This would enable proactive management of full dataset lineages, encourage certification to be applied/broken taking downstream effects into consideration, and encourage the use of certified objects more due to the added governance this brings.
Categories
- All Categories
- 2K Product Ideas
- 2K Ideas Exchange
- 1.6K Connect
- 1.3K Connectors
- 311 Workbench
- 6 Cloud Amplifier
- 9 Federated
- 3.8K Transform
- 656 Datasets
- 115 SQL DataFlows
- 2.2K Magic ETL
- 811 Beast Mode
- 3.3K Visualize
- 2.5K Charting
- 80 App Studio
- 45 Variables
- 771 Automate
- 190 Apps
- 481 APIs & Domo Developer
- 77 Workflows
- 23 Code Engine
- 36 AI and Machine Learning
- 19 AI Chat
- AI Playground
- AI Projects and Models
- 17 Jupyter Workspaces
- 410 Distribute
- 120 Domo Everywhere
- 280 Scheduled Reports
- 10 Software Integrations
- 142 Manage
- 138 Governance & Security
- 8 Domo Community Gallery
- 48 Product Releases
- 12 Domo University
- 5.4K Community Forums
- 41 Getting Started
- 31 Community Member Introductions
- 113 Community Announcements
- 4.8K Archive