Domo Documentation

what if i posited the challenge to you / domo to give us a tool that allows us to document a data pipeline
what might the final functionality look like?

• generate a (set of) markdown or html document that provides a summary of the pipeline so that we could integrate that documentation with whatever 3rd party enterprise tools (alation / confluence) tools

features:

• allows us to create summaries of specific features of the pipeline
  • ex. if certification is enabled on a dataset, we can provide a description of what that certification means and it's included in the documentation
  • if we enable / disable pdp on datasets, we can provide an overview of how it works and optionally apply include it in our documentation
    • ex. "DASH_dataset" PDP policy is based on region and allows our sales directors ...
    • DWH_dataset does not include PDP because ...
• documentation configuration takes all the data currently available in the data lineage screen and allows us to add notes (and which can then be pretty printed in a nice document).
  • imagine being able to roll up the functionality of datalineage diagram into a flowchart (MIRO or LucidChart with all the notes you  usually spend hours agonzing to collate for the major domo.
• is configurable in that we can choose which datsets and dataflows appear in the documentation (b/c documentation for different audiences may need different levels of specificity
  • don't need to include DIM_number_generator or DIM_date
• documentation can capture update schedule and ingest method (REPLACE, APPEND, UPSERT) and include settings like identity columns or sample partition keys and notes.

Integration into some management tools
Day 1 necessity for sure is the ability to documentDay 2 ask would be to include the ability to tag everything included in this pipeline so it shows up in the future workspaces product.  or we can eventually write a script that says "delete everything without a project tag"Day 3 ask would be that the lineage / documentation tool can span publish / subscriber instances.  with control over level of detail.  For PA documenting all 3000 syndicated subscribers may be less interesting, but for use cases where customers have 3 or 100 instances, keeping track of where assets are published may be less obvious and more interesting.

• our users work in departmental instances.  each department starts in a _DEV or _SANDBOX environment before publishing to a PROD instance.  They may have a _QA instance before content goes to  a distribution hub (playstation.domo.com) and some content ends up in playstation-mobile.domo.com.  
• Data lineage is good, but beautiful documentation would be greater!

Why do we need this?
We are currently migrating 300 dashboards.  Each dashboard consists of N datasets.  It would be ideal to be able to have auto generated documentation so that 
P.P.S.  I write all this b/c i know you're making efforts to integrate tiles and hopefully tiles with Notes.  ETL documentation is very granular, but the big dream is how does all of that translate into documentation that spans an entire data pipeline.
AND FROM THERE.  How can domo leverage tagging, and knowledge graphs, contextualized by the words fed in via documentation to produce a recommendation system that does not necessarily need an LLM 🙂 to improve data discovery?