Magic ETL/MySQL ETL and Datasets

MReznick
MReznick Member
in Connectors

Is there a way to see what ETL created a dataset?  If the ETL tool showes and error at 46,796,637 rows, how are there 85,722,082 rows in the dataset?

 

 

Best Answers

  • marcam
    marcam Domo Employee
    Answer ✓

    Looking back historically, we currently do not have a way to attribute a dataflow to a dataset. Please visit the Dojo Ideas Exchange http://dojo.domo.com/t5/Ideas-Exchange-suggest-and-vote/idb-p/Ideas. You can search existing ideas and then vote for it if it matches what you have in mind. If you cannot find it, please create a New Idea which can then be voted on by your fellow Dojo members.

     

    As far as best-practices moving forward, we recommend including the ETL attribution in the dataset title as part of your naming convention. Or you can  add the dataflow reference in the description of the related dataset by using the Edit Name & Description tool from the Wrench menu in the Detail view of the dataset.

    Domosapien
    **Say “Thanks" by clicking the thumbs up in the post that helped you.
    **Please mark the post that solves your problem by clicking on "Accept as Solution"
  • CoreVest
    CoreVest Member
    Answer ✓

    I have found that if i where to add the suffix or prefix of ETL to the actual dataset name, locating the dataflow is a lot easier. However, if there were a link from the dataset to the dataflow, that would be better.

  • MReznick
    MReznick Member
    Answer ✓

    please vote for the Idea

    https://dojo.domo.com/t5/Ideas-Exchange-suggest-and-vote/Linking-Back-to-Original-Dataflow-from-Dataset/idi-p/6995

Answers

  • marcam
    marcam Domo Employee

    Would you please elaborate a little more? I foresee several potential answers to this request however, I want to make sure I am answering your question succinctly.

     

    My initial response is yes... indirectly. You can see which datasets are resultant of dataflows. This is apparent by both the split downward arrow icon symbol on the dataset and then by the description under the title of the dataset. To see which ETL tool was used to buld the related dataflow, you will have to move to the DATAFLOWS  and look at the description on each DATAFLOW.

     

    The catch: you must have access to the ETL tool to see the attribution in DATAFLOWS. For example, if I have access to both Magic ETL(everyone) and MySQL or Redshift (both available via feature request) in my instance then I will only be able to see the datasets that are built with the ETL tools that I access to in my instance. If I do not have access to Redshift, then I will not see those dataflows even though I have cards built from those datasets in my instance.

     

    As for the error portion of your question, I'll need additional detail/clarification.

    Domosapien
    **Say “Thanks" by clicking the thumbs up in the post that helped you.
    **Please mark the post that solves your problem by clicking on "Accept as Solution"
  • MReznick
    MReznick Member

    A dataset has a Name "Prod Do Not Rent List Daily DataSet' that is set up in the ETL, then below it this information.

    "DataFlow 16,075 rows Last updated 3 hours ago".
     
    I would like to know which ETL 'Dataflow' created the DataSet.  It will make backtracking errors or modification to the Dataflow without opening ever Dataflow to find it.
     
    And I have a test dataflow that errorred out and when researching what might have happended I notice the variance in records.
     
     
  • marcam
    marcam Domo Employee
    Answer ✓

    Looking back historically, we currently do not have a way to attribute a dataflow to a dataset. Please visit the Dojo Ideas Exchange http://dojo.domo.com/t5/Ideas-Exchange-suggest-and-vote/idb-p/Ideas. You can search existing ideas and then vote for it if it matches what you have in mind. If you cannot find it, please create a New Idea which can then be voted on by your fellow Dojo members.

     

    As far as best-practices moving forward, we recommend including the ETL attribution in the dataset title as part of your naming convention. Or you can  add the dataflow reference in the description of the related dataset by using the Edit Name & Description tool from the Wrench menu in the Detail view of the dataset.

    Domosapien
    **Say “Thanks" by clicking the thumbs up in the post that helped you.
    **Please mark the post that solves your problem by clicking on "Accept as Solution"
  • CoreVest
    CoreVest Member
    Answer ✓

    I have found that if i where to add the suffix or prefix of ETL to the actual dataset name, locating the dataflow is a lot easier. However, if there were a link from the dataset to the dataflow, that would be better.

  • marcam
    marcam Domo Employee

    Can you provide a trimmed screenshot (not revealing any sensitive data) or the verbatim wording of the error you are receiving? It could be as simple as a timing error or a little more complex such as a syntax error.  I want to ensure that we are headed down the right path to resolution.

    Domosapien
    **Say “Thanks" by clicking the thumbs up in the post that helped you.
    **Please mark the post that solves your problem by clicking on "Accept as Solution"
  • MReznick
    MReznick Member
    Answer ✓

    please vote for the Idea

    https://dojo.domo.com/t5/Ideas-Exchange-suggest-and-vote/Linking-Back-to-Original-Dataflow-from-Dataset/idi-p/6995

Categories