Download the Schema of a dataset

Brian
Brian Member
in Magic ETL

Is there not a way to download just the schema of a dataset? I'm doing some documentation and it would be really helpful if I didn't have to retype the field name and types of >100 fields.

 

TIA!

 

 

Comments

  • guitarhero23
    guitarhero23 Contributor

    Probably not the answer you're looking for but I think you can only do something like that by utilizing the API.

     

    Ex. I have a short python script that I have to get a specific datasets details which includes information about the dataset and the schema. So via the API you could get a list of all dataset ID's in  your system and then loop through those and create the schema for all of them assuming you know how.

     

    {'columns': 480,
    'createdAt': '2017-01-11T17:53:17Z',
    'dataCurrentAt': '2019-04-30T07:23:12Z',
    'description': 'Select All on JIRA',
    'id': '45hgrdd3-1w4r-789r-j65f-4aeb1a20g357',
    'name': 'JIRA - All Data',
    'owner': {'id': 1686765431, 'name': 'Travis Simpson'},
    'pdpEnabled': False,
    'policies': [{'filters': [],
    'groups': [],
    'id': 51,
    'name': 'All Rows',
    'type': 'open',
    'users': []}],
    'rows': 39021,
    'schema': {'columns': [{'name': 'issue_id', 'type': 'LONG'},
    {'name': 'issue_key', 'type': 'STRING'},
    {'name': 'Status Category Changed', 'type': 'STRING'},
    {'name': 'Resolution', 'type': 'STRING'},
    {'name': 'Release - Distribution List Notification',
    'type': 'STRING'},
    {'name': 'Parent Link_hasEpicLinkFieldDependency',
    'type': 'STRING'},
    {'name': 'Parent Link_showField', 'type': 'STRING'},

    'updatedAt': '2019-04-30T07:23:12Z'}


    **Make sure to like any users posts that helped you and accept the ones who solved your issue.**
  • guitarhero23
    guitarhero23 Contributor

    My original message dissapeared so re-adding, sorry if you get notified twice.

     

    This probably isn't the answer you want but I think it's only possible via the API. I have a script that gets the response from one of the API endpoints which returns the text seen below. You could do the following to generate the schema for all datasets programatically.

     

    1: GET request to the datasets endpoint which would return information around ALL datasets in your system, which includes the dataset id.

    2: Loop through all dataset id's found in #1 and make a GET request to datasets endpoint and supply the dataset ID. It will return info specific to that dataset including the schema which you could deserialize and format. (response example below).

     

    {'columns': 480,
    'createdAt': '2017-01-11T17:53:17Z',
    'dataCurrentAt': '2019-04-30T07:23:12Z',
    'description': 'Select All on JIRA',
    'id': '87cfecc2-0d9g-447r-t65e-4aeb1b43d678',
    'name': 'JIRA - All Data',
    'owner': {'id': 1686425654, 'name': 'Travis Simpson'},
    'pdpEnabled': False,
    'policies': [{'filters': [],
    'groups': [],
    'id': 51,
    'name': 'All Rows',
    'type': 'open',
    'users': []}],
    'rows': 39021,
    'schema': {'columns': [{'name': 'issue_id', 'type': 'LONG'},
    {'name': 'issue_key', 'type': 'STRING'},
    {'name': 'Status Category Changed', 'type': 'STRING'},
    {'name': 'Resolution', 'type': 'STRING'},
    {'name': 'Release - Distribution List Notification',
    'type': 'STRING'},
    {'name': 'Parent Link_hasEpicLinkFieldDependency',
    'type': 'STRING'},
    {'name': 'Parent Link_showField', 'type': 'STRING'},
    {'name': 'Parent Link_nonEditableReason_reason',
    'type': 'STRING'},
    {'name': 'Parent Link_nonEditableReason_message',
    'type': 'STRING'},

    'updatedAt': '2019-04-30T07:23:12Z'}

     


    **Make sure to like any users posts that helped you and accept the ones who solved your issue.**
  • cwolman
    cwolman Contributor

    You will want to look at the Domo Governance Datasets.  This is available in the app store via a connector.  Do a search for Domo Governance Datasets and you should find it.

     

    There is a dataset available for Dataset Schema which will give you a nice format to create Domo reports or to export.

     

    Chris


    -----------------
    Chris

Categories