Data Connector Fields Refresh?

Jsun82
Jsun82 Member
in Connectors

I have a question and I'm not sure if this has been asked (couldn't find it) and I'm not sure if anyone else has this issue, but:

 

If I set up a data connection with columns A, B, and C and then I add column D, all columns show as expected.

 

However, if I have columns A, B, and C and then delete C from the import, the data comes over empty for C, but the column header is still there.  This is the most annoying when I've just renamed the column or if the deleted column name looks very similar to an active column.

 

Is there a way to refresh this without creating an entirely new connection?  As an FYI, I have seen this happen with the NetSuite connector and I haven't confirmed how this affects other types of data imports.

 

Thanks!

Best Answer

  • NewsomSolutions
    NewsomSolutions Domo Employee
    Answer ✓

    If you have your Netsuite dataset, set to APPEND, and you have C in a previous run and then in the next run someone deletes C, the next run will show C as blank like you describe. 

     

    If you have the connector set to Replace, then you wouldn't see C at all because the schema will change to the new format.

     

    What I've always recommended to my users is that you don't point cards directly back to connector datasets..you build an ETL and point them to the output of that.  So for this example, maybe create a recursive ETL and use 'SELECT COLUMNS' in your ETL so you can add/remove columns as you see fit.  

     

    Attached is a guide for recursive ETL.

     

    I hope that helps.

    Matt

     

Answers

  • NewsomSolutions
    NewsomSolutions Domo Employee
    Answer ✓

    If you have your Netsuite dataset, set to APPEND, and you have C in a previous run and then in the next run someone deletes C, the next run will show C as blank like you describe. 

     

    If you have the connector set to Replace, then you wouldn't see C at all because the schema will change to the new format.

     

    What I've always recommended to my users is that you don't point cards directly back to connector datasets..you build an ETL and point them to the output of that.  So for this example, maybe create a recursive ETL and use 'SELECT COLUMNS' in your ETL so you can add/remove columns as you see fit.  

     

    Attached is a guide for recursive ETL.

     

    I hope that helps.

    Matt

     

  • Jsun82
    Jsun82 Member

    Hi, thanks for the reply.

     

    The datasets in question are using the replace method so I don't think the recursive ETL is necessary.  I already use that for certain reports that I want to append.

     

    I'm the only one building anything in Domo at all so it's more of an annoying cleanup issue for myself more than anything.

  • Jsun82
    Jsun82 Member

    Ok, that was weird.  I didn't make any changes to the data, but just reran it and even though the error was showing before (I had A, B, and C and then changed C to D and it showed all 4.  Now it only shows the corrected A, B, and D).

     

    I also can't find other examples where this issue existed so I'm going to mark your answer as the solution. Thanks!

Categories