Best practices for joining multiple external/3rd party datasources in Domo

John-Peddle

First off, not sure I'm posting in the right forum so please let me know if this should be moved elsewhere! I'm guessing many other organizations out have dealt with something similar and hope you may have some feedback on how you resolved this. We tend to use a lot of 3rd party applications (Talkdesk & Intercom for communication, Salesforce for tasks, Humanity & BambooHR for HR/payroll related activities, etc).

 

The problem we face is the manual entry needed to enter employee related data into multiple systems during the on-boarding process; often times there are spelling errors between systems (yes, they SHOULD be copying/pasting from a master file, but that can also sometimes be incorrect) as well as name changes (marital or otherwise), job/title changes, etc.

 

What we end up with is slightly different user related data in several systems, which causes issues in Domo when we're trying to join various datasets in dataflows to create output datasets/cards. We've used the Domo Webform to create an employee roster, but it's functionality is lacking, is difficult to account for job/name changes so they stay with the position during the dates worked, and it's just a very manual process.

 

Has anyone else faced problems similar to this when bringing data into Domo from several 3rd party applications and often times running into issues joining it due to inconsistencies, and if so how were you able to rectify the issue and clean that data up?

 

We're looking to use one of our HR/payroll related applications as a single source of truth but that still doesn't account for name changes, incorrect name spellings in other systems, etc. Obviously we have many data tables with replicated data, so we find out selves having to identify which ones need to be changed when updating historical data, and it's become an absolute nightmare for us to keep up with this.

 

Is there documentation anywhere here, or on an external site, that provides best practices for these nuances or personal experiences from others on they've handled these issues? Any assistance would be greatly appreciated, and let me know if there's anything additional we can provide regarding this. Thanks!

Jarvis

Hello John,

 

There is always the option to put into an ETL or similar transform, a function that catches known issues where there are rogue data entries and changes it to be their correct values. However this is a bandaid, and best practice is to work with the administrators of the systems generating this bad data. This will help you improve not only the quality of data into Domo but also any other systems using your company data.

 

Jarvis

bdavis

We try to attach an employee ID value into every possible system we have. This way when we bring in the data into Domo, we have a key to tie all employee records together. Doing this also allows us to create "error cards" where we can identify mismatching information for an employee that may exist between our different systems. Sometimes it was nice an easy, using an existing employee ID field, and other times we had to create a custom field to hold the data, or even shoe-horn it into a field that won't be used because we had no other options.