Identifying what data came from which connector on a card.

Parkerains

Hello all, and thank you for any help. I have my magic ETL flow built out so a report is pulling information from two different connectors. There does not seem to be a batch ID or file path as options for me to select, which I thought would be the method to identify which system they pull from. Does anyone know how to filter on a card by connector? Thanks any help would be appreciated.

Jonathan53891

Hello @Parkerains, this will depend on how you have the Magic ETL dataflow set up. One solution to this would be to use the "Add Constants" ETL tile in your dataflow if you are appending your data sources together. Once you have finished creating the dataflow logic for each of your respective data sources, you can place a constant tile before the sources get appended together.

For this, you would need to use the same column name for each ETL tile (I use "Source" as the column name for this example). Then, you can input the unique name of the data sources they came from (i.e., Source = Facebook, Source = Instagram Business, etc). Since they are all appended to the same column, you can use this column in a filter on your dashboard to toggle between the various data sources in addition to having a clear distinction for where the data came from.

Please also note that the batch IDs for the cloud-based connectors provide a reference for how many times the dataset has updated. This can be utilized in a variety of ways, but the _BATCH_ID_ column does not provide any descriptive information about the connector itself.

DEV - Social Media Consolidated (EXAMPLE).png

DEV - Social Media Consolidated Add Constants Tile (EXAMPLE).png