Handling "Replace vs. Append" Logic with Email Connector

DataLawton

Hello everyone,

I'm seeking advice on configuring the Domo Email Connector for importing reports in a "month-to-date" format via email attachments. I want to a) retain the reports for previous months, but also b) replace the latest "month-to-date" report.

Our goal is to achieve a combination of "Replace" and "Append" logic (if possible):

1. Replace Logic: Replace data for the current month during the month.
2. Append Logic: Append new data when a new month starts.

Example: During May 1st through 31st, do a replace with the latest "month-to-date' report but also append/retain the month-to-date reports from April 30th, March 31st, etc.

Has anyone implemented something similar or can offer guidance on how to handle this using the Email Connector and DataFlows?

Thanks in advance for your help!

Sean_Tully

I think you'll end up with a couple of different datasets here:

1. The email connector dataset, which appends the new data from each email
2. A dataset where you run the full email connector dataset through Magic ETL. Here, you would find the max batch date for each month, and then isolate those rows before outputting to the final dataset. This way you get the latest report for the current month, plus the "final" report for each previous month.

To find the max batch date for a given month, I would probably create a "Month" field using the formula tile, then group by that field to find the max batch date. An inner join on month = month and max batch date = batch date back to the data input should isolate the appropriate rows.

Sean_Tully

I think you'll end up with a couple of different datasets here:

1. The email connector dataset, which appends the new data from each email
2. A dataset where you run the full email connector dataset through Magic ETL. Here, you would find the max batch date for each month, and then isolate those rows before outputting to the final dataset. This way you get the latest report for the current month, plus the "final" report for each previous month.

To find the max batch date for a given month, I would probably create a "Month" field using the formula tile, then group by that field to find the max batch date. An inner join on month = month and max batch date = batch date back to the data input should isolate the appropriate rows.