Joining two data sets on a monthly aggregate

KVincent
KVincent Member
in Magic ETL

I am looking to combine two datasets one is a sales volume the other is an instance where we receive a claim. These are done on the day level in my data sets and I want to be able to compare them by monthly aggregates so that I can compare sales volume vs claims by month. When I try to join these two sets in an ETL it creates duplicates or omits data. So is there a way perhaps in SUMO to have the months summed up by a certain location or customer then join them?

Best Answer

  • jaredpk
    jaredpk Domo Employee
    Answer ✓

    In Excel there is a function, "EOMONTH", that does the same thing.  So you could modify your excel file with a new column that includes that value.

     

    I know you had mentioned magicETL in the beginning.  Unfortunately there is not an equivalent function in magicETL, but if you did a magicETL / excel combination you would add the new column in your excel sheet.  Then in MagicETL, group by the end of month and sum your totals before the joins.

     

    Of course, being Domo we like to do as much as possible automatically in the product.  That would involve the dataflow option discussed, but I realize dataflows do require basic SQL.  In this case it would be pretty simple to write.

     

     

    I work for Domo.
    **Say "Thanks" by clicking the thumbs up in the post that helped you.
    **Please mark the post that solves your problem as "Accepted Solution"

Answers

  • jaredpk
    jaredpk Domo Employee

    Hi -- The best way to accomplish this is to use the last_day function in a mysql data.  See below for how this would work based on the following data:

    Date    Amount

    Jan 1   10

    Jan 2     5

    Jan 3   10

     

    If you write the following statement it will group all the data to the last day of the month:

    select

    last_day(Date) as 'Month End Date'

    ,sum(Amount) as 'Month Total'

    from table

    group by 

    last_day(Date)

     

    This would yield the following result

    Month End Date   Month Total

    Jan 31                        25

     

    Now sometimes there are situations where you are running on a fiscal calendar.  It becomes a little more complicated, but not much.  The concept is similar.

    I work for Domo.
    **Say "Thanks" by clicking the thumbs up in the post that helped you.
    **Please mark the post that solves your problem as "Accepted Solution"
  • KVincent
    KVincent Member

    Thanks for your reply. These two data sets come from excel is there a way to use this function from datasets coming from excel?

     

    Thank you!

  • jaredpk
    jaredpk Domo Employee
    Answer ✓

    In Excel there is a function, "EOMONTH", that does the same thing.  So you could modify your excel file with a new column that includes that value.

     

    I know you had mentioned magicETL in the beginning.  Unfortunately there is not an equivalent function in magicETL, but if you did a magicETL / excel combination you would add the new column in your excel sheet.  Then in MagicETL, group by the end of month and sum your totals before the joins.

     

    Of course, being Domo we like to do as much as possible automatically in the product.  That would involve the dataflow option discussed, but I realize dataflows do require basic SQL.  In this case it would be pretty simple to write.

     

     

    I work for Domo.
    **Say "Thanks" by clicking the thumbs up in the post that helped you.
    **Please mark the post that solves your problem as "Accepted Solution"
  • KVincent
    KVincent Member

    I am still new to domo so I did not know I could do a MySql with my uploaded data set. I got it figured out with mysql thank you so much for your help!

Categories