alert if no datasets run on current day

user09600
user09600 Member
in Magic ETL

Hi, i have 4 datasets which are supposed to be updated daily from API.

we have some daily monitoring through domo stats that alert us in case of the number of datasets which are update is less than 4.

But we notice if no datasets have been updated on the day then alert don't trigger as it has no data to track.

I would like to know how we can handle such a case?

how we can detect if no data to track?

 

Comments

  • jaeW_at_Onyx
    jaeW_at_Onyx Coach

    The Datasets dataset last run date.  So create a beast mode

     

    case when <last update within 24 hours> then 'we are fine'

    else 'error'

    end

     

    replace the pseudocode.

     

    To only track 'the important' datasets, build a webform 'LU_ImportantDatasets' with the dataset ID.

    Then do an INNER JOIN to the Datasets dataset.

     

    Now you can alert on a beast mode as well as visualize the in error datasets.  You could even create stages. 'kinda bad' 'terrible' 'call the head of IT'.

     

    Only caveat is the stats dataset has to be up to date...

     

    Jae Wilson
    Check out my 🎥 Domo Training YouTube Channel 👨‍💻

    **Say "Thanks" by clicking the ❤️ in the post that helped you.
    **Please mark the post that solves your problem by clicking on "Accept as Solution"
  • user09600
    user09600 Member

    Hi Wilson, thank for your reply.

    Not sure to understand "The Datasets dataset last run date"

    Can you specify with some example?

  • GrantSmith
    GrantSmith Coach

    @user09600 

    Are you saying that your datasets run and you're able to detect that but if it happens to run and doesn't pull in any data then you don't have an alert for this case?

     

    What I've done in the past for those cases is to check the latest date that my dataset had within it's data. If it wasn't yesterday (I run this early in the morning so I'm not checking today as there is no data for today) then I send out an alert on a card based on that date.

     

    CASE WHEN MAX(DATE(`timestamp`)) < CURRENT_DATE - INTERVAL '1' DAY THEN 'Stale' ELSE 'Up to date' END

    timestamp is a field in my dataset which I'm pulling in data on a daily basis. So then I set this as a summary number on a card and created an alert to send if it's not up to date.

    **Was this post helpful? Click Agree or Like below**
    **Did this solve your problem? Accept it as a solution!**
  • jaeW_at_Onyx
    jaeW_at_Onyx Coach

    This is a Domo Stats dataset called "Dataset" it has attributes of ... datasets.  You can only schedule it to run once per day.

     

    capture.PNG

    Jae Wilson
    Check out my 🎥 Domo Training YouTube Channel 👨‍💻

    **Say "Thanks" by clicking the ❤️ in the post that helped you.
    **Please mark the post that solves your problem by clicking on "Accept as Solution"
  • user09600
    user09600 Member

    I grantsmith.

    If i understand what you meant you have summary number as 'stale' or 'up to date'?

    If value is stale you get an alert?

    can you show me how you set up the alert on top of this?

    what i have created is a beast mode variable :

    CASE WHEN MAX(DATE(`Start Time`)) < CURRENT_DATE then '0' else'1'end

    and use this value as summary number.

    then trigger alert if value =0

    the problem is as in file attached it said the summary number is currently empty

  • GrantSmith
    GrantSmith Coach

    Hi @user09600 

     

    I used a beast mode like this:

    CASE WHEN MAX(`Date`) = CURRENT_DATE() THEN 1 ELSE 0 END

    Then I used that beast mode as the summary number on a card. You then click the alert (bell) icon and set it to alert you if a certain criteria is met (in this case the summary number = 0). My beast mode was named "Up To Date" in this example.

    Screen Shot 2020-06-01 at 7.32.39 AM.png

    **Was this post helpful? Click Agree or Like below**
    **Did this solve your problem? Accept it as a solution!**
  • GrantSmith
    GrantSmith Coach

    @user09600 Mine was a bit different since I was only looking at a single record however you might try and remove the MAX function.

    **Was this post helpful? Click Agree or Like below**
    **Did this solve your problem? Accept it as a solution!**
  • user09600
    user09600 Member

    Hi like i said my summary number show it is empty even i tried to use the value created in beast mode

    I use current value as summary number.can you show me how you use it as summary number? maybe i missed some settings.

  • user09600
    user09600 Member

    i found the reason it needs to be an aggregation otherwhile it cannot be use as summary number

Categories