Optimizing a Domo Workflow for Sending Emails Based on Dataset Markers

lindseyw
lindseyw Member
in Workflows

I’m working on a project that involves sending emails to individuals based on specific markers in a dataset. Currently, I’ve set up a Domo workflow that iterates through the dataset row by row, checks certain conditions, and sends emails accordingly. While the workflow functions as intended, it is extremely slow.

To address this, I’ve tried cleaning up the dataset in an ETL to filter and prepare the data before the workflow starts. However, even with a pre-filtered dataset, the workflow still needs to iterate through each row to determine who to send the emails to and to personalize the content. This row-by-row processing seems to be the main bottleneck, and I’m looking for ways to optimize this process.

I’m curious to know how others in the community might approach this kind of project. Are there best practices for minimizing iteration time in workflows? Have you had success with bulk email sending or batching methods within Domo? Would integrating external tools or APIs to handle email sending, while still leveraging Domo datasets, be a better solution?

If you’ve tackled a similar problem or have any ideas, I’d greatly appreciate your insights. Thank you in advance for your help!

Tagged:

Best Answer

  • DanHendriksen
    DanHendriksen Domo Employee
    Answer ✓

    My first thought, is why not adjust the query so it only returns rows that you know you need to action? You can add a WHERE clause to the query.

    SELECT * FROM DATA WHERE ` Number_Column ` = 1

    Then you don't need to process through everyone and check to see what the value is - you already know the value. You can also remove the "Get Number from Object" block by defining the existing object, and evaluating it's child. You can edit the "current_row" (or whatever you have it called) object to say it includes "number_column" (or whatever the column is) and then evaluate current_row.number_column. Eliminating that step will make it not only faster, but also less costly.

Answers

  • DanHendriksen
    DanHendriksen Domo Employee

    You could do things concurrently…so have one workflow that identifies all the rows, this kicks off a second workflow that does the sending of the email.

    If the first workflow identifies 100 emails to send, it will start 100 iterations of the step 2 workflow that all run concurrently.

    That said, none of this should take very long. What type of processing time are you seeing, and how many rows are you processing/emails are you sending?

  • lindseyw
    lindseyw Member
    edited January 6

    Hey Dan,

    This is my first workflow, so I’m not entirely sure if I’m taking the right approach, but here’s how it’s currently set up:

    Right now, the workflow isn’t configured to send emails. Its primary function at this stage is to loop through the dataset, evaluate a specific column’s value, and increment counters. This allows me to verify that I’m obtaining the correct results before enabling the email functionality.

    Currently, the workflow processes about 430 rows of data in 12 minutes. However, the number of rows should drop significantly once the workflow is live and people start acting on the data.

    Workflow "Pseudocode":

    1. Start workflow.
    2. Initialize variables (index & 4 counters).
    3. Query dataset with SQL (dataset is an ETL output).
      • SELECT * FROM data
      • Returns: array of objects (each row is an object).
    4. Loop through rows in the array:
      • Get the current object/"row."
      • Check the numerical value in one of the "columns" using the "Get Number from Object" block.
      • Increment a counter based on the result.
      • (Future step: Replace counters with "send email" blocks.)
    5. Check for more rows:
      • If more rows exist (based on the index variable), repeat.
      • If none, end the workflow.

    I’d really appreciate any feedback on whether this approach makes sense or if there’s a better way to accomplish this. Let me know if you need more details!

  • DanHendriksen
    DanHendriksen Domo Employee

    So it sounds like what you want to do is send an email if a numerical value meets certain criteria? Is that correct?

    And no worries at all! If this is your first workflow you're doing great! There's a definite learning curve.

  • DanHendriksen
    DanHendriksen Domo Employee
    Answer ✓

    My first thought, is why not adjust the query so it only returns rows that you know you need to action? You can add a WHERE clause to the query.

    SELECT * FROM DATA WHERE ` Number_Column ` = 1

    Then you don't need to process through everyone and check to see what the value is - you already know the value. You can also remove the "Get Number from Object" block by defining the existing object, and evaluating it's child. You can edit the "current_row" (or whatever you have it called) object to say it includes "number_column" (or whatever the column is) and then evaluate current_row.number_column. Eliminating that step will make it not only faster, but also less costly.

  • lindseyw
    lindseyw Member

    Hey Dan!

    That is exactly what I started implementing after your response yesterday. I am utilizing the "SQL Query Table" function to grab each row, then iterating through each row in 3 separate/parallel loops, which is a lot faster!

    Thank you! :)

Categories