Federated dataset: can I prevent a non-filtered preemptive query?

JohnKalan_HITRUST
JohnKalan_HITRUST Member
in Datasets

I am using a federated dataset (SQL Server) with hundreds of millions of records, for thousands of users. It appears that Domo retrieves or queries the entire data set before any card is displayed, resulting in tremendous resource usage. When a user views the card, I am providing a filter that restricts the query to one user via WHERE clause, and the query is much faster and less resource intensive.

Is there a way to prevent the non-filtered call to the dataset? I have already tried setting TTL to zero (0).

Tagged:

Answers

  • ssharma
    ssharma Domo Employee

    @JohnKalan_HITRUST
    I believe the issue is:

    When using federated datasets, especially with large volumes of data (e.g., SQL Server), Domo may attempt to preview or materialize a full scan of the dataset to build cards or load schema metadata, even if your card includes user-specific filters.

    This leads to excessive resource consumption on your SQL Server, because:

    • The initial call may bypass user-level filtering until after data is returned.
    • The card preview or editor view can trigger full dataset scans.
    • Setting TTL (Time-To-Live) to 0 only disables caching — it doesn’t stop Domo from querying the entire dataset when a user opens a card.

    To get suggestions to prevent the non-filtered call to the dataset, you can reach out to domo support team so that they can investigate your use case and provide you with a suitable alternative, if that exists.

  • JohnKalan_HITRUST
    JohnKalan_HITRUST Member

    Thank you for the excellent summary. Domo support has indicated that is not possible to prevent the query and that we should add resources.
    We have reconsidered the need for the card and dataset and removed them.

Categories