ETL fails due to "This data set does not support data version filtering."

nickcorona
nickcorona Member
in Magic ETL

I have my SQL raw dataset set to merge new rows which is working correctly:

image.png


This SQL dataset is the input for my ETL attempting to process New rows then Upsert in the output dataset:

image.png


However my ETL fails to run due to the stated error:

Screenshot 2025-02-13 151704.png


Am I setting something up wrong? My raw SQL data is not partitioned but it is using the same column to determine the Merge new rows and Upsert to output.

I am using the Microsoft SQL Server JDBC (preferred) connector

Tagged:

Answers

  • Bshukla
    Bshukla Domo Employee

    Hello,

    I’ve thoroughly checked this and unfortunately, there isn’t a workaround at the current time. The only solution available is to handle the partitioning on the ETL (Extract, Transform, Load) side, rather than using the Upsert functionality at the workbench level, specifically for the input dataset.

    To clarify, Upsert at the workbench level doesn't support partitioning, which limits its ability to handle large datasets efficiently in some cases. By shifting the partitioning process to the ETL stage, we can better manage data updates and ensure smoother handling of larger volumes of data.

    I hope this helped!

Categories