Where is my data going??

ljb18
ljb18 Member
edited July 24 in Magic ETL

My data seems to be disappearing between the ETL and the final data that it actually outputs. Here are some pictures to try to explain what's happening. The first image shows the preview on the final tile in the workflow (being the output file titled Work Numbers (Final)). The second image shows the same chunk of data, using that same output and examining the data with "Views Explorer."

Screen Shot 2024-07-24 at 1.28.26 PM.png
Screen Shot 2024-07-24 at 1.28.52 PM.png

Where did my data go? Absolutely NOTHING changed between these two steps. It's driving me crazy that it's seemingly there, but then disappears, thus preventing me from being able to use it at all.

Tagged:

Best Answer

  • ColemenWilson
    ColemenWilson Coach
    edited July 24 Answer ✓

    In MagicETL you are only previewing 100 rows. I see in the second image that there are 1,737 unique values for the BMI Work Number in your final output data. It appears that the data is there and that there are also null values for that field.

    If I solved your problem, please select "yes" above

Answers

  • ColemenWilson
    ColemenWilson Coach
    edited July 24 Answer ✓

    In MagicETL you are only previewing 100 rows. I see in the second image that there are 1,737 unique values for the BMI Work Number in your final output data. It appears that the data is there and that there are also null values for that field.

    If I solved your problem, please select "yes" above

  • ljb18
    ljb18 Member
    edited July 24

    I understand that there's more data that isn't shown in the preview, but that doesn't really answer the question because I'm looking at the exact same chunk of data in both screenshots. If I filter just for the track title names that I'm showing in the preview section, it still shows all that data erased. There is nowhere in my entire data set where "TRACK TITLE: With Version" with a value of "Grits 'n Gravy" has a "BMI Work Number" of "51911889" outside of the preview. So I don't understand where it's going.

  • ljb18
    ljb18 Member

    Actually I increased it to 400k and now it's showing it as null lol… I'm not really sure what's going on. I'm going to take a deeper look into the data. Sorry, looks like I don't understand how the preview works and you were probably right 😅

Categories