ETL using Append object not showing all records

Jessica

Has anyone noticed that all the other object/functions in ETL in the preview tab show the full number of records except Append Rows.

 

In a current ETL I am building and troubleshooting, I was trying to find out where data is dropping off in error and I got to the Append Rows object I was using and it is by default showing only 100 records.  I tested to make sure this is not where my error is occurring by creating a Test Output object directly after the Append object and the test dataset accurately includes the 2400 records I expected.

 

It makes it difficult to test when the preview function is not working accurately.

MarkSnodgrass

I have found that I can't trust the Preview window, regardless of how many rows I tell it to be in the preview. The row amount you choose (10k, 100k, etc.) only affects how many rows it will pull from your input datasets. It doesn't affect how many rows will show in the tile preview windows. Unfortunately, this means that you will need to run the actual ETL and look at your output dataset to know for sure if your tiles are configured correctly. 

jaeW_at_Onyx

.. it's not that Preview isn't showing your preview accurately, it shows a max of 100 rows by design.  That's just ... how it works.

 

What you can do is FILTER some of your data before you get to the APPEND tile.  If you know you're interested in a specific date, customer, or amount, apply the filter and then look.

 

Also, an APPEND will pull all the data from Dataset A and then UNION Input B.  If you're worried about a specifici input .... why not preview the data further upstream?