Add CSV filename as column

dacorson

I am running a few workbench jobs, and I need to able to add the file name from each CSV in the folder I'm loading. When you create a workbench job using an xlsx file, you have the option to check a box to add the file name. But for CSV there is no option I can see. Is there a way through calculated columns or any other method to add the file name? Seems like this should be possible.

GPirie

Thanks for the question.
In Workbench job settings, you can add a file name for a CSV file (single file name per job).

To accomplish what you want, you want to use a Workbench feature called “Data Sync”

To access Data Sync from Workbench:

You can add as many files as you want, and Workbench will watch the folder and upload when there is a file change (timestamp or other change), workbench will upload the changed file.

dacorson

I need to batch the files from a single folder, so this won't work for me. There should be a check box just like in the excel transport type. Thanks for your response, though!

GPirie

If all the files are identical (they must have the same column layout (schema), but the file names can be different), do not use a file name at the end of the Source Edit path. In my example, the path is C:\

Any file in that path that is a CSV file will be used to create a single Workbench job (TIP: Put the files into a separate directory and use that directory as the Source path to the files in the directory. Any change to any file will cause a batch run of all files in the Source directory.

WHM

That did not answer how to bring the file name in as a column value.

WHM

Data Sync is a terrible option! It creates a new dataset for every CSV in the folder! I just had to delete over a hundred datasets after testing to see what data sync does. It also does not bring the filename in as a column, I do not see a way to dynamically assemble all the data from all the CSVs into a single dataset that includes the file name. the files would constantly have to be added to a dataflow…

our situation … each csv is a report for a single day… 1-june-2023-report.csv etc… that June 1 date is not in the data. I need it to be a column in my dataset.