Add CSV filename as column

dacorson
dacorson Member
in Workbench

I am running a few workbench jobs, and I need to able to add the file name from each CSV in the folder I'm loading. When you create a workbench job using an xlsx file, you have the option to check a box to add the file name. But for CSV there is no option I can see. Is there a way through calculated columns or any other method to add the file name? Seems like this should be possible.

Tagged:

Answers

  • GPirie
    GPirie Domo Employee
    edited March 2023

    Thanks for the question.
    In Workbench job settings, you can add a file name for a CSV file (single file name per job).

    To accomplish what you want, you want to use a Workbench feature called “Data Sync”

    To access Data Sync from Workbench:

    WB Data Sync.png

    You can add as many files as you want, and Workbench will watch the folder and upload when there is a file change (timestamp or other change), workbench will upload the changed file.

    If I have answered your question, please click "Yes" on my comment option.

  • dacorson
    dacorson Member

    I need to batch the files from a single folder, so this won't work for me. There should be a check box just like in the excel transport type. Thanks for your response, though!

  • GPirie
    GPirie Domo Employee

    If all the files are identical (they must have the same column layout (schema), but the file names can be different), do not use a file name at the end of the Source Edit path. In my example, the path is C:\

    Any file in that path that is a CSV file will be used to create a single Workbench job (TIP: Put the files into a separate directory and use that directory as the Source path to the files in the directory. Any change to any file will cause a batch run of all files in the Source directory.

    Screen Shot 2023-03-22 at 10.56.21 AM.png

    If I have answered your question, please click "Yes" on my comment option.

  • WHM
    WHM Contributor

    That did not answer how to bring the file name in as a column value.

  • WHM
    WHM Contributor

    Data Sync is a terrible option! It creates a new dataset for every CSV in the folder! I just had to delete over a hundred datasets after testing to see what data sync does. It also does not bring the filename in as a column, I do not see a way to dynamically assemble all the data from all the CSVs into a single dataset that includes the file name. the files would constantly have to be added to a dataflow…

    our situation … each csv is a report for a single day… 1-june-2023-report.csv etc… that June 1 date is not in the data. I need it to be a column in my dataset.

Categories