Deleting records from a dataset using MySQL

amycaroline
amycaroline Member
in Connectors

Hi dojo -

 

I have a Box connector dataset that has duplicates from a run happening multiple times. I've written a simple SQL statement to delete the records, but am not seeing the results.

Since I'm not generating an output dataset, I can't save the dataflow, yet when Run/Apply, the rows I'm trying to delete are still there.

 

Any thoughts?  

Thank you!

amy

image.png

image.pngimage.png

Best Answer

  • PodiumMason
    PodiumMason Contributor
    Answer ✓

    Hey Amy!

     

    The way dataflow works is that it creates a temporary environment and loads your input tables and any transforms you create in the dataflow into temporary tables that only exist in that environment. 

     

    This means that any changes you make to the input datasets or other transforms only happen at the temporary table level. 

     

    In your case, in order to remove your duplicates, you'll want to run those SQL statements and then output the result. 

     

    As an example you could run your delete statements and then output a Select * from the table you deleted your records from.

     

    Let me know if you have additional questions!

    **Say 'Thanks' by clicking the thumbs up in the post that helped you.
    **Please mark the post that solves your problem as 'Accepted Solution'

Answers

  • PodiumMason
    PodiumMason Contributor
    Answer ✓

    Hey Amy!

     

    The way dataflow works is that it creates a temporary environment and loads your input tables and any transforms you create in the dataflow into temporary tables that only exist in that environment. 

     

    This means that any changes you make to the input datasets or other transforms only happen at the temporary table level. 

     

    In your case, in order to remove your duplicates, you'll want to run those SQL statements and then output the result. 

     

    As an example you could run your delete statements and then output a Select * from the table you deleted your records from.

     

    Let me know if you have additional questions!

    **Say 'Thanks' by clicking the thumbs up in the post that helped you.
    **Please mark the post that solves your problem as 'Accepted Solution'
  • amycaroline
    amycaroline Member

    aha!

    makes sense - thank you, xervamason!

    amy

  • hamza_123
    hamza_123 Contributor

    Hello, 

     

    I had a similar question. Can I delete records from my input dataset, and then use the same dataset (from which I have deleted records) as output. 

     

    In other words, can I change my input dataset through mysql commands (or even ETL)?

Categories