Calculating Distance In Magic ETL

RyanIllum
RyanIllum Member
in Magic ETL

I have around 400 locations with different coordinates. I need to find if the distance between any the locations is above 1 mile. Should I use ETL or another tool?

Best Answers

Answers

  • GrantSmith
    GrantSmith Coach
    edited September 2023 Answer ✓

    https://www.domo.com/blog/improving-supply-chain-logistics-with-geographic-proximity-matching/

    @MichelleH has written up a great article on the community blog you can reference on how she did geospatial distance calculations using a Redshift dataflow.

    **Was this post helpful? Click Agree or Like below**
    **Did this solve your problem? Accept it as a solution!**
  • ColemenWilson
    ColemenWilson Coach

    Agreed with the above. If you want to do it in Magic ETL and just need to see any distances greater than 1 mile you could do this:

    1. Join all 400 rows to each of the 400 rows (note this would be 160,000 rows of data)
    2. Compare each location to each other location using the DISTANCE() function
      1. You'll need the coordinates of each location to do this.
      2. You'll need to convert from KM to Miles by mulitplying the result by 0.621371
    3. Use a filter tile or filter in a card to only show those where distance > 1 mile

    If I solved your problem, please select "yes" above

  • MarkSnodgrass
    MarkSnodgrass Coach
    Answer ✓

    Here is a video walkthrough of how to do it:

    https://youtu.be/xzi-fhEmN-Y

    **Check out my Domo Tips & Tricks Videos

    **Make sure to <3 any users posts that helped you.
    **Please mark as accepted the ones who solved your issue.
  • RyanIllum
    RyanIllum Member

    @ColemenWilson how do I compare each location for every location without having to write a formula for every location

  • MarkSnodgrass
    MarkSnodgrass Coach

    You will want to change your join since you are comparing it to the same data set. Here is how I typically do it to get a row for every combination.

    image.png
    image.png
    image.png

    Use the add constants tile to create a column called JoinKey and set it to a value of 1. Do that twice. Connect them to the join tile and join on the JoinKey with an inner join. Rename columns so that you have your first and second coordinates. You will then have a row for each combination and then you can use your distance function.

    **Check out my Domo Tips & Tricks Videos

    **Make sure to <3 any users posts that helped you.
    **Please mark as accepted the ones who solved your issue.
  • ColemenWilson
    ColemenWilson Coach
    Answer ✓

    @MarkSnodgrass laid it out exactly how I did it. Keep in mind that your data can grow rapidly. For example, right now you have 400 rows, 400 x 400 = 160,000. Bumping that up to 500 takes your total rows to 250,000. So be cautious when using this type of self join.

    If I solved your problem, please select "yes" above

  • RyanIllum
    RyanIllum Member

    Thank you guys I was able to figure it out.

Categories