Return Capitalized (UPPER or CamelCase) Characters in Column Names from Redshift DataFlow

adam_mx
adam_mx Member
in Magic ETL

Any tips or tricks to avoid the auto-lowercasing of column names in Redshift DataFlows?

 

The reason we need this is because of how page filters are case sensitive, and the page being filtered runs off of several datasets, some are the output from MySQL dataflows and one from Redshift (due to the size, ~4.5M rows). Would prefer to conform the naming style of the Redshift output to that of the other MySQL ones, since it is just a single dataset, and capitalized letters are more visually appealing anyway. 

Tagged:

Best Answer

  • AS
    AS Coach
    Answer ✓

    Options are limited unfortunately.  You could run the output of the Redshift through a MySQL dataflow just to rename columns.  You could try renaming all of your MySQL columns so they're all lowercase.  That might be considered a best practice if you're mixing SQL engines. You could also try running through MySQL originally.  4.5m rows isn't awful; you'd just have to weight the benefit of a potentially slower dataflow versus not being able to fully use page filters.

    Coming down the line also should be a feature that allows you to change column names from within the Domo data center.  That's probably your best option, but it's probably at least a couple months away.

    Aaron
    MajorDomo @ Merit Medical

    **Say "Thanks" by clicking the heart in the post that helped you.
    **Please mark the post that solves your problem by clicking on "Accept as Solution"

Answers

  • AS
    AS Coach
    Answer ✓

    Options are limited unfortunately.  You could run the output of the Redshift through a MySQL dataflow just to rename columns.  You could try renaming all of your MySQL columns so they're all lowercase.  That might be considered a best practice if you're mixing SQL engines. You could also try running through MySQL originally.  4.5m rows isn't awful; you'd just have to weight the benefit of a potentially slower dataflow versus not being able to fully use page filters.

    Coming down the line also should be a feature that allows you to change column names from within the Domo data center.  That's probably your best option, but it's probably at least a couple months away.

    Aaron
    MajorDomo @ Merit Medical

    **Say "Thanks" by clicking the heart in the post that helped you.
    **Please mark the post that solves your problem by clicking on "Accept as Solution"
  • adam_mx
    adam_mx Member

    Thanks @AS for the quick reply. 

     

    What I am currently doing as a work around is actually pumping the Redshift output through a Magic ETL and doing the renaming there, similar to what you suggest with MySQL. It is messy though having that extra step. Looking forward to the data center updates! 

  • TheLookout
    TheLookout Contributor
    @AS wrote:

    ...

    Coming down the line also should be a feature that allows you to change column names from within the Domo data center.  That's probably your best option, but it's probably at least a couple months away.

    Is there an updated timeline for this features implementation? 

  • AS
    AS Coach

    The feature I saw was probably early beta but I haven't heard of any advancements since last summer.

    Aaron
    MajorDomo @ Merit Medical

    **Say "Thanks" by clicking the heart in the post that helped you.
    **Please mark the post that solves your problem by clicking on "Accept as Solution"
  • TheLookout
    TheLookout Contributor

    Thank you for the reply!

    If you hear anything about this feature, please let me know.

  • user19223
    user19223 Member

    Any update on this?

  • ijeffries
    ijeffries Member

    Hey @adam_mx , @user19223,

     

    I did something similar when uploading to our Snowflake instance. You can use the Python tile to automatically convert all your columns names to uppercase. 

     

    Python Tile Code:

    # Import the domomagic package into the script 
    from domomagic import *

    # read data from inputs into a data frame
    data = read_dataframe('Select Columns')

    # changes columns to uppercase
    data.columns = map(str.upper, data.columns)

    # write a data frame so it's available to the next action
    write_dataframe(data)

     

    Hope this helps!

    -Ian

  • jaeW_at_Onyx
    jaeW_at_Onyx Coach

    The Domo DataViews just went into beta, this would be the best/easiest way to create a VIEW of the data that renames columns without having to run it into an ETL.  talk to your CSM if you're not already part of the beta program.

    Jae Wilson
    Check out my 🎥 Domo Training YouTube Channel 👨‍💻

    **Say "Thanks" by clicking the ❤️ in the post that helped you.
    **Please mark the post that solves your problem by clicking on "Accept as Solution"

Categories