Return Capitalized (UPPER or CamelCase) Characters in Column Names from Redshift DataFlow

adam_mx

Any tips or tricks to avoid the auto-lowercasing of column names in Redshift DataFlows?

 

The reason we need this is because of how page filters are case sensitive, and the page being filtered runs off of several datasets, some are the output from MySQL dataflows and one from Redshift (due to the size, ~4.5M rows). Would prefer to conform the naming style of the Redshift output to that of the other MySQL ones, since it is just a single dataset, and capitalized letters are more visually appealing anyway. 

AS

Options are limited unfortunately.  You could run the output of the Redshift through a MySQL dataflow just to rename columns.  You could try renaming all of your MySQL columns so they're all lowercase.  That might be considered a best practice if you're mixing SQL engines. You could also try running through MySQL originally.  4.5m rows isn't awful; you'd just have to weight the benefit of a potentially slower dataflow versus not being able to fully use page filters.

Coming down the line also should be a feature that allows you to change column names from within the Domo data center.  That's probably your best option, but it's probably at least a couple months away.

AS

Options are limited unfortunately.  You could run the output of the Redshift through a MySQL dataflow just to rename columns.  You could try renaming all of your MySQL columns so they're all lowercase.  That might be considered a best practice if you're mixing SQL engines. You could also try running through MySQL originally.  4.5m rows isn't awful; you'd just have to weight the benefit of a potentially slower dataflow versus not being able to fully use page filters.

Coming down the line also should be a feature that allows you to change column names from within the Domo data center.  That's probably your best option, but it's probably at least a couple months away.

adam_mx

Thanks @AS for the quick reply. 

 

What I am currently doing as a work around is actually pumping the Redshift output through a Magic ETL and doing the renaming there, similar to what you suggest with MySQL. It is messy though having that extra step. Looking forward to the data center updates! 

TheLookout

---

@AS wrote:

...

Coming down the line also should be a feature that allows you to change column names from within the Domo data center.  That's probably your best option, but it's probably at least a couple months away.

---

Is there an updated timeline for this features implementation? 

AS

The feature I saw was probably early beta but I haven't heard of any advancements since last summer.

TheLookout

Thank you for the reply!

If you hear anything about this feature, please let me know.

user19223

Any update on this?

ijeffries

Hey @adam_mx , @user19223,

 

I did something similar when uploading to our Snowflake instance. You can use the Python tile to automatically convert all your columns names to uppercase. 

 

Python Tile Code:

# Import the domomagic package into the script 
from domomagic import *

# read data from inputs into a data frame
data = read_dataframe('Select Columns')

# changes columns to uppercase
data.columns = map(str.upper, data.columns)

# write a data frame so it's available to the next action
write_dataframe(data)

 

Hope this helps!

-Ian

jaeW_at_Onyx

The Domo DataViews just went into beta, this would be the best/easiest way to create a VIEW of the data that renames columns without having to run it into an ETL.  talk to your CSM if you're not already part of the beta program.