Is there a way to create a chart that displays the percentage of null values across all datasets?

luisamedinam

I want to know the percentage of null values across all datasets that we have, I am not interested in one particular dataset, I want to know the overall percentage.

GrantSmith

There isn't a simple way to get the total percentage of null values within your entire data warehouse. You could utilize a scripting language to query a list of your datasets then using each dataset ID you could query each dataset with the query endpoint and have it count the number of values in each. This would take time to query each dataset depending on the size of each dataset and the number of datasets you have.

pauljames

Hm. Perhaps you could create a template magic etl with a new column “Null Counter” you could add a column that uses the ifnull(column name, 1)? Then on that column name you could sum all rows that are 1, etc. You’d have to run every dataset you got through this etl example but perhaps it could work. Not easy necessarily, just an idea. :/

MarkSnodgrass

I wonder if you could reach out to support to get them to give you at least a one time report on this. You can see the null percentage on a dataset when you go to the stats view on the data tab, so the information is there.

It could also be worth submitting an idea in the Ideas Exchange as a Domo Stats/Governance dataset to include this information.

GrantSmith

There isn't a simple way to get the total percentage of null values within your entire data warehouse. You could utilize a scripting language to query a list of your datasets then using each dataset ID you could query each dataset with the query endpoint and have it count the number of values in each. This would take time to query each dataset depending on the size of each dataset and the number of datasets you have.

pauljames

Hm. Perhaps you could create a template magic etl with a new column “Null Counter” you could add a column that uses the ifnull(column name, 1)? Then on that column name you could sum all rows that are 1, etc. You’d have to run every dataset you got through this etl example but perhaps it could work. Not easy necessarily, just an idea. :/

MarkSnodgrass

I wonder if you could reach out to support to get them to give you at least a one time report on this. You can see the null percentage on a dataset when you go to the stats view on the data tab, so the information is there.

It could also be worth submitting an idea in the Ideas Exchange as a Domo Stats/Governance dataset to include this information.