I'm researching what's involved in migrating many cards to a series of new DataSets and have been doing a bit of reading and experimenting. I already posted some information to an old thread on this, but it seems like a big enough subject for its own topic.
For background, my current task is to shift three DataSets to a new import method. Right now, we push out an Excel file, import it with Workbench and all is well. We went with this route as it's easy to set up and experiment with. Going forward, we've got too many different machines contributing data (and too much data) to continue this way. Instead, we're centralizing all of the data in Postgres and pulling it from there. All of that's fine.
When we started out, we didn't appreciate that when you great a DataSet you are locked into that delivery method forever. Our new and old DataSets are identical, it's only the connection method (Postgres pull instead of Workbench push) that changes.
Question: Is it really true that there is no way to change how the data is supplied? That's all we need to change. If there is a way to switch the method behind the scenes, that would solve all of our problems and the rest of my post is meaningless. But I don't think so. But I can hope!
Okay, so these three DataSets have a lot of cards, 73, 29, and 34. I don't want to migrate them and blow things up. I've tried this out in the past and once a card is broken, it can be impossible to fix. I don't know of any way to safely clone or back up Cards, so I'm approaching migrating our 136 cards....nervously.
Okay, now a bit on findings and question to match. Any and all comments, suggestions, best practices, tricks, magic incantations, etc. are most welcome. And, for a bit more background, I've already written a tool that uses the Domo APIs to pull all accessible information down about our DataSets, etc. I use this to automatically detect name and type problems, and it should help comparing user access and PDP rule settings too. Unfortunately, there is no way to find the cards associated with a DataSet. (Please vote for this feature Card API? List cards that use a DataSet?)
Findings and questions broken down by area below. Did I miss any areas? I didn't even think of Drill Paths or filters when I first sat down with this.
------------------------------
Column Names
------------------------------
These have to match exactly and case-sensitively. I don't get the case-sensitive part, but case-sensitive mismatches bring up the warning message.
I think that there's no problem with a DataSet having extra columns, but it can't be missing anything from the original.
------------------------------
Column Data Types
------------------------------
These have to match exactly. I've got this covered, so far as I can tell.
Good: Domo checks for column name and type name conflicts during migration, but it does not give you any details about what exactly is wrong. (Please vote for this feature Please add details to the warning dialog when moving a Card to a new DataSet)
------------------------------
Beast Modes
------------------------------
Beast Modes seem to be recreated atuomatically when you migrate. Nice! There's something in the docs about this behavior, but I don't know what the nuances are.
Any warnings or words of wisdom?
Idea: Speaking of Beast Modes, they're hard to detect. They're invisible to the API (Please vote for this here API: Please expose Beast Modes), and there's no UI in Domo to see them in one place. Unless I've missed something, there's also no way to distinguish cards or DataSets with calculated columns.
Idea: I'd like to be able to migrate calculated fields to normal fields when changing DataSets. Please vote for this feature here: Please allow Beast Modes to be mapped to full columns when changing a card's DataSet.
------------------------------
Drill Paths
------------------------------
It sounds like filters and Drill Paths are not migrated. There's a long-standing thread on this subject. It's marked as "solved", but it doesn't sound like there's a working solution.
The Domo APIs include no trace of Drill Path details. The child chart don't appear, there's nothing about them. Same wth Beast Modes, they're invisible in the API. You seem to have to go in and review every single card by hand. It would be great to have simpler UI tools for finding, managing, and editing calculated columns. For the minute, I'd be happy to have an API to call. Please vote for this idea here: Please add Drill Path details to the APIs.
------------------------------
Filters
------------------------------
I read that they break. I'm not sure what that means. I'm building up a checklist of steps to take to pre-flight each card and to then verify the card after the move. What would be suggest I put down for filters?
------------------------------
User Access
------------------------------
Haven't looked closely. I think we'll end up having to do it all by hand.
------------------------------
PDP Rules
------------------------------
Haven't looked closely. I think we'll end up having to do it all by hand.
Thanks!
