How to change schema of a Dataset created through the Streams API?

Hi,

We have data that we want to visualize in domo, and we are planning on loading it into domo using the Streams API, and then building cards based off that data.

However, we often need to add columns to our data (eg we start collecting a new property on an event). I can't see any way to change the schema of a data set in the Streams API. By contrast, with datasets created in the GUI, it accepts schema changes without any problem.

Can anyone help? Am I overlooking something?

Thanks!

Find more posts tagged with

Comments

KaLin

Could someone help with this issue?

robsmith

I've used the Update DataSet MetaData method. But if I remember correctly, I had to delete all my rows first (things may have changed now since I haven't done this in a while). I was unhappy that I had to delete all my data rows, but at least I was able to continue streaming to the same dataset without creating a new one.

matthallett

We are having the same issue.

If the schema changes on a api cloud app, everything else downstream seems to break.

How do we modify the data source to adjust the schema? Or do we need to recreate the data source and then migrate all of the cards?

I have scoured the training material and haven’t found the solution yet...

user14952

So I'm still trying to find an answer to this issues. The closest thing I could find is this: Python Example

# Update a DataSets's metadata
    update = DataSetRequest()
    update.name = 'Leonhard Euler Party - Update'
    update.description = 'Mathematician Guest List - Update'
    update.schema = Schema([Column(ColumnType.STRING, 'Friend'),
                            Column(ColumnType.STRING, 'Attending')])
    updated_dataset = datasets.update(dataset['id'], update)
    domo.logger.info("Updated DataSet {}: {}".format(updated_dataset['id'],
                                                     updated_dataset['name']))

It seems that you can add an update.schema value to this and it will be picked up by the API and add those fields to your Schema. I'm currently working with support to verify this and determine how to properly construct the fields, but this gives the best hints I've found so far.

user01098

Any update on this and the best practices to follow?