-
Find and append a value to a field
I'm trying to align some inconsistencies with our data. What I'd like to do is find any value that does not end with a '/' and append it with that character. Is there a case statement where we can look for a field that does or does not end in a character? I assume if I can do that, I can just append those without with the…
-
Appends Should Default to Include All Rows
Appends should default to "Include All Rows" and not change every time inputs are added/removed. Huge quality of life upgrade!
-
Has anyone gotten replacement variables to work with the ODBC connector on a consistent basis
I am trying to utilize replacement variables on ODBC connector queries to move towards incremental APPEND queries instead of REPLACE queries. However, it is taking multiple attempts to get the workbench to save the replacement variable. Without saving it, I keep getting "Replacement parameter does not exist for column XXX"…
-
Combining two datasets based on two potential fields
Hi, I have two datasets. One has pacing data, the other has monthly data. All rows have unique IDs, however the pacing data may have either an ID from the old system (legacy ID) or the new system (new ID). Pacing: ID 1 2 3 4 11 Monthly: New ID Legacy ID 10 1 11 12 13 2 14 3 15 How would I go about joining these two…
-
Issues Joining Transaction Details to Primary Dataset
I have two datasets. One is structured as the "primary" expense file, where each row is dates and other dimensions for each credit/debit. The next is credit card transaction details. They share transaction IDs in a column named Doc, share GL #'s, and share posted dates. The combination of all 3 of these things will result…
-
Selecting only newer records from a remote datasource to append to a Domo datasource
Hi here… When setting up a query to grab data from a remote datasource and append to data in DOMO. Is there a way to grab newer records based on the data that was previously imported. Page: DATA / <DATASET NAME> / SETTINGS / QUERY I'd like to run something like: SELECT * FROM myRemoteDataSchema.myDataSet ← remote dataset…
-
How to append data so you can get a snapshot every time your dataset runs
Hi I have created a date field in my dataflow using this formula to give me the current date: DATE(TODAY()) I am trying to use this date field to get a snapshot of the data every time the dataset refreshes. I have my dataset scheduled to refresh every Friday. The first Friday the dataset refreshed it gave me the correct…
-
How do I set up the append option for updating data correctly?
Use case: I have callrail data that begins in 1/1/2021. Currently the dataset is updating everyday and has REPLACE as an update method. The dataset is now taking at least 1-3 hours to update. I thought about changing the update method to append but when I did, I had duplicate data from 2021. How can I change the update…
-
best way to replace a subset of data
I have a large historical dataset to which I append records at the end of each day. I have been asked to correct the distribution of values for a small subset of the data over the past nine months. I can use the data repair tool feature to remove all of the data imports for that time, but this would force me to destroy and…
-
Use Empty csv in ETL
Greetings! I am sending csv reports to box and using the Advanced box connector to retrieve the files and add them to a magic ETL. The reports from the current and previous years work great. I would like to set up this process to run through 2026 so I am sending empty reports for 2024-2026. They currently populate box with…
-
PyDomo - DS_Update to Append Data?
Hello all, I've written a script that ideally will create a datastream into Domo. At the moment though, I'm struggling to find a way to replace the update method of ds_update() from PyDomo. The default appears to be replacing the data, but I need to append the dataset. Does anyone have any experience with this? Any help…
-
How to append Doc cards with XLS data?
I have several Doc cards with XLS data inside them. I'm trying to find a way to combine these Doc cards' data by just appending them. Is there a way to do this in Domo, or perhaps even a third-party system?
-
How to periodically REPLACE in addition to APPEND jobs?
I have a VERY large dataset that I've uploaded to Domo. I initially did a REPLACE to load the whole dataset. I then changed the workbench job to APPEND and use an 'update date' in the table to run HOURLY while checking back the past couple days for any new/missed data. Due to some bugs on our side, sometimes very old data…
-
Manually adding data to a existing DOMO dataset
Hi Team, I am trying to add the data to a existing DOMO dataset..Can you please let me know how I can achieve the same.. Thanks, Arun
-
Workbench Replacement Variable not saving
I've created a handful of Workbench jobs over the past few months, but every time I do, I seem to have to struggle for about an hour each time to get the replacement variables working with the 'Append' update method. This typically includes trying every imaginable order of operations, deleting and recreating the job…
-
Adding Budget Rows to Actual
Hello, I have an original cases database in DOMO each row has a case submission and case start, and then I later replicate it all the columns and added the budget we have for the year. I had to unpivot this data in order to create graphs that will put both dates in the same axis. I would like to show the budget as a third…
-
Assembler Upsert Date Column
Hi. I need to take a snapshot of today's data in order to compare it with tomorrow's data once it will be available. The problem is that the SQL dataflow and Magic ETL only can use Replace method. I need to find a way how to append the data each day to the already existing historical table. I created Assembler and I chose…
-
Append a dataset without using recursive design
Is there any other way to append to a dataset without using a recursive design in Magic ETL?
-
Joining or Appending Data Help
I have a dataset (dataset 1) with individual rows as a specific job (with an identifier key) with details including year and country of operations. I have created cards to show the total number of jobs per year. I have another dataset (dataset 2) which shows historical revenue by Year and Country but it is not specific to…
-
Append to dataset based on month and year partition value
Hi everyone, I have a table in my database that is updated everymonth and the table has a partition column like 201911 for Nov 2019, 201912 for Dec 2019 and 201001 for Jan 2020. I want to append these values from the table to a Domo dataset every month without duplicates but not sure how to do that using the replacement…
-
Historical Database I inherited
Hi all. I'm dealing with CRM data, and have a snapshot of all of our data that is appended to a full historical daily database. This is done in a SQL transform, but I am completely unable to figure out why the input data is being appended to the output instead of clearing it. I am completely dumbfounded. I'm attaching…
-
ETL joining causes an output dataflow that is too big?
I am very new in using Domo, so hopefully I can explain my situation well. I have three datasets that have different column names and structures, even though each dataset has same columns too, like 'Employee Name' and 'Employee ID'. I tried to join them (using inner join) and run preview with 10k limit, but it showed the…
-
JSON with custom parsing pagination
I have a question regarding the JSON with custom parsing connector. I have a dataset using the pagination settings to find the next page based (I think) on the ID of the last row returned. If I set this dataset up for a regular refresh, will it look at the last ID in the dataset, or would it just restart from the…
-
Append data when columns missing from input data
Hello. I have a master dataset in domo which was originally extracted from a legacy system, and has 'timestamp' and 'id' columns which were specific to the legacy data set. New data is identical but with the exception of the fact that it doesn't include those columns. It seems if I try to append the data to the master data…
-
Create stream for existing dataset
I created a dataset with the Dataset API and since I want to APPEND (not UPDATE/REPLACE) data, I need to use the Stream API. However, creating a stream requires a dataset object. Can I create a new stream for my existing dataset? If so, how?
-
Append Issues (Not matching & not appending)
We have two Append issues which cropped up this week that I'm wondering if other people have run into. It's worth noting that WoW, our process and dataflow *didn't change* which makes this especially baffling. Not Matching Columns We upload an excel sheet via the file upload connector using the append method. This week,…