Troubleshooting for Unable to retrieve data error

Rifqi

I want to create a datasetview using some datasources in redshift. However, I frequently facing unable to retrieve data error message while preparing the data. It really being a bottleneck to get my job done.

For context, I want to perform simple inner join between Table A and Table B. Table A contains 120 million rows and Table B contains 115 million rows. I also limiting the data by only select necessary fields (10 columns from Table A and 15 columns from Table B) but still no luck for me.

I am not sure where to start to find the root cause, my internet connection, computer memory, domo instance or any else? Is there any guide to troubleshoot this issue?

Thanks in advance!

ArborRose

It can be difficult to work with large datasets across systems. In SQL, I prefer using OPENQUERY to run the process on the remote server and return just the result.

Similarly, Dataset Views and Magic ETL in Domo pull in full datasets before doing joins, which isn't ideal for massive tables. Limiting fields helps, but not enough when row counts are huge.

Check where the issue happens—during preview or when joining? Try to offload work to Redshift first. Use pre-joined views or filtered tables. Federated datasets may help since they run queries directly on Redshift without importing full data into Domo.

ArborRose

It can be difficult to work with large datasets across systems. In SQL, I prefer using OPENQUERY to run the process on the remote server and return just the result.

Similarly, Dataset Views and Magic ETL in Domo pull in full datasets before doing joins, which isn't ideal for massive tables. Limiting fields helps, but not enough when row counts are huge.

Check where the issue happens—during preview or when joining? Try to offload work to Redshift first. Use pre-joined views or filtered tables. Federated datasets may help since they run queries directly on Redshift without importing full data into Domo.

ArborRose

This link may be helpful:

https://community-forums.domo.com/main/discussion/52282/what-are-best-practices-for-combining-huge-datasets-in-domo