Connectors

Connectors

Recommendation Needed - Import NetSuite Transaction Data By Last Import

I am looking for the most effecient approach to import NetSuite sales orders into a Dataset. I was able to get data imported with a SQL query with no issues. However, the dataset refresh is reloading all order data every time it runs.

Is there a method where I can filter on createddate on the NetSuite transaction to be > that the last time the Domo import process started? Only need to import hundreds or couple thousand records a day. Seems improper to load over 1M rows of data to just get most recent additions.

Welcome!

It looks like you're new here. Members get access to exclusive content, events, rewards, and more. Sign in or register to get started.
Sign In

Answers

  • If you store the last successful created date in a Domo variable, dataset or other….you could use that as part of your SQL query

    1. -- Get the last loaded date from the tracking dataset
    2. SELECT MAX(loaded_date) as last_loaded_date
    3. FROM last_import_tracking;


    4. -- Use that date to filter new records from NetSuite
    5. SELECT *
    6. FROM sales_order
    7. WHERE createddate > (SELECT MAX(loaded_date) FROM last_import_tracking);

    ** Was this post helpful? Click Agree or Like below. **
    ** Did this solve your problem? Accept it as a solution! **

  • @ArborRose Could you expound on this more, I am looking to do the same thing.

  • In December, I made my New Year's Resolution to stay off the leader board in this forum {sad face}.

    My response to the question was that you can identify the most recent load date with a query, and use that to filter your query so it's increment (only gets the newest information).

    I haven't looked at any of the NetSuite connectors in a minute, but I believe you can use incremental loading in the Domo NetSuite connector. When you set the data pull, use the `createddate` field as a filter by setting it to be greater than a specific timestamp (last import start or end date).

    I'd have to think through a process. Something to store the timestamp of the last run. So after every successful import, you track the date. Then when configuring, choose Append instead of replace.

    ** Was this post helpful? Click Agree or Like below. **
    ** Did this solve your problem? Accept it as a solution! **

Welcome!

It looks like you're new here. Members get access to exclusive content, events, rewards, and more. Sign in or register to get started.
Sign In