Already in queue for execution

milesscott

We are having a problem with Workbench where it looks as though the jobs are not running because the queue is backed up.  When I look at the Workbench Service log files I see the following: 

 

Job Id 82 - next run date of 2015-06-26 05:30:00 Local is past due. Already in queue for execution.

 

Any thoughts or ideas how to clear out the queue to make the jobs begin running as scheduled again?

nalbright

I agree with you.  Perhaps something on the order of an iterative experiment is needed.  Since you know that it will upload manually, and will upload at least daily, but will not upload ever 15 minutes, perhaps you can simply try in fifteen minute increments to allow the most effective use of workbench and also allow for the most frequent updating.  Try 30 minutes, and if that does not work, try 45, and then 60, and so on.  Assuming that there will be future data feeds or larger data feeds that will be uploaded in the future, it would be good to check out the queue from time to time and update the frequency of updates accordingly, if necessary.  Does this sound like a workable solution to you?

milesscott

Sorry for the slow response, but we did get this issue resolved.  We ended up turning off all jobs for a day, then went back and scheduled them over much broader intervals.  Maybe some of the files were to big and backing up the whole system, especially if we were trying to run them every 15 minutes or half hour.

 

Since we did that things, have been running smoothly.  Thanks for all the input and response everyone.

nalbright

If you find an issue with one of those jobs, in workbench, you can manually validate and upload the file.  This is a bit time consuming but will at least clear up the queue as fast as you can click and wait.

kshah008

@milesscott, did nalbright's response help address your issue with the queue?

milesscott

 

I know I can manually update, but the scheduler was working fine until a week or so again.  I'll post the log files from teh workbench service.  It looks like the scheduled jobs aren't running because of this.  Anyone seen this before and how do you clear the queue?

 

 

 

 

nalbright

I think that you're correct that the scheduled jobs aren't running because something is stuck.  Do you have any idea what changed between the before and after?

milesscott

I don't know of anything that really changed and that is the frustrating part.  I think I am going to remove the schedule feature from all the jobs and let it go for a day and see what, if anything happens.  I'll manually update the critical ones and see if it somehow goes through and clears out the queue in the meantime.

nalbright

I can see how that would be very frustrating.  Could you let us know if anything changes as a result of your test?

milesscott

Sure will.

milesscott

So I changed some of the jobs to Manuelly update, some to update every 15 mintes, some to every day.  

 

It appears as though the manuel ones didn't run at all, as expected.  And the everyday ones only ran once a day, also as expected.

 

However the ones every 15 minutes only ran once a day, if at all.  And it looks in the log files as though it is still all backed up and looks the same as the post above.  

nalbright

Now that's very interesting.  Is the data that you're trying to update every 15 minutes actually changing that frequently?

milesscott

It does, and maybe every hour would work fine too.  But our executives would certainly like to see this data updating more frequently then once a day.

nalbright

Certainly so.  Alright, so if the data is actually updating, but Workbench has trouble uploading it, and there is nothing in the data that is changing any of the parameters for the data flow, then the question is, what is causing the trouble for those particlar uploads?  Is it just taking too long to get them all done in 15 minutes so that the entire workbench workflow looks like rush hour traffic where no one is going anywhere, or is there something else that is a problem?

milesscott

I have a feeling that workbench is just backed up.  The problem then is how to clear the queue to allow the jobs to run automatically again, albeit not nearly as frequently. 

nalbright

I agree with you.  Perhaps something on the order of an iterative experiment is needed.  Since you know that it will upload manually, and will upload at least daily, but will not upload ever 15 minutes, perhaps you can simply try in fifteen minute increments to allow the most effective use of workbench and also allow for the most frequent updating.  Try 30 minutes, and if that does not work, try 45, and then 60, and so on.  Assuming that there will be future data feeds or larger data feeds that will be uploaded in the future, it would be good to check out the queue from time to time and update the frequency of updates accordingly, if necessary.  Does this sound like a workable solution to you?

jeremyhurren

Just a couple of ideas for troubleshooting...

 

1. You can restart the Workbench service using the system tray icon - right click, then choose Service and Restart. The running job will be terminated, all the queued jobs will be lost since they are all in memory. But as soon as the service starts again it will re-queue all those jobs since they haven't been successfully completed.

 

2. You can look in task manager for a workbench.datacollector process. If you view the command-line parameters you will be able to see the jobid that is running. You can look at the log for that specific job and it may give some additional information about what is stalling. (Alternatively, you can find this information from the log you posted, you just may have to search a bit to find where it logs the most recent time that it's starting a job.)

 

kshah008

@milesscott, were you able to resolve your issue? Please let us know!

Austin

@milesscott,
Did @jeremyhurren reply help you with your issue? Were you able to get this resolved?
-Austn
Domo Support

milesscott

Sorry for the slow response, but we did get this issue resolved.  We ended up turning off all jobs for a day, then went back and scheduled them over much broader intervals.  Maybe some of the files were to big and backing up the whole system, especially if we were trying to run them every 15 minutes or half hour.

 

Since we did that things, have been running smoothly.  Thanks for all the input and response everyone.

[Deleted User]

Greetings,

 

I went ahead and marked @nalbright & the last post by @milesscott as solutions for this thread.

Thanks all for the great collaboration!

 

Regards,

Dani

nalbright

You're very welcome; I'm glad to see it resolved ?