Workbench Weird Issues

imelendez

Hello community,

Has anyone encountered the inability to restart/stop the Domo workbench service? Even via command line as administrator on a Windows 2019 Server?

Everytime I run the command or try to stop/restart via the GUI in task bar I get a message saying I cannot do it.

I checked the workbench logs today and saw this:

[ERROR] [5280:16] 2022-03-28 03:59:25,327 - Could not query job from the server.

System.Net.Http.HttpRequestException: Response status code does not indicate success: 401 (Unauthorized).

  at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()

  at DomoApi.DomoClient.HandleErrorStatusCode(HttpResponseMessage response) in C:\jenkins\workspace\WorkbenchBuild_release-fall-2021\DomoApi\DomoApi\DomoClient.cs:line 767

  at DomoApi.DomoClient.EnsureSuccessStatusCode(HttpResponseMessage response) in C:\jenkins\workspace\WorkbenchBuild_release-fall-2021\DomoApi\DomoApi\DomoClient.cs:line 380

  at DomoApi.DomoClient.<SendRequest>d__138.MoveNext() in C:\jenkins\workspace\WorkbenchBuild_release-fall-2021\DomoApi\DomoApi\DomoClient.cs:line 749

--- End of stack trace from previous location where exception was thrown ---

  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

  at DomoApi.DomoClient.<PutAsync>d__130.MoveNext() in C:\jenkins\workspace\WorkbenchBuild_release-fall-2021\DomoApi\DomoApi\DomoClient.cs:line 622

--- End of stack trace from previous location where exception was thrown ---

  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

  at DomoApi.Manager.AgentsManager.<SetJobNextRun>d__57.MoveNext() in C:\jenkins\workspace\WorkbenchBuild_release-fall-2021\DomoApi\DomoApi\Manager\AgentsManager.cs:line 736

--- End of stack trace from previous location where exception was thrown ---

  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

  at WorkbenchSDK.Configuration.WorkbenchJob.<UpdateNextRun>d__70.MoveNext() in C:\jenkins\workspace\WorkbenchBuild_release-fall-2021\WorkbenchSDK\Configuration\WorkbenchJob.cs:line 638

--- End of stack trace from previous location where exception was thrown ---

  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

  at Workbench.Service.WorkbenchService.<UpdateJobNextRun>d__101.MoveNext() in C:\jenkins\workspace\WorkbenchBuild_release-fall-2021\Workbench.Service\WorkbenchService.cs:line 2098

--- End of stack trace from previous location where exception was thrown ---

  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

  at Workbench.Service.WorkbenchService.<DomoJobWorkItemCompleted>d__66.MoveNext() in C:\jenkins\workspace\WorkbenchBuild_release-fall-2021\Workbench.Service\WorkbenchService.cs:line 997

GrantSmith

I've had some issues where I attempt to stop the service and it'll say it's unable to be stopped but actually stopped the service. Typically I need to reboot the server to fix the issue properly though.

imelendez

Boo. Yeah I had to reboot the service like 5 times and also kill the service via command prompt in order to get it to work again. So odd.

GrantSmith

I'd recommend logging a ticket with support and raising the issue with them so they have better visibility into the frustration it's causing to help encourage them to see if they can fix it in the future.

user02961

I see this post is about a year old, but this one just happened to us today. We've had Workbench running perfectly healthy for months, and apparently late yesterday without warning or notification, Workbench jobs just stopped running. The server did not experience any power issues or network interruptions - it remained 100%. Checking the logs, the same "Response status code does not indicate success..." errors that @imelendez mentions above. It took a server reboot and then three successive Workbench service restarts to finally get it back to synchronizing our jobs. Running a job manually worked fine, but nothing within the Workbench program would kick the scheduler back into gear. I am opening a support ticket just to get it on their radar because of the stark nature of data stopping without any notification and having to find out about it from an end-user wondering why metrics aren't updating. I will post an update if support determines root cause.

BrianBeau

We had this issue this morning @user02961 1- all workbench jobs stopped running at 11p EST last night. IT was pushing a security patches and it looks like it caused the services to not connect properly. Reinstalling workbench, and rolling back the patch got it working again.