An unexpected error occurred during export

[klaus]

Good morning everyone,

The Mozambique NSO has just started a survey and are unable to export any data (“An unexpected error occurred during export” message).
Synchronization, viewing interviews, etc. all works fine.

Background information:

Version: 21.09.5 (build 31862)

They had two questionnaires on the server for training purposes and export was working fine.
They then deleted these training questionnaires, they are now shown greyed-out in the Survey Setup.
Then they imported new qx versions and started data collection two days ago. When they try to export they get the error.

When I try to start the export through the API I get a “403 Forbidden” response.

The logs show a number of errors of type “Questionnaire was deleted”:

2022-01-12 04:44:23.038 -08:00 [ERR] Error when dispatching ‘OnConnectedAsync’ on hub.
WB.Core.SharedKernels.DataCollection.Exceptions.InterviewException: Questionnaire was deleted
at WB.Core.BoundedContexts.Headquarters.Interview.Validators.QuestionnaireStateForInterviewValidator.CheckQuestionnaireAndThrow(QuestionnaireIdentity questionnaireIdentity)
— End of stack trace from previous location —
at WB.Core.Infrastructure.CommandBus.CommandRegistry.<>c__DisplayClass18_0.b__0(IAggregateRoot aggregate, ICommand command)
at WB.Core.Infrastructure.CommandBus.Implementation.CommandExecutor.ExecuteEventSourcedCommand(ICommand command, String origin, Type aggregateType, Guid aggregateId, IEnumerable1 validators, IEnumerable1 preProcessors, IEnumerable1 postProcessors, Action2 commandHandler, CancellationToken cancellationToken)
at WB.Core.Infrastructure.CommandBus.Implementation.CommandExecutor.ExecuteCommand(ICommand command, String origin, CancellationToken cancellationToken, Guid aggregateId)
at WB.Core.Infrastructure.CommandBus.Implementation.CommandService.<>c__DisplayClass16_0.b__1(ICommandExecutor ce)
at WB.Core.BoundedContexts.Headquarters.Implementation.UnitOfWorkInScopeExecutor.Execute(Action1 action, String workspace) at WB.Core.GenericSubdomains.Portable.NamedLocker.RunWithLock(String name, Action body) at WB.Core.Infrastructure.CommandBus.Implementation.CommandService.ExecuteImpl(ICommand command, String origin, CancellationToken cancellationToken) at WB.Core.Infrastructure.CommandBus.Implementation.CommandService.Execute(ICommand command, String origin) at WB.UI.Headquarters.Code.WebInterview.Pipeline.PauseResumePipelineModule.OnConnected(Hub hub) at WB.Enumerator.Native.WebInterview.WebInterview.OnConnectedAsync() at Microsoft.AspNetCore.SignalR.Internal.DefaultHubDispatcher1.OnConnectedAsync(HubConnectionContext connection)
at Microsoft.AspNetCore.SignalR.Internal.DefaultHubDispatcher1.OnConnectedAsync(HubConnectionContext connection) at Microsoft.AspNetCore.SignalR.HubConnectionHandler1.RunHubAsync(HubConnectionContext connection)

Note: We are not trying to export data for one of the deleted questionnaires.

Thank you,

Klaus

[sergiy]

@klaus

The logs show a number of errors of type “Questionnaire was deleted”:

This message is probably unrelated to the export problem. The log may contain other error messages. See what else is there.

@benj is also reporting strange deletion behavior where the questionnaires being deleted get stuck for a long time:

Deleting a questionnaire

Dear Support, We are preparing for data collection in 2 days and when preloading the 16,000+ assignments, I noticed that Unicode did not appear well on the dashboard. This is because I did not set the preloading .tab file in the correct encoding format. I fixed two small errors in my system in Designer, fixed the .tab file to display Khmer Unicode and deleted the old questionnaire version (version 9) with thousands of wrong assignments. I imported the new questionnaire (version 10) and successf…

[klaus]

Dear Sergiy,
You are right, the deleted questionnaire message doesn’t seem to be the cause of the export problem.
Apparently there is a single export job which does not terminate since days now. It repeatedly throws an error:

Npgsql.PostgresException (0x80004005): 42703: column “e9ce9f37-eaee-560f-c890-d63912348c4e” of relation “IOF_NUTRICAO$1-e” does not exist

@vitalii is looking at the logs, we are waiting to see how to get rid of this blocking export job.

Thanks,
Klaus

[sergiy]

Thank you for sharing the logs.

Please try the maintenance M1 procedure according to Procedures for maintenance .

[klaus]

Dear Sergiy, thank you for the link to the procedure. We only removed the temporary schema for the workspace that had the problem and now everything works perfectly again.
Klaus

[sergiy]

Great!
Thank you for confirming!