The Mozambique NSO has just started a survey and are unable to export any data (“An unexpected error occurred during export” message).
Synchronization, viewing interviews, etc. all works fine.
Background information:
Version: 21.09.5 (build 31862)
They had two questionnaires on the server for training purposes and export was working fine.
They then deleted these training questionnaires, they are now shown greyed-out in the Survey Setup.
Then they imported new qx versions and started data collection two days ago. When they try to export they get the error.
When I try to start the export through the API I get a “403 Forbidden” response.
The logs show a number of errors of type “Questionnaire was deleted”:
2022-01-12 04:44:23.038 -08:00 [ERR] Error when dispatching ‘OnConnectedAsync’ on hub.
WB.Core.SharedKernels.DataCollection.Exceptions.InterviewException: Questionnaire was deleted
at WB.Core.BoundedContexts.Headquarters.Interview.Validators.QuestionnaireStateForInterviewValidator.CheckQuestionnaireAndThrow(QuestionnaireIdentity questionnaireIdentity)
— End of stack trace from previous location —
at WB.Core.Infrastructure.CommandBus.CommandRegistry.<>c__DisplayClass18_0.b__0(IAggregateRoot aggregate, ICommand command)
at WB.Core.Infrastructure.CommandBus.Implementation.CommandExecutor.ExecuteEventSourcedCommand(ICommand command, String origin, Type aggregateType, Guid aggregateId, IEnumerable1 validators, IEnumerable1 preProcessors, IEnumerable1 postProcessors, Action2 commandHandler, CancellationToken cancellationToken)
at WB.Core.Infrastructure.CommandBus.Implementation.CommandExecutor.ExecuteCommand(ICommand command, String origin, CancellationToken cancellationToken, Guid aggregateId)
at WB.Core.Infrastructure.CommandBus.Implementation.CommandService.<>c__DisplayClass16_0.b__1(ICommandExecutor ce)
at WB.Core.BoundedContexts.Headquarters.Implementation.UnitOfWorkInScopeExecutor.Execute(Action1 action, String workspace) at WB.Core.GenericSubdomains.Portable.NamedLocker.RunWithLock(String name, Action body) at WB.Core.Infrastructure.CommandBus.Implementation.CommandService.ExecuteImpl(ICommand command, String origin, CancellationToken cancellationToken) at WB.Core.Infrastructure.CommandBus.Implementation.CommandService.Execute(ICommand command, String origin) at WB.UI.Headquarters.Code.WebInterview.Pipeline.PauseResumePipelineModule.OnConnected(Hub hub) at WB.Enumerator.Native.WebInterview.WebInterview.OnConnectedAsync() at Microsoft.AspNetCore.SignalR.Internal.DefaultHubDispatcher1.OnConnectedAsync(HubConnectionContext connection)
at Microsoft.AspNetCore.SignalR.Internal.DefaultHubDispatcher1.OnConnectedAsync(HubConnectionContext connection) at Microsoft.AspNetCore.SignalR.HubConnectionHandler1.RunHubAsync(HubConnectionContext connection)
Note: We are not trying to export data for one of the deleted questionnaires.
Dear Sergiy,
You are right, the deleted questionnaire message doesn’t seem to be the cause of the export problem.
Apparently there is a single export job which does not terminate since days now. It repeatedly throws an error:
Npgsql.PostgresException (0x80004005): 42703: column “e9ce9f37-eaee-560f-c890-d63912348c4e” of relation “IOF_NUTRICAO$1-e” does not exist
@vitalii is looking at the logs, we are waiting to see how to get rid of this blocking export job.
Dear Sergiy, thank you for the link to the procedure. We only removed the temporary schema for the workspace that had the problem and now everything works perfectly again.
Klaus
We finished our 2021 survey quite some time ago (Jan 2022). But we kept the server live as some interviews with commercial agriculture holdings needed postponing and because it was easier for interviewers to get back to the original interview cases when supervisors or data cleaning staff identified issues. Now is the time to close the instance and we successfully downloaded all data from the two questionnaire versions we had, except for the paradata for the version with the highest number of cases. We would like to try and access this paradata as it could help answering some questions we have about data quality.
It seems like we have the same error as described in this thread, which is why I am following up on it. We tried to update the SuSo version to see if it could help solve the issue (it did not), and I downloaded the logs, as suggested (access them at https://drive.google.com/drive/folders/1AZ8sHox-hqVXTmi0tpXDQzMEUrJwIdhg?usp=share_link). I am not sure about what I should be looking for here. Can you spot any issues? What should my next move be?
Thanks a lot.
Thank you for your assistance for resolving the issue. We manage to download the paradata.
The server storage was full due to multiple exported files during the survey (197 exported files available at C:\Survey Solutions\Data_Site\export. Will it be possible to add the delete button next to the download on the data export page? Administrator can delete those files after downloading and never run into storage full issue.