Reject interviews in locked mode

When checking interviews at the supervisor and HQ level often there are issues in interview files that need further clarification from the interviewer before the issue can be addressed or confirmed.

It would be useful if one could reject interview files in a mode that allows interviewers to read comments and respond, but does not allow them to change/delete answer.

With that functionality one could easily communicate with field workers about issues in an interview file and have a record thereof, while at the same time removing the possibility of an issue being “patched up” while the interview is being rejected. This would be particular useful if one does not trust field workers to take the right action to address the issue. It would allow for a wider range of data quality set-ups, e.g. a systems in which HQ follows up with field workers on issues but organizes the cleaning centrally.

1 Like

Andreas,

It seems your proposal could be addressed through analysis of paradata. You can know from paradata whether and when any changes on the interview were made and informing interviewers about the possibility to observe their actions, most likely, will prevent them from doing any modifications in the questionnaire when it was rejected.

On the other hand, your proposal would lead to creation of a new status which might be not obvious to many of our users and that will complicate the process of survey management.

Hi Misha

Very true, one can identify it with paradata. I am usually counting the number of answers set since the file was rejected as part of my para data mining exercises, and have tried to use it, but
a) it is cumbersome since one needs to actively check files that have come back and fish around the para data potential changes, and
b) while you can discourage field workers from changing answers you don’t actually have control over it.

We sometimes ended up not rejecting and instead getting the information one needs externally via the phone or whatsapp, but then one loses the (really good) comment function that allows you to easily communicate with field workers about specific issues in their interviews (and have a record thereof).

This really is part of a larger concern that we often find field workers just fixing errors in rejected files without gathering additional information. This is problematic and possibly requires revisiting the assumptions underlying the data quality circle in the long run and updating the functionality accordingly. I agree with your concerns about usability.

1 Like

Andreas,

Out of literary 100s of feature requests we receive from our users I cannot name a single one that is absolutely useless. Some requests are pushing that limit, but even these have some merits. For example, we received a request from one of our advanced user to design a tablet UI that would compensate for vibration if one decides to work on a tablet while riding a car on a bumpy rode.

In your case, I am not questioning a usefulness of your proposal. It is useful. No questions about that.

But, I am looking at your feature from the broader costs and benefits of the proposed feature for our user community. Your feature is not passing this cost-benefit analysis. It is expensive to implement, it affects all parts of Survey Solutions increasing a risk of bugs through the system, it complicate the process of survey management, complicates UI, can increase support load because users are likely to misuse it, and, is appealing to a rather small group of users. And, which is very important, there is a way to achieve the same result you are looking for through existing means (albeit not very elegantly).

I also face the issue where interviewers patch up rejected surveys rather than addressing it properly (just flat out removing roster items if they didn’t fill it properly the first time is a common example)

The work around I use is creating my own interviewer account and assigning those surveys to myself.
I then message or talk to the interviewer and try to resolve the issue.

However, this is not scalable.

I think this sort of functionality could be partially addressed with the previous suggestion (From Andreas?) of a feedback system (link) that was able to reference specific interview/interview questions.

There is already a read-only mode for supervisors and HQ on the web interface that allows comments to be added - perhaps enabling this functionality for the interviewer only on the web interface could be a partial solution.

Scott, good to see you have the same experience. I have had many more cases since where rejecting to interviewers has not worked due them patching up the interview rather than fixing it.

In some projects I have also used the “cleaning tablet” as a workaround, but I agree with your assessment that it is not scalable.

Another feature that would be very helpful here is if supervisors could change the interviewer answers. This has also been asked by others. With the para data or maybe a variable in the diagnostics file counting the number of answers changed by a supervisor it is fairly easy to keep track of what they have done.

1 Like

Andreas,

there is only a limited amount of situations where one would need to reject an interview to an interviewer in a read-only mode. (such as to discuss something with the supervisor responding to comments). In most cases the rejection implies changes are to be done by the interviewer. Of course rather than filling out the missing information about person X the interviewer may simply delete person X. That is a discipline issue. Providing a selective control over which questions may be modified and which not is not trivial from the user interface perspective.

So I see this thread as mixing several new features:

  1. review mode for an interview, which would necessitate a new status (as explained in earlier messages);
  2. detection of what was changed by the interviewers since the last review by the supervisor;
  3. selective control over what may be changed by an interviewer after rejection.

First sounds most clear from development perspective, but is of very limited use, and may result in a deadlock where the supervisor thinks the interviewer is working on fixing a problem, while the interviewer can’t do so because not given permission to change anything.

Second sounds interesting and has been requested before, but leaves some ambiguities that need to be worked out (of the kind if I delete “Mary” from the list and then add “Mary” again is it the same “Mary” or a different one and should this be treated as a change??)

Third sounds too complex for both development and practical use.

Using a cleaning account (doesn’t have to be a tablet, unless using questions not supported in web mode) should solve the need for the supervisors to make changes.