Reject interviews in locked mode


(Andreas Kutka) #1

When checking interviews at the supervisor and HQ level often there are issues in interview files that need further clarification from the interviewer before the issue can be addressed or confirmed.

It would be useful if one could reject interview files in a mode that allows interviewers to read comments and respond, but does not allow them to change/delete answer.

With that functionality one could easily communicate with field workers about issues in an interview file and have a record thereof, while at the same time removing the possibility of an issue being “patched up” while the interview is being rejected. This would be particular useful if one does not trust field workers to take the right action to address the issue. It would allow for a wider range of data quality set-ups, e.g. a systems in which HQ follows up with field workers on issues but organizes the cleaning centrally.


(Misha) #2

Andreas,

It seems your proposal could be addressed through analysis of paradata. You can know from paradata whether and when any changes on the interview were made and informing interviewers about the possibility to observe their actions, most likely, will prevent them from doing any modifications in the questionnaire when it was rejected.

On the other hand, your proposal would lead to creation of a new status which might be not obvious to many of our users and that will complicate the process of survey management.


(Andreas Kutka) #3

Hi Misha

Very true, one can identify it with paradata. I am usually counting the number of answers set since the file was rejected as part of my para data mining exercises, and have tried to use it, but
a) it is cumbersome since one needs to actively check files that have come back and fish around the para data potential changes, and
b) while you can discourage field workers from changing answers you don’t actually have control over it.

We sometimes ended up not rejecting and instead getting the information one needs externally via the phone or whatsapp, but then one loses the (really good) comment function that allows you to easily communicate with field workers about specific issues in their interviews (and have a record thereof).

This really is part of a larger concern that we often find field workers just fixing errors in rejected files without gathering additional information. This is problematic and possibly requires revisiting the assumptions underlying the data quality circle in the long run and updating the functionality accordingly. I agree with your concerns about usability.


(Misha) #4

Andreas,

Out of literary 100s of feature requests we receive from our users I cannot name a single one that is absolutely useless. Some requests are pushing that limit, but even these have some merits. For example, we received a request from one of our advanced user to design a tablet UI that would compensate for vibration if one decides to work on a tablet while riding a car on a bumpy rode.

In your case, I am not questioning a usefulness of your proposal. It is useful. No questions about that.

But, I am looking at your feature from the broader costs and benefits of the proposed feature for our user community. Your feature is not passing this cost-benefit analysis. It is expensive to implement, it affects all parts of Survey Solutions increasing a risk of bugs through the system, it complicate the process of survey management, complicates UI, can increase support load because users are likely to misuse it, and, is appealing to a rather small group of users. And, which is very important, there is a way to achieve the same result you are looking for through existing means (albeit not very elegantly).


(Scott Sheridan) #5

I also face the issue where interviewers patch up rejected surveys rather than addressing it properly (just flat out removing roster items if they didn’t fill it properly the first time is a common example)

The work around I use is creating my own interviewer account and assigning those surveys to myself.
I then message or talk to the interviewer and try to resolve the issue.

However, this is not scalable.

I think this sort of functionality could be partially addressed with the previous suggestion (From Andreas?) of a feedback system (link) that was able to reference specific interview/interview questions.

There is already a read-only mode for supervisors and HQ on the web interface that allows comments to be added - perhaps enabling this functionality for the interviewer only on the web interface could be a partial solution.