Allow Supervisor to mark when rejected question issue is resolved


(Scott Sheridan) #1

In the current workflow there is no way to mark a reason for interview rejection as resolved.
This means that when an interview is rejected a second time, it is difficult to determine which questions require review, as all questions with comments are still marked with red on the cover page area that lists issues.

It would be nice if on the interviewer app, Flagged questions were marked, rather than those with comments.
This would allow the Supervisor to flag and comment on questions that require review and then unflag them when the issue is resolved.

It would also be great if the questions requiring review could be marked with an icon in the side menu of Interviewer.
It is currently unclear which questions require review unless the user goes back to the cover.
The interviewer could then clear their interviewer side flag, indicating that they believed it resolved so it doesn’t show on their UI unless the supervisor rejects the interview again.

An alternative would be to have a ‘Flagged’ subsection in the interviewer app denoting questions that the supervisor has flagged for review, enabling the interviewer to view just these questions.
If this was implemented, there could also be a ‘Questions with Commented’ subsection.


(Arthur Shaw) #2

Scott, I really like the proposal. While this might be a reductive reading, I see a few ideas here:

  • Facility for flagging issues. For HQ/Supervisor, the only existing mechanism is to post a comment. But it strikes me that perhaps the flag tool on the interview details screen, presently used only as the digital equivalent of a sticky note, could be re-purposed to do what you’re suggesting. Here’s how it all might work. When HQ/Supervisor adds a flag to a question, it would appear as something that the recipient needs to review. (Note: flagged questions may also have comments.) If the interview is rejected, flagged questions would appear in a list of things the interviewer needs to review and resolve. (I need to think more about where in the UI.) If the interview is approved by a supervisor, flagged questions could be ones that require HQ input (e.g., checking an industry code).
  • Mechanism for marking issue as resolved. Each role in the system needs to mark an issue as resolved. When an interviewer remedies issues in a rejected questionnaire, they need to mark the issue as resolved (at their level). When HQ/Supervisor reviews a previously rejected interview, they need to mark issues as effectively resolved by lower levels in the quality control hierarchy.

A few follow-up questions:

  • Does that seem like a faithful summary of your ideas? Did I miss anything?
  • Would you prefer this functionality to what Andreas is proposing here–to have user-defined tags that can communicate finer grades of status (e.g., rejected interview to check again carefully) or next actions (e.g., check industry code)? Or do you view your feature idea as a complement rather than a substitute for Andreas’?
  • How are you emulating/approximating the workflow you suggest with current functionality? What are your work-arounds?

(Scott Sheridan) #3

Yes, that’s a faithful summary.
On your note - I would suggest Flagged questions should (must?) have comments. Additional functionality could differentiate between flag related comments and other comments, linking the comment to the flag and remove it when the issue is un-flagged.

I think you posted the FAQ link a second time rather than the one to Andreas’ proposal so I can’t answer that question with certainty yet.

The workflow I have to emulate this currently is only supervisor side - flagging the issues and commenting on them outlining the issue and then un-flagging them when they have been resolved.
In order to communicate which questions still have issues, I am just sending more detailed supervisor notes on the second rejection, individually indicating which questions still have issues.

I have noticed that in many cases my interviewers only fix the first issue because they click it, fix it and then it’s not clear that there’s more issues to be fixed.

It’s a failure on my part for not training them sufficiently to ensure they understand the current review process (which will be addressed with the next set) but also it could be made more intuitive.
My interviewers come from fairly low-tech backgrounds so this is often the first time they’ve used a tablet and they can be a bit tentative in exploring and in retaining the knowledge from training as they’re also learning how to navigate a tablet application for the first time.


(Arthur Shaw) #4

Apologies, here’s the link I meant to provide to Andreas’ proposal


(Scott Sheridan) #5

I think they’re separate suggestions - he’s talking flagging/tagging whole interviews whereas this suggestion is on the per question level.

I think both suggestions could complement each other well but neither could address the other’s needs.

I like his suggestion and it would also be nice to filter on the interviews with flags/comments and have a column that shows how many an interview has so you can track the troublesome interviews


(Scott Sheridan) #6

Just a slight bump on this one - especially the marking issues as resolved from the supervisor level.

It really isn’t clear on the interviewer side when they receive a re-rejected interview which of the questions with comments are the ones that still have issues.
Just a facility for the supervisor to remove a question from the list of questions shown on the cover of a rejected interview would be tremendously helpful.


(Sergiy Radyakin) #7

Colleagues, what are the best supervision practices currently used during the field work for indicating questions that need interviewer attention?

Please, take a minute to describe below how do you currently manage this in other countries, surveys.


(Scott Sheridan) #8

Specifically when an interview is rejected for a second time, either by the supervisor or maybe for additional reasons by HQ.

Apart from the rejection comment, it’s a bit clunky as the questions with comments that have already been resolved are given just as much priority on the cover as issues that have not been resolved.

I feel like it should be the flagged issues that are highlighted on the cover rather than those with comments.