Audio audit by question

Feature Suggestions
#1

Hello Survey Solutions team.

Sometimes we create (in the designer) questionnaires with many questions or lists, then we decide to activate the audio audit, but this could generate a large file (big size) to sincronize and exporte.

So, it might be a good idea to define in the designer what questions we want to audit.

For example, if my questionnaire has 300 questions, but I’m only interested in auditing 10 questions, in this case, my file of audio audit of only 10 questions is smaller than my file when I decide to activate the audio audit for the questionnaire.

Perhaps this could be a good idea to active the audio audit in questionnaires with many questions or dinamic rosters.

Thanks for this amazing tool.

#2

Hi, Kevin,

suppose you’ve marked question q121 in the section A (with questions q1…q300) as one with desired audio audit. When should the recording start? when should it stop?

There is no way of telling which question the interviewer is reading/asking now. Hence the whole interview is recorded.

While the files may be large, you don’t have to activate audio audit for every interview, but place the flag strategically. this will reduce the subsequent work load on supervision.

#3

Hi, Sergiy,

I agree with you, that is a problem know when should start and stop the recording.
But maybe can define a process like this (It’s just an idea):
If Q15 is the question that wish recording with audio audit.
Start recording when Q14 is answered
Stop recording when Q15 more one minute (maybe needs a few time) is answered.

But what happen if the interviewer answered Q14 and he stop the interview or move to another section in the interview or Q15 isn’t enable? Then for those cases should be necessary define a default time to recording.

It’s just an idea and can be more analyze if you know some event reported in the paradata file that let you start or stop the recording audio audit.

Exactly that is the best way to do it. Just somes interviews choosen random strategically.

#4

just some problems with this approach would be:

  1. the interviewer answers q14, recording starts, q15, recording finishes, then goes to q14 (for any reason to change the value there), recording starts again, the interviewer proceeds to q16, but the recording expects to record q15.

  2. interviewer answers q14, which actives q14.5, which was not active before, recording starts, q14.5 is asked by the interviewer.

  3. interviewer reads q14, gets, but doesn’t enter the answer, asks q15, then enters both values (typical behavior in “Did you buy any…?”, if yes “How much?”)

etc, etc.

If there is any software that is doing this in a NOT-one-question-per-screen mode, please point me to it.

In other words, we can only start the recording BEFORE the interviewer gets a chance to see the question. Otherwise he can ask the question, and then start doing something to which we can react. So we start the recording BEFORE the interview, which means before any question is displayed. Still the interviewer who has done a bunch of interviews would know the first question by heart (in simple surveys, without branching on preloaded data) and can ask it even before opening the interview.

#5

I know those problem with this approach.

Yesterday when I commented it also thought in those cases where the interviewer has others actions differents than expected.

No, I don’t know.
I had this idea because one person asked me if Survey Solutions allowed audio auditing by questions or by sections.

But with your explain I am clear about how works audio audit in Survey Solutions.

#6

Dear survey solutions team,

I realize this in an old suggestion, but I did want to respond as it would be a useful feature I think. In our survey doing a complete audio check is unlikely to be possible given limited internet and interviews that can take up to an hour.

I understand the limitations that you mentioned Sergiy, but perhaps they would not preclude such an option completely.

  1. this is about auditing, not the primary data collection. Therefore, in my humble opinion, it does not have to be perfect as primary data-collection does. E.g., the ‘errors’ that you give in the example (change to q14 and therefore rerecording, or enabling 14.5), would not be ‘terrible’. Yes the audit perhaps fails in some of these instances, but does not compromise anything about the primary data collection, and in most cases it will succeed (especially if the designer is clever about which questions to audit, avoiding some of the possibility for error you mentioned - see point 2).

  2. Couldn’t you avoid some of these problems by enabling conditions for the audio to start (e.g., it only activates recording if 14.5 is not enabled).

  3. Perhaps it could be considered to allow sections to be audio audited rather than individual questions. I understand that doesn’t solve the principle of some of the issues you raised (sections can be skipped, etc), but would perhaps reduce the chance of error.

My main argument however would be that audits are valuable already if they work in the (far) majority of cases. A supervisor would recognise the situations in which it didn’t work as intended and this would therefore not be problematic. This could be explained in the accompanying information to selective or partial auditing, so designers and users (HQ/supervisors) are able to take it into account.

Perhaps there are other arguments that preclude such an option, but because of the value I think it could have, I did want to share my thoughts. As always, I should say that I enormously appreciate the Survey Solutions tool and team and the continuous development and new possibilities, so nothing I said is meant to criticize, only to think along with you.

Best wishes,

Sebastiaan

#7

This is what this forum is for.

On the subject, imagine the interviewer submits the interview without income - a key question in the survey, insisting that the respondent refused categorically, hence no answer. The supervisor marks as need to be answered and rejects the interview. The interviewer opens the interview, reads the comment, goes to the household, asks the question and touches a single button (e.g. income category 0-500, 501-1000, etc). There was absolutely no activity with the program from the point where the interviewer read the comment (“ASK INCOME”) till the answer was recorded. (Perhaps it took him 2 hours to travel to the respondent, perhaps 2 minutes, perhaps he just called him on the phone and you can’t hear the response). So no anchor point to start recording. And since it can happen at any time, then the recording must be continuous (similar to dash-cams / auto-crash-cams where a crash may occur at any unpredictable instance, the recording then must be constant). This is also hard. Covert recording is discouraged by the security considerations by the OS manufacturers, so we might just be seeing the last days of it.

#8

Hi Sergiy,

If it is on the way out anyway because no longer supported than that is a shame, and I understand that you don’t want to invest building a new addition for a short period of usability.

However, I still think that the audit audio does not have to be full proof (other than with primary data collection) and should as with anything be used wisely by supervisors and designers together with other ways to check data and enumerators. I would not use audio in the example you mention (to collect missing data afterwards). If a designer does this in that scenario, perhaps they are silly and deserve to waste time. In an example like this, I would ask the enumerator to comment, and if I don’t trust them, ask the supervisor to do a repeat visit or call.

I still think though that it could be a valuable addition to existing checks in other scenarios, e.g., that difficult questions are read out as supposed, that the enumerator does not lead the respondent, and other cases. That it isn’t useful in all situations, doesn’t mean it can’t be useful.

Cheers,

Sebastiaan