Survey duration and actions

Hello everyone!

I have a doubt that I wasn’t able to resolve with the documentation and past forum discussion.
We are fielding a survey and having trouble reconciling the data on survey duration.
Here’s an example: this survey shows a duration of 2 minutes, but the paradata shows that it lasted 1hr and 40min (see picture below).

Having noted the issue in the past, this survey was conducted by an enumerator under the supervision of a monitor who kept record of start/end time. The paradata matches our observations, but the duration displayed on the website is confusing.
Am I missing something? Is there a way to reconcile these different durations?

HI @andrea_s,

According to the official documentation, the interview duration (which you can more accurately track in the interview__duration variable of the interview__diagostics file), interview duration is the “active time it took to complete the interview”. See documentation below and look for the interview__diagnostics section.

In light of this:

  1. I believe this same interview__duration is the one shown in the interviewer webview, as in your screenshot.
  2. The “paradata” in your screenshot, such as for “Created”, “Completed”, “Approved” etc., are simply timestamps that correspond to specific actions.
    • Created tells you the point in time at which an assignment was created
    • Completed tells you the point in time at which the corresponding interview was completed and synchronized. Etc…
    • This information can also be found in the system-generated file interview__actions.
  3. Which one you should use depends on what you want to measure
    • Use the interview__duration from the diagnostics file when you want to know exactly how long an interview lasted.
    • Use the timestamp paradata in the interview__actions when you want to measure time intervals between different actions.
    • The former gives you a better idea about actual interview duration. The latter simply captures timing of action. If I receive an assignment today, but I complete and sumbit it 3 days later, it does not mean that the interview duration was 3 days.

Hope this helps!

Hello @giansib - thanks, this is helpful.
I guess I’m more puzzled by the fact that we observed the survey being open on the tablet for a time that matches the paradata, but the interview__diagnostics does not reflect this. Surely there’s an issue somewhere - not sure whether with the hardware or software.

Based on what I can infer about the size of your questionnaire in the screenshot, 2 minutes does seem too short! Perhaps a Survey Solutions support staff has some more input here.


In addition, I found this:

Where the main recommendation in the thread is to check your paradata files

Clicking on the Download Interview Transcript will give you an easy way of checking.

Realistically, I don’t see a way somebody could answer 309 questions in 2 minutes.
But then again, you are comparing apples and oranges. The Interview status history shows you when certain events have happened, and this doesn’t mean that the space in-between is to be prescribed to a certain activity - the interviewer may have very well switched to a different interview.

Regarding paradata you are probably not using this, based on the screenshots. What matters is the paradata export file itself, which you can download to see the recording of actions in a particular interview. Then, once you have it, it will be easier to detect any discrepancies (if any).


Realistically, I don’t see a way somebody could answer 309 questions in 2 minutes.

I agree, that’s why I wrote :slight_smile:

I should stress, one of our team members was with the enumerator and confirmed they did not take 2 second to run the survey. The paradata also confirm this.

Going through the paradata, I did however notice something unexpected: the “order” variable and the “timestamp_utc” variables are NOT perfectly aligned. That is, looking at the timestamps, event 446-464 happened in between 440 and 441. Any thoughts as to why this might be the case?

Anyway, this doesn’t quite explain the 2 seconds duration, as by that point (event 440) the paradata shows close to 1hr worth of work.

Any idea?

Tablet clock was adjusted (if this was a tablet-based/CAPI interview).

It does explain. There was no interviewing between 13:40 and 14:52 as evidenced by the Paused/Resumed pair of events, shown in below in the green block.

I’d be more puzzled why the total is 2 minutes and not 3 minutes and 45 seconds.

You may want to find out from the staff what was happening in the red block, that the clock got adjusted.
From the practical standpoint, I’d exclude such interviews from processing, as manipulating the clock will undermine the indicators you are trying to derive from this, like duration of interview.

Hello Sergiy,
indeed, clock adjustments were also my prior - even from the actions data, we sometime see surveys completed earlier than when they started. That’s why we sent monitors on the field.
I got confirmation that there was no funny business (during the monitoring). I guess it must be an issue with the tablets.

Going back to the example, though, what I showed is an extract of the paradata - there were ~400 events before the ones shown, the first one at 12:53. Even accounting for pauses, the total running time was more than an hour. Is there any particular reason why the duration shown on the portal would be only 2min? Even allowing for clock adjustments etc, I cannot quite grasp why the duration recorde would be so low.
We are discussing the issue of low survey durations with the survey firm we contracted, and I cannot quite square this data.

@andrea_s , you have shown a portion of the data with some 30 events and asked this to be commented on.

I have commented.

Now you are asking me to comment on some 400 events that you do not show.

I can’t.

here’s the full example, extracted from the dataset.
it includes interview_actions, interview_diagnostics and paradata. I have removed the “parameter” field from the latter file as it includes PIIs
Events [6,89] are pre-filled questions.

Thank you for showing the data.

You had nine editing sessions for this interview with the following durations:

  1.     454  
  2.     111  
  3.     217  
  4.     336  
  5.      82  
  6.     471  
  7.     159  
  8.     900  
  9.   -2581  

Survey Solutions averaged added up this to obtain 149 seconds, which is 2*60+29=2m29s or about 2 minutes as you are seeing in the Survey Solutions interface.

Any adjustments of the tablet clock invalidate the duration as calculated using this method.

Thanks, this is what I was looking for.
Just to be super clear: how did you define an editing session? And then these are added up, right?

Any period between Resumed and Paused events is counted as an editing session. If the Paused event is missing (in rare cases where the tablet for example completely ran out of power) Survey Solutions will generate a synthetic Paused event with some assumptions: Adds 15 minutes to that session as per this file (see around line 980). This is rare.

1 Like