Quirky things about CAWI

Dear SuSo team,

Here are a few comments re: the CAWI feature:
• After you download a list of links for interviews, the next time you download links, the links from the previous file are gone. I would prefer them all to remain in that file.
• When a respondent opens link, they have 1 chance to complete the whole questionnaire otherwise the link will expire. It would be nice if the link stays active until they complete the survey.
• If a respondent opens the questionnaire with the link, answers a few questions, and closes it, the status remains, “InterviewerAssigned”. It should change to Started, or something of that nature.

Thanks, Michael

Michael, not exactly

  1. you can download the links multiple times in the current version too. But you download the links to the assignments that haven’t been fulfilled yet (the ones that need to be mailed out).

  2. the link leads to an ASSIGNMENT not to an INTERVIEW. Once the quota for interviews is exhausted no new interviews will be created from the assignment. The respondent may retain the INTERVIEW link and continue the interview at a later time.

  3. The InterviewerAssigned status means this is a web interview in progress. No other interview will have this status meaning anything else on the server.

Dear Survey Solutions team,

On point 2, I realise the difference between assignments and interviews, but for web interviews where each respondent receives one assignment, doesn’t this difference disappear in practice?

I agree with Michael that in this context it would be useful to somehow keep the original link that was emailed to participants open, retaining answers entered in a previous session. For example, I can imagine this to be useful if respondents’ internet connection fails temporarily or the respondent forgets to copy the INTERVIEW link. If this happens in the current set up, wouldn’t it be impossible for the respondent to complete their started interview or even start over (if they click their original ASSIGNMENT link they will get the message that the link has expired)?

I am a little worried this might lead to a lower response rate if the interviews are not the top priority of respondents (in which case they are unlikely to contact the initiator of the survey).

Do you think it might be possible to build in this option in the future, and would you have any advice to reduce this risk in the current set up (besides the obvious of advising respondents to copy the INTERVIEW link at the start of the interview)?

Many thanks,

Sebastiaan

We are working on the improvement of CAWI in this sprint. Many, if not all, your requests will be addressed. We plan to release the new version with improved and extended CAWI functionality in early April 2019.

Ah, thank you Misha,

That will come too late for my current survey, but I think it will be very helpful.

I suppose that in the mean time a work around could be to start all individual assignments myself, copy the INTERVIEW links and send those instead of the ASSIGNMENT links to the respondents. Do you see any problem with that approach (except the amount of work it will entail)?

Thank you,

Sebastiaan

Dear Sebastiann,

Here is a brief summary of my last CAWI survey which may be helpful.

Our survey had about 2000 respondents for which we were hoping for 500 completed questionnaires. For each respondent, I created an interviewer AND generated a web link. My questionnaire was quite short, so I expected most interviews to be completed by the respondent in one sitting. However, each respondent had a user name and password in case they needed to access the interview again.

I automated emailing the links with instructions to all 2000 respondents using R and the RDCOMClient package. Then, I created weekly follow up emails using the API to check which interviews had been completed. I’d be happy to send you the script if you like.

In the end, only 1 respondent wrote me asking if they could re-access their interview. So, I just sent them their user name and password with instructions.

One thing to note is that you should use static text, and names of sections when possible to guide the respondent. For CAPI surveys, obviously you train enumerators, but for CAWI you don’t train respondents and the navigation is sometimes challenging. So, for example, we titled the first section, “Click here to Start”, because otherwise it’s not so clear they need to click the button to open a section. Also, at the end of the each section, we added static text , “Click the [insert name of section] below to continue.” The one thing that didn’t work well in the questionnaire was the roster. We had one roster, and less than 10 respondents filled it out correctly despite instructions in static text. Anyway, I’m hoping that there will be a grid option for rosters in CAWI survey which would definitely help.

Dear Misha,

Any highlights on the new coming CAWI features?

Best, Michael

Hi Michael,

Thanks very much for the extensive reply and offer of help! First on the rosters, there is a new feature with plain mode rosters that looks much simpler (especially useful for one-question rosters - which I have in my current questionnaire - and self-administered surveys). It was ideal for me, and came just in time, but I don’t think it supports the grid pattern you mention.

Although my questionnaire isn’t long and simple in structure (one section with questions, and I’m using one other section to include messages with static text with tips on how to complete the questionnaire), I do expect people to have breaks, so the fact they can continue later is vital for response (they are country experts who might have to consult others/look stuff up).

My plan was to create one interviewer for each respondent (with password) and create one assignment per interviewer/respondent. From what you say, I gather you did the same, but how would they access the interview again with their password if they paused it, and vitally, would the info entered in their previous session be stored? I didn’t think interviewers could login to the server to reset their assignments, can they? And the assignment link doesn’t require a password, but expires (as you also mentioned at the start of the thread)? Curious how you set it up.

Right now, as my sample frame is fairly small (200+), my plan is to start each interview myself, copy the interview links and mail those rather than the assignment links to the respondents. In my initial tests that allows them to re-enter with earlier answers stored. It is a bit of work, means that there are no survey solutions intro screen or welcome messages (will have to create those in the email), but has the advantage of bringing the respondents straight to the first question. I also thought the intro screen with section headers might be complicated and came up with that same trick you mentioned (“click here to start” :slight_smile: ), although I don’t think I’ll need it in my current set up.

Thanks again, very nice to think together about these things! What was your final response, did you make the 500?

Cheers,

Sebastiaan

Hi Sebastian,

Indeed, I set my survey up the same way (1 assignment & 1 interviewer per respondent). Regarding saving responses, this is automatic. Once the respondent enters a response, and moves to the next question, the response is saved. If they were to log off, or simply close the window, then they can just go back to the URL of the server,enter their credentials, and open up the interview to start where they left off. It doesn’t matter if they accessed the assignment first time w/ the link or not. For me, I sent them all the links, and then waited to see what happened. Only 1 or maybe 2 respondents wrote me asking how they can re-access the interview, and I sent them their login credentials w/ instructions. But again, my questionnaire was short, and subjective, so most interviews were completed in one sitting.

I would not start the interviews yourself b/c this will inhibit your ability to monitor which interviews have been started. Response rates for emailed surveys is notoriously low, and you’ll want to monitor which haven’t been started to send reminder emails. Opening each interview to look if additional responses have been entered would be a major pain. I tried script counting responses, and using the API, but counting responses is also tricky.

Note that the UI is not so intuitive for respondents. You need to provide them instructions which tells they have to select “started”, then right click on the assignment to open it. Otherwise, they’ll log on, not see the assignment immediately, and give up.

Finally, I highly recommend coming up w/ an automated way to send emails. If you’re handy w/ R, it can be done w/ the RDCOM package, and it can also be done using VBA and outlook.

Our final number of responses was actually 501.

Good luck! Michael

Hi Michael,

Good you met your target!

Ah, I see, you made them go through the UI. I would really like to avoid that, because it isn’t very intuitive as you say (just tried it again). I had planned to periodically download the data to see which respondents had started the interviews. Yes, a bit more work for me, but in my view worth it to make the process easier for respondents.

Thanks for the tip on generating the invitations automatically. I had planned to use a mail merge to generate them, but am a little tied down as the invitations have to be sent through the organising institution’s secretariat. My options depend on what they can do.

Thanks again!

Sebastiaan

Dear Sebastian,

I only made the ones that requested to access the interview for a second time go through the UI. For all the others, they accessed it using the unique link.

Good luck! Michael

Thanks for the clarification!