In our case the first wave of data collection is done in CAPI mode, and the second wave is in CAWI mode. When an interviewer or respondent enters an incorrect email address, we end up creating an assignment with an invalid email.
The issue is that when we send CAWI interviews to the field, we don’t receive any notification that the email wasn’t delivered because the address doesn’t exist.
Does anyone have an idea of how we could detect or handle undelivered emails?
We’re using an SMTP server to send emails.
version: 24.06.6
the entered value is not an email in principle. For example, it doesn’t contain the @ character.
the entered value is not an address that is usable. For example, doesn’t exist. E.g. when interviewers misses a letter or adds an extra (enters JhnSmith@somesite.com instead of JohnSmith@somesite.com).
When dealing with emails, it is always recommended to apply validation to check for errors of the first kind. Hence the Designer will issue you a warning as soon as you have a question about email without a validation:
[WB0254]:Use function IsValidEmail() to validate email address.
Knowing if there is a problem of a second type is not possible without sending an email. And even if it is passing at the time of collection, the situation may change by the time the invitations are sent (e.g. the email account was deleted). Hence, you need to monitor your SMTP server’s non-delivery reports for finding out which emails were not delivered.
Yes, we are already using the IsValidEmail(). But still in our case there is like 3-5% of emails with wrong/misses letter.
And yes, that’s exactly what we’re currently doing — monitoring the SMTP server’s non-delivery reports. However, this approach is somewhat impractical, as I need to generate these reports manually every day after sending emails. When an email is sent to a non-existent address, the bounce response is returned to the server. I was wondering if Survey Solutions could possibly capture those bounce messages and display them in a list or dashboard within the system. That would greatly simplify monitoring and follow-up.
Just for example why I asking for that:
In some CAPI surveys, we instruct the household to complete certain parts via CAWI, so they are expecting an email. We usually create the assignment and send the email the following day, which is why it’s important for us to frequently check whether the email was delivered or not.
Perhaps that can be changed. Rather than acquiring the contact information from the respondent, deliver the contact information to the respondent.
if the CAWI part is independent from the CAPI part, you could prepare a blank assignment in advance, then pass the link to the respondent.
if they are not independent, then possibly use the CAPI-CAWI switch
Even from the emails that didn’t bounce, there could be a great deal of emails that are not being checked, or where they would have troubles to login, etc. Neither of the two relies on respondents’ emails.
If you are constrained with the current approach for whatever reason, consider switching to an SMTP server that does provide an API endpoint to track bounce-backs, and automate the process (perhaps call back to the household if you collect phone numbers and consider that information reliable, or send the interviewer for another visit), rather than rely on discipline of checking the reports manually.
Switching the SMTP server can be done during the operation, and will apply to all surveys in that workspace, but should otherwise be completely seamless.
We need to prefill some data like name of persons and CAPI is quite difficult to let respondents fill in directly, so we send only part of it to CAWI. I understand that idia but this is not a options for us.
Currently, we don’t have an SMTP server with an API endpoint to track bounce-backs, so we’re considering alternative solutions.
One option might be developing our own application for sending emails.
What I meant is that during CAPI, the interviewer is responsible for entering the answers, and they are trained to do so correctly. So in CAWI we use different question order and wording for some text questions, even though the questions are the same. To make it easier for the respondent to answer.
