External validation system which verifies collected data against some external sources of information and rejects automatically the incorrect interviews

Hi there! I am new to survey solutions and would like to use it for my project. What I am trying to achieve is basically be able to verify collected data (eg. unique id of interviewee) against an external API. Assuming that every interviewee knows their unique ID, I want to only collect data for people present in my other db and reject it otherwise. I have an API that responds with true/false when the unique ID exists/not.

A case example will help as I learn how to perform validations.

This sounds super cool! Have you already decided what language you will use to implement your external validation system?

1 Like

Thanks for getting back to me @sergiy. I would like to use python and I am flexible about that.

A good starting point would be installing an API package for python written by Zurab Sajaia and using it to download the data from the server, then load the person IDs to the memory. The rest is up to your reference DB, from which you can inquire whether each ID is known or not, and reject the interviews if unknown.

Is it possible to validate before submitting from the interviewer app?

No, you cannot ‘not receive’ information from your respondents, be it from the mobile application, or web submission.
But you can automatically reject or move those not in your ‘expected list’. You can have a virtual interviewer (name it ‘deleted’, ‘trash’, etc), so all the incoming interviews that you do not want to keep can be reassigned to that user, and therefore not be visible to your ‘real’ supervisors/interviewers…

Headquarters doesn’t have webhooks or cannot otherwise call external service, so what you’d do is to poll the Interviews endpoint every x minutes and check if there are any new interviews; run the validation; reject/reassign.

1 Like

The Interviewer App on the tablet must be able to work autonomously. So no, cannot call your online API. Do it from the server.

1 Like

Thank you so much for your help.

A few more leads.

  • Similar projects. I’ve done something like this in the past via some R scripts. The idea is to: download the data, identify data problems, and reject interviews with problems, posting a summary of the issue(s) that merited rejection. The mechanisms were use of the Survey Solutions API for communicating with the server and some R and Stata scripts for identifying errors and moving from errors to rejection messages. Check it out here. (Sorry, my only public repo of this type is in French.) I’m planning on reviving this project soon and, if time allows, rewriting the Stata part in R and “locking” the R environment for replicability. If this takes the form of another repo, I can post the link here, if it’s of interest.
  • Other Survey Solutions API wrappers. Here’s one I’ve created in R. Zurab’s Python library probably has better coverage of API endpoints. But I think mine might have the best coverage of R API wrappers that I know. Here’s a sister package that executes common workflows–typically combinations of API calls–in one or two functions.
1 Like