API request to retrieve the Paradata for a single interview

Proposal

Add an API request to retrieve the Paradata for a single interview.

Rationale

Paradata contain exclusive information about an interview which under certain circumstances is extremely important. (Example: has the answer to a sensitive question like nationality been deleted/changed?).

Currently Paradata can be exported through the API, resulting in huge files taking up lots of server space and taking a long time to generate and download.
If I need Paradata only for a specific interview I can limit the export request to a time interval just containing a point in time from this interview. But in a census with 10,000+ interviews/day the request will still return many hundreds or even thousands of interviews, resulting in a large overhead (time and space).

Introducing an API request for returning the paradata of a single interview would eliminate this overhead.

Use Case

This is a real world example. For the population census in a multi-ethnic country A, whose territory is also claimed by another country B, there is a strong preoccupation (by the NSO and the World Bank) that for interviewed citizens of ethnicity and language of country B this information will be correctly recorded in the interview, but later changed to A.
Such irregularities have to be detected immediately in order to take appropriate corrective action.

Affected subsystems

Only the new API endpoint, or a modification of the existing paradata export request.

Users

HQ users monitoring an ongoing survey / API implementers.

Surveys

Population Census

Alternatives

Export a potentially large set of (irrelevant) interview paradata.

What is the effect of this feature?

Reducing the cost (both time and server space) of downloading paradata.

Compatibility

No effect on users who don’t need this API request.

Applicability

Applicable for situations where only the paradata of specific (suspicious) interviews need to be analyzed.

Limits

Doesn’t affect any limits.

Observation

There exists another feature request for the same functionality, although for a different use case:

How do you identify specific interview? are you able to ‘guess’ what interview(s) must be checked or will end up needing to loop over list of interviews and requesting/checking paradata content for each?

I have to get a list of completed interviews, analyze several aspects for each one,
and identify (‘guess’) a couple of suspicious ones.
Only for those I want to fetch the paradata.

there is an endpoint returning interview history at api/v1/interviews/id/history. Currently it is a bit of a hybrid of outputting a bit more than what users see on the interview history screen from the GUI and less than what is in the exported paradata. Perhaps this endpoint can be extended to cover the use case, maybe with a parameter of returning ‘all’ or only ‘some’ items to serve the both needs of more or less-detailed output.

Hello @klaus , if this is a real World Bank project, please have the TTL to contact me to supply some more details. Thank you, Sergiy