Nutrition Smart Survey in Malawi

Around the World
1

Just came across a report “NUTRITION SMART SURVEYS CONDUCTED IN FLOOD- AND DROUGHT-PRONE LIVELIHOOD ZONES OF MALAWI”.

The report covers a lot of practical recommendations that were followed by the interviewers to safely conduct face-to-face interviews using Survey Solutions software on the tablets during the Covid-19 pandemics.

What puzzled me somewhat is the response rate above 100% for some administrative areas (Table 3.1 on p.32). And the same figures are also mentioned in the main text of the report. The definition that comes to my mind is aligned with “What is a survey response rate?”:

The survey response rate is the number of people in your sample who successfully completed your survey. Theoretically, it could be anywhere from 0% to 100%, although it’s almost unheard of to get a 100% response rate.

If anyone can comment on how exactly those numbers were calculated, please do.

2 Likes
2

Hi Sergiy, interesting report and intriguing response-rate question. On a side-note, its good to see more country NSOs using Survey Solutions as their primary data collection tool.

On to the response-rate question. In practice, we usually define response rate as the proportion of sampled units who successfully completed our survey.

Malawi seems to have followed this definition when calculating household response rates*, but not when calculating individual response rates. When calculating individual “response rates”, they seem to be reporting what we can call “target rates”. Although the report seems refer to a “sample” of individuals, it was really a target. For example, in page 20 they determine their target of adolescents per livelihood zone and per cluster

An average of 13 adolescents per cluster was used in each livelihood zone.

and then they appear to calculate how many of the sampled households (17/cluster) should be sub-sampled in order to meet the target (table 2.3).

However, later on they explain in page 22 that

In the selected households, all eligible children (aged 6-59 months) were measured and the household questionnaire applied

and

For adolescent and adult surveys, as well as the mortality survey, all eligible participants were assessed, and respective questionnaires administered.

Therefore, at the individual level, they are reporting the ratio of successful respondents to their pre-determined target (respondents/target-size).This is not what we typically refer to as a “response rate”.

They could have reported response rates at the individual level had they used the formula (total respondents / eligible respondents).

*Note: Apparent household-level sampling method:

  1. The number of households (N) for each “livelihood zone” was determined (using a variety of indicators and selecting the maximum size. See section 2 and tables 2.1 to 2.4).
  2. They established that 17 households would be selected per cluster.
  3. The number of clusters to sample were determined (N/17) and subsequently sampled using probability proportional to size (PPS) sampling.

After that, page 22 states

The 17 required number of households per cluster was selected using simple random sampling in MS Excel prior to data collection.

In page 32, they correctly reported household response rate according to the definitions we have shared in this thread.

1 Like