Use reusable categories as source for fixed rosters

[arthurshaw2002 (Arthur Shaw)]

Proposal

Designer allows answer options to be drawn from reusable categories, but this is not true for fixed rosters.

Affected subsystems

This should affect only Designer.

Users

While this feature may be generally useful, I think the greatest utility would be for agricultural surveys where:

1. Capture main crop on a plot
2. Capture all crops grown on a plot
3. Capture total production for each crop

The first two could clearly use a reusable category. The last–a fixed roster–should use the same reusable category, but cannot, since there’s no mechanism in Designer to designate a reusable cateogory as the source.

Surveys

This is useful for most agricultural surveys or surveys with a strong agricultural component.

A few surveys that may benefit from this:

• Enquête Harmonisée sur les Conditions de Vie des Ménages (EHCVM). The first wave goes to the field in a week or so for many countries. The second wave will begin in April 2022.
• Nigeria Genernal Household Survey (GHS), an LSMS survey, scheduled for 2022.

Alternatives

The current work-around is to populate fixed rosters by copying and pasting the items in reusable categories for crops.

Market examples

This type of feature is available in:

• ODK (and its forks)
• Surveybe

(Links to follow)

What is the effect of this feature?

It would have two effects:

1. Make easier the creation of fixed rosters of this type
2. Reduce the scope for errors that arise from failing to update fixed roster sources to match changes in the crop list used in reusable categories

Compatibility

This should not imply any breaking changes.

Applicability

This feature would benefit most agricultural surveys or surveys with a strong agricultural component.

Does it make sense?

Limits

No limits or conflicts

[klaus (Klaus Blass)]

Arthur, maybe I misunderstand your use case:

@arthurshaw2002

1 Capture main crop on a plot
2 Capture all crops grown on a plot
3 Capture total production for each crop

But I wouldn’t use a fixed items roster for 3, instead I’d let it refer to the multi-select question in 2.
Does this make sense?

[sergiy]

I agree with @klaus , it makes sense to ask about total production of only the crops that are being cultivated, hence the production should be a roster triggered by the multiselect ‘crops cultivated’ question. So the example shown by @arthurshaw2002 is not perfect. Yet the need for the fixed roster to make use of reusable categories is indeed practical. Consider, for example, a survey where you ask about various activities: sports, entertainment, shopping; each is a separate section. And you wish to inquire about their frequency by days of week (fixed roster). In that case it makes sense to re-use the same list of days of the week (months of the year, etc). In enterprise surveys you commonly encounter repeated rosters with types of workers (permanent, temporary, replacement, consultants, etc) for various activities.

On the compatibility - this is clearly a change that will have compatibility constraints: the old HQs will not be able to ‘play’ the new questionnaires.

As formulated, the feature appears to be only of interest to the designers of questionnaires. But there is one implication for export. Currently Survey Solutions does stitch together the rosters triggered by the same trigger, but since there is no trigger for the fixed rosters - they stay aside from the stitching process. While for the above example with days of week, it would probably make sense to combine. The traditional way of solving this is to make explicit the concept of a data source with its own properties, kind of what is done in Delphi:

We have discussed this approach at the time the rosters were introduced and revisited it on several occasions, but introducing this abstract concept was deemed to be too complex for an average user.

@arthurshaw2002 , could you please show an example of an ODK questionnaire that uses reusable categories for 2 or more fixed rosters? (xlsForm) thank you.

[arthurshaw2002 (Arthur Shaw)]

A few quick reactions below. I’ll try to link to provide links to how other software handles this later (and retract my claims if I’m wrong).

Questionnaire setup

@sergiy

I agree with @klaus , it makes sense to ask about total production of only the crops that are being cultivated, hence the production should be a roster triggered by the multiselect ‘crops cultivated’ question.

The setup is as follows:

• Crop roster nested in parcel and plot rosters. This crop roster is triggered by a multi-select question.
• Crop roster for total production and disposition. This roster is fixed roster with the same items as in the first bullet. The relevant roster rows are enabled for crops that appeared on at least one plot.

Please find below a few screenshots and links. Where possible, I’m pointing to a public version of the last EHCVM. Where not possible, I’m taking screenshots of the current EHCVM that is being finalized. Throughout, for lack of a better readily available example, everything is in French.

Crops on each plot

Link to relevant segment in public questionnaire.

Crop use/disposition

Enablement condition to enable those crops grown on at least one plot

// sélectionner les champs, toutes leurs parcelles, et toutes leurs cultures
champs.SelectMany(x => x.parcelles).SelectMany(y => y.cultures)
// où la culture a été totalement ou partiellement récoltée
.Where(z => z.s16cq11 == 1 || z.s16cq12 < 100).
// prendre ces identifiants et voir si l'identiant en cours est parmi eux
Select(a=>a.@rowcode).Contains(@rowcode)

Note: champs = parcels; parcelles = plots; culture = crops

Feature implementation

@sergiy

Currently Survey Solutions does stitch together the rosters triggered by the same trigger, but since there is no trigger for the fixed rosters - they stay aside from the stitching process. While for the above example with days of week, it would probably make sense to combine. The traditional way of solving this is to make explicit the concept of a data source with its own properties, kind of what is done in Delphi:

Personally, I would be content if fixed rosters continued not to be stitched. My only aim is to have a cleaner way to help keep reused categories in sync across the various places they are used in the questionnaire.

[sergiy]

Hello Arthur, agree, if your total crops collection is being constructed as a aggregation of crops grown on multiple individual plots, then using a fixed roster is the only tactic I see for handling this. Yet you will not be able to create a fixed roster with a realistic number of crops varieties (you will be bound by a 200 or so limit), which may be fine for major commercial crops, but not enough to get the full assortment.

You could restructure your Qx by asking:
Q1: Which crops do you grow on ANY of your plots? (multiselect up to 200 from up to 15,000).
Q2: Which (of the selected in Q1) do you grow on plot [PLOTNAME]? (filtered multiselect with up to 200 from 15,000)

This would put the burden of aggregation on the respondent in Q1, something which I do not see as desirable, but should allow you to

• subsequently use that question as a trigger for the crops roster without an additional condition.
• capture a wider range of crop varieties.

[arthurshaw2002 (Arthur Shaw)]

@sergiy

Yet you will not be able to create a fixed roster with a realistic number of crops varieties (you will be bound by a 200 or so limit), which may be fine for major commercial crops, but not enough to get the full assortment.

This is a good point in principle.

For the ag surveys with which I’ve been involved, this has not been a problem. This seems to be true for some or all of these reasons:

1. Number of crops (of interest) is reasonably small for the survey area
2. No desire/need to report production at the crop variety level (or no presumed ability to report on the part of respondents)
3. Practice of splitting survey by type of crop: annual or permanent. This reduces the crop list for any given crop type, since the crop list is not a master crop list.

But for ag censuses or surveys desirous of reporting at the crop variety level, this may indeed be a concern for the practical reasons you outline.