GPS points selectively dropped from subset of records

Hi SurveySolutions Team,

We are currently fielding a household survey that includes collecting agricultural plot outlines using the Geography polygon questions and the tablets’ location services. For a subset of our records (approximately 5%), only 3 GPS points are returned in the geometry column even though anywhere between 20-100 points have been collected (the variable automatically generated by survey solutions that tabulates the number of unique GPS points associated with a record as well as the map in the Headquarters view prove that the drawn polygon is based on many more than 3 points).

The image below exemplifies this problem, with 3 distinct GPS points being available in the download and in the Headquarters view despite 22 GPS points having been collected for this particular outline.

In other cases (95% of cases) we obtain the correct number of GPS points as calculated in the count column, even when they are in excess of 100, and we have not been able to discern some pattern between records in which we can retrieve the full GPS point listing versus only three.

We have ruled out the data export format as the reason, since we are exporting in the Tab format to avoid truncation issues that arise when exporting directly as .dta (i.e., character length limits when interviewers collected dozens of GPS points).

These are the versions we are using:

  • Interviewer app: version 21.09.5 with maps
  • Headquarters: version 21.09.5 (build 31862)

Do you have any recommendations for how to recover the complete string of GPS points for such records and how to prevent this issue from arising in future exports?

Many thanks,
Anthony

How comes your interviewer app is newer than the HQ?

After checking in with our field team, it turns out that Interviewer and Headquarters are on the same version (21.09.5).

One possibility is that the blue value 22 we see here is calculated using a different question due to copy/pasting from somewhere. Can’t be sure without seeing the questionnaire.

Can you reproduce the problem on the demo server with either the original or (preferably) a minimal questionnaire with just a geography question?

Hi Sergiy, we have added our questionnaire to the demo survey. It’s called “PLOT MEASUREMENT IN SK2/THE BASSE TERRASSE AND COMPARISON AREAS IN NIGER”.

The affected geography questions are K3, K9, and K15.

We think this issue is unlikely to be a copy/paste variable error because for the vast majority of our records the number of collected GPS data points matches the reported number, and the actual values are something like a random draw from a [20,100] distribution.

We wonder whether interviewers editing individual GPS points during the point of taking the outline might be the cause of the issue.

Dear @adagos,

We found an issue in tablet application code that produces this result.
If the polygon has self-intersections it produces incorrect result and depending on location of intersection the result could be truncated.
We are working on this issue.

Thank you @vitalii - that would be very helpful. On our end, regardless of the code fix, will emphasize to our interviewers to avoid self-intersections. One unescapable problem is GPS accuracy issues which can inadvertently create the self-intersection, but this might be exacerbated by them making manual fixes to vertex location on the screen after they’ve completed the polygon outline.

The precision of consumers tablets and hand drawing on the screen would produce very inaccurate results. You should not rely on them in any meaningful assumptions except the approximate location. Two different measurements with these tools would certainly produce overlaps and gaps for neighboring parcels. Several measurements of the same plot will produce different results on every measurement.

Thank you @vitalii - we spent much time deliberating as a team whether external GPS sensors would be worthwhile, but we are not doing cadastral mapping and so accuracy levels of 2-4m, while not ideal, are acceptable. [Purportedly] sub-meter accuracy rigs, like the Juniper Geode, are $1K+ each and we’ll have a dozen enumerators out in the field, making it pretty costly and adding to our data collection team’s overall risk exposure (let alone that USDA doesn’t observe sub-meter accuracy in their field testing anyway). Without the functionality of interviewers being able to continuously and automatically drop pins based on their location, we have to use a manual approach since the outlines we’re collecting are often extremely irregular, and can’t be properly approximated by just collecting corner points. Ultimately the outlines will be used with Sentinel-2 data, and so we’re always going to be constrained by S2’s 10m resolution.

On our side, we’ve reviewed the records and indeed found self-intersections in them. The 3 points that are displayed are the vertices of the resulting triangles.

Thanks again for your help on this.

Hi @adagos,
We’ve released a new version that addresses this issue on the tablets. In some cases if polygon is edited with zooming area (stronger press on the screen starts mode with better precision) the result of editing produces multipart polygon.
Coordinates should be handled now despite this behavior, but duplicates are possible in the array.

Hi @vitalii - thank you for the really quick turnaround on releasing that update – we are immensely appreciative. We will have the whole team update and will let you know if we encounter any issues or unexpected results. Many thanks.