For a pilot survey there is a single categorical question “Can the interview be done?” with several codes for not conducted interviews (refused, no one at home, etc.).
I exposed this question variable and filtered by dynamic filtering for a single code (“unable to respond”). It returns these interviews but also some interviews with a different code (“accepted the interview”).
This happens for other codes as well.
I discovered this because I did this filtering in a graphQL query which gave me the same wrong results.
I tried to reproduce it on the demo server with a simple example and a few interviews, but failed. Probably more interviews are necessary (in the pilot there are 1100), and certain conditions (unknown to me) must be present which cause a wrong interview to be included.
This problem also has the following effect (that’s how I discovered that there is a problem):
Interviews for code 23: 64
Interviews for code 24: 33
Interviews for OR(code 23, code 24): 95
This should be 64 + 33 = 97
This is because the wrongly selected interviews are returned in the first and the second filter, but in the union of the two filters they appear of course only once.
(I checked if the wrongly selected interviews had a clientkey which was a duplicate of another interview key, but this is not the case.)
This happens on a server (version 22.02.5) which is not mine, so I cannot give you access to the server, but I could demonstrate it to you.
Is this question inside a roster, perhaps?
Such as not the interview result, but the visit result?
No, it’s on the top level, right at the beginning.
if I understand you correctly then there are two interviews that are being selected as part of each of the above? is there anything specific about those interviews? can you locate them? Inspect them? how are they different from the rest? What is the actual code? (23? 24? one 23 and one 24? )
What is the exact GraphQL query that you’ve submitted?
Does anyone else experience a similar issue?
Yes, some interviews are selected by both codes. I can inspect them and did so for one: it has a code of 11 (= accepted interview), so neither 23 or 24. I cannot see anything irregular with it.
And there are several such wrongly selected interviews. Out of 556 interviews 61 are wrongly selected:
If you have 3 minutes, I can demonstrate the problem to you.
Could you please check if the paradata for this question mentions the answer was a different value (e.g. 23 or 24) at some point and then changed, perhaps to 11?
There is an issue in the code and it will be fixed and released soon.
@sergiy, you are right, the answer changed several times:
@vitalii, good to hear you have found the problem.
Thank you for reporting this problem! It has been fixed in v20.06.4.32762:
- For new installations the issue will not appear.
- For existing installations, if you’ve been using exposed variables earlier, you need to change the set of exposed variables after upgrading the server software. You can do this by removing any variable from the set of exposed variables, and then subsequently adding it back.
Great. Thank you for fixing it so quickly.
During my tests I also discovered a kind of undocumented feature:
While in the Dynamic Rules for an exposed categorical variable there are only 4 operations:
equals, not equals, answered, not answered.
an explicit graphQL query also accepts:
gt, gte, lt, lte.