Variable field not calculating

Dear Survey Team,

Could you kindly advise if you have encountered anything like the below.

We have encountered a peculiar issue with one of our pre-loaded Interviews. We have an Age variable field of type Long Integer. The Age variable field is calculated based on the current visit date which is a Date type question (Current Timestamp) and the DOB which is a Text type question (with Pattern ###*--). The date is captured manually and the DOB is pre-loaded with the option to correct if need be Below is the Expression for the Age variable field. This works perfectly fine for all our Interviews except this particular one that does not want to calculate the age for any of the members within the Interview members roster. I’ve even created a blank Interview based on the same questionnaire and recaptured the same data and this works fine, but for some reason this particular preloaded Interview does not want to calculate the age.

Fields

DOB → hm_civil_id_dob
Current Visit → current_visit
Age → var_mem_age =

Expression

!string.IsNullOrEmpty(hm_civil_id_dob) ?
(hm_civil_id_dob.Substring(8, 2).IsIntegerNumber() &&
hm_civil_id_dob.Substring(5, 2).IsIntegerNumber() &&
hm_civil_id_dob.Substring(0, 4).IsIntegerNumber() ? FullYearsBetween(Convert.ToDateTime(hm_civil_id_dob), current_visit) :
(hm_civil_id_dob.Substring(0, 4).IsIntegerNumber() && hm_civil_id_dob.Substring(5, 2).IsIntegerNumber() ?
(Int32.Parse(hm_civil_id_dob.Substring(5, 2)) >= Int32.Parse(current_visit.Value.Date.ToString(“yyyy’-‘MM’-'dd”).Substring(5, 2)) ?
((current_visit.Value.Year - Int32.Parse(hm_civil_id_dob.Substring(0, 4))) - 1) :
(current_visit.Value.Year - Int32.Parse(hm_civil_id_dob.Substring(0, 4)))) :
(hm_civil_id_dob.Substring(0, 4).IsIntegerNumber() ? (current_visit.Value.Year - Int32.Parse(hm_civil_id_dob.Substring(0, 4))) : 0)
)
) : 0

Kind regards

You wrote:

This works perfectly fine for all our Interviews except this particular one that does not want to calculate the age for any of the members within the Interview members roster.

Unfortunately nobody in this forum knows really what “this interview” is, since we don’t have it in front of our eyes.

If the problem is with one particular interview, could it be your preloading data is wrong? E.g. the birthday that you have on file is the 30th of February? (which happens more often then you, perhaps, imagined): see src1 or src2 for the gentleman in India, or src3 for a gentleman in Belarus.

Or it could be that your expectation with regard of the input data format is wrong, and whereas you expect the data to contain leading zeroes somewhere, it may not have it.

Avoid monster-like expressions like these. Break them up into several steps. If you run into a strange behavior - you will be able to inspect the intermediate results, to figure out at which steps something becomes wrong.

Dear Sergiy,

We have found 2 more Interviews that has the same problem with the blank variable value. I can confirm that our preload data is correct and there is nothing wrong with our DOB dates. I’ve uploaded the same file to the same questionnaire on our development server and the Age variable field for the same interviews is correctly displayed but on our production server the Age variable field is blank and no matter if we change any of the DOB values on the Interview, the age variable field is not updating. This happens to all the members within the members roster for these Interviews and not just some of them.

See below screen shots of the variable field on our questionnaire and the actual Interview on our Production and Development servers.

Questionnaire

Interview on Production
SS_Interview_Prod

Interview on Development

Kind regards
Eugene Ehlers

!string.IsNullOrEmpty(hm_civil_id_dob) ?
(hm_civil_id_dob.Substring(8, 2).IsIntegerNumber() &&
hm_civil_id_dob.Substring(5, 2).IsIntegerNumber() &&
hm_civil_id_dob.Substring(0, 4).IsIntegerNumber() ? FullYearsBetween(Convert.ToDateTime(hm_civil_id_dob), current_visit) :
(hm_civil_id_dob.Substring(0, 4).IsIntegerNumber() && hm_civil_id_dob.Substring(5, 2).IsIntegerNumber() ?
(Int32.Parse(hm_civil_id_dob.Substring(5, 2)) >= Int32.Parse(current_visit.Value.Date.ToString(“yyyy’-‘MM’-'dd”).Substring(5, 2)) ?
((current_visit.Value.Year - Int32.Parse(hm_civil_id_dob.Substring(0, 4))) - 1) :
(current_visit.Value.Year - Int32.Parse(hm_civil_id_dob.Substring(0, 4)))) :
(hm_civil_id_dob.Substring(0, 4).IsIntegerNumber() ? (current_visit.Value.Year - Int32.Parse(hm_civil_id_dob.Substring(0, 4))) : 0)
)
) : 0

What does Convert.ToDateTime(hm_civil_id_dob) do?
My guess is that it is culture dependent, and you will have a failure if the date is more than 12 and the culture is such that the format is yyyy-dd-mm (such as in Kazakhstan, for example)

  1. Make a test questionnaire with two fields: hm_civil_id_dob and current_visit.
  2. Reproduce the problem with this expression and input data.
  3. Refactor the expression to have all subexpressions as several variables (month_of_birth as number, etc)
  4. Display each such intermediate variable
  5. See where it fails.

Dear Sergiy,

As I mentioned, it is impossible for me to reproduce the scenario because I’ve already uploaded the exact same pre-load data to a copy of the exact questionnaire and on the copy the data is displayed correctly, so there seems to be no problem with the questionnaire / fields / expressions nor the data. The variable seems to sporadically choose not to be refreshed or update.

Kind regards
Eugene Ehlers

If anyone observes the same with a different questionnaire/server/version/data, please provide details.

Not sure if this is related, but one of our interviewer reported this:

One of the section in our questionnaire is to be filled only for female respondents. So when the gender is selected as Male, the section is not enabled. This interviewer reported seeing this section on 2 separate instances even when the selected gender was Male. But, closing the interview (this sends the interview to the started section) and starting it again fixed it (the section wasn’t enabled anymore) for her.

Likely a separate issue, but still important.

My reading of it is that the software has recorded the answer, but failed to redraw the new presentation of its state on the screen for some reason. Re-launching the interview made it repeat the rendering procedure and it succeeded. In other words, it is probably similar to this issue reported by @sebastiaanhesss :

Still, this should not be happening, as it may be sending the interviewers into the wrong trajectory. Perhaps would be interesting to see if the software permits any answers to be entered into the section that is supposedly locked, and if it does, please capture the tracing information from that tablet right away.

Best, Sergiy

We have found 26 Interviews thus far which all relates to the blank Age variable that is not being calculated. All these interviews where captured after we did our Survey Solutions upgrade from version 20.08 to our current version 21.05. It seems like somewhere in between version 20.08 and 21.05 something is causing this issue. All our Interviews which are on the same questionnaire that was in version 20.08 was all working fine, we never experienced this with version 20.08. I’ve also checked if I restart one of our Interviews with the missing age and this seems to have correctly calculate the age after I restarted the Interview, however, this is not a solution for us as there are many questions that need to be recaptured each time. We are busy updating to the latest version of Survey Solutions to see if there is any improvement but the issue definitely seems to be when we are capturing CAPI interviews using the Survey Solutions headquarters software for our call centre.

Further to my previous reply to ashwinikalantri above, If we export the Interview data with the missing Age variable data, we found the age not to be present in the export data either.

I see these possibilities:

  • the variable was never calculated (the calculation itself has not started for any reason);
  • the variable has been calculated, but the result of the calculation has not been saved for any reason;
  • the variable has been calculated, result saved, but then somehow overwritten with a blank value;
  • the calculation aborted with an error, and thus the result is missing.
  • etc

These are essentially different situations in terms of seeking where the problem may be.

None of the symptoms that you’ve described helps narrowing down to one of the above possibilities (they will all result in no value on the screen/in the export file, which you are insisting on).

I have asked you above to review the documentation for the Convert.ToDateTime() and hypothesized that it may be culture dependent. (to put it bluntly, I don’t mean your culture, I mean the C#'s class Culture and the setting CurrentCulture). In particular that would mean that the result of the function will be different on different tablets, and your attempts to reproduce the issue on your tablet will not be fruitful. Using it is intrinsically unsafe and should never be programmed that way. Here:

  • you should have had a hidden Date question DOB and preload it with a date value. Or at most preloaded three numeric values for the components of the date (YMD) and then construct a variable DOB of the DateTime class/type.
  • the acrobatics like Int32.Parse(current_visit.Value.Date.ToString(“yyyy’-‘MM’-'dd”).Substring(5, 2)) is beyond my comprehension, but in all probability you’ve meant current_visit.Value.Month which avoids serializing the date to string and trying to recover it from it back (both operations will be culture dependent).

This looks like a helpful message but it is not. We need to understand what happens in fact. Is it because YOU restart the interview (on your tablet, with your settings, etc) or is it because you RESTART the interview (a second opportunity, where the environment is different due to timing) Ambiguities like these give me hope that you may be right and there is something in there that needs fixing, otherwise I’d give up much sooner, and insist on simplifying the expression.

Very good. Out of 30? Or out of 30mln? If the former, then it should be relatively simple to reproduce manually. If the latter, then this 0.000… of 1 % It is not zero. And if this is an error it must be fixed. But this would mean that there is no sense in sitting with the tablet and keep on trying to provoke it.

This would be good to find out. Version 20.12 introduced the variables in the cover page, so this functionality may have been touched in the code.

Something similar also happened to me, some variables did not show the calculated value, but if I changed the section within the questionnaire and returned to the section where the variable was, it already showed the calculated value, apparently as if it occupied a refresh to be able to show the calculated value.

Hi Sergiy,

Apologies for the delay in reply but we are trying to resolve our interviews and trying different approaches but we did not have much luck in resolving this issue yet. I just need to clarify that our Interviewers are capturing these Interviews on their computer (call centre) and not on a tablet.

When I mentioned in my previous communication that the Interview was resolved after it was restarted, it was actually restarted by the exact same Interviewer who initially started the Interview and not by myself.

We also played around with the Culture settings and we concluded that all the computers are setup as South Africa (English) with the date format as yyyy/MM/dd. So if this was culture related the more or all Interviews should fail, which is not the case.

We do parse the DOB to a numeric question in our interviews which is used with the current visit date to calculate the age but we do not always have the exact DOB for an individual so we could sometime only have a year or a year and month only, so our DOB do sometime look like 1955-04-UU, hence why we have the Int32.Parse() in our expression to check if the full DOB should be used in the calculation or only the year and month or only the year. Also like I’ve mentioned in my previous communication, we have not had this issue on the previous version of Survey Solutions we were on. This only started after we upgrade from version 20.08 to version 21.05

We have found 56 Interviews out of ±1000 thus far which does not calculate the age in the Variable field. We have also tried to preload the age variable with the age at the beginning of the year and hoping that the variable expression would either update the variable with the newly calculated age or if the variable expression fails then at least leave the preloaded age untouched, but we have found that the variable is still updated with blank data in some Interviews. If the user restart the interview then the data is correctly updated for that same Interview.

Kind regards

Eugene

Hi Sergiy,

Is there any news or development on this issue? We are are receiving similar Interviews with age calculations that did not calculate on a daily basis.

Kind regards
Eugene

Please note the formula gives incorrect (negative) age for the following inputs:

image

I am also not sure what are the fancy quotes in the posted expression. It doesn’t compile as-is, so I had to remove some of the quotes characters from it.

Can you show the 3 variables current_visit, hm_civil_id_dob, and age? (exactly as they appear in the output of Survey Solutions! without any further transformation), perhaps as a CSV file?

Dear Sergiy,

I’ve uploaded a very shortened version of our questionnaire (Age Calculation) on the Survey Solutions Demo server. the age calculation is exactly as is in our production questionnaire. I’ve also made this questionnaire public for you to view. Let me know if you are having difficulties in viewing the questionnaire or if you need me to add you as a collaborator. I’ll email you the pre-loaded upload assignments. I’ve tested this on our local test server and the age calculations work as it should and there is a few records in the pre-load file that was giving us blank ages when we did the interviews on our production server (both our test and production server is on the same version 21.06.2)

Kind regards
Eugene

I could not reproduce the issue using

  • the questionnaire as posted by @eehlers to the demo server, and
  • the preloading sample that he shared privately, and
  • a PDS server v 21.06.3 (current at the time of this writing)

10 interviews were completed with 99 persons (99 age calculations). Here is the control check in Stata after the data was exported.

image

I suggest @eehlers inspects the paradata to verify the values that are assigned during the variable calculation. This is what we expect to see:

Dear Sergiy,

Thank you for confirming that my calculation code does indeed work correctly. As I have mentioned, whenever I try to replicate the issue it functions as intended, but in some random Interviews the calculation does not function as intended for an unknown reason. only after we restarted the Interview the calculation would work correctly. This never used to be a problem when we were on v20.08 of Survey Solutions. This only became an issue when we upgraded to v21.05.

Below is one of our Interviews on our production server which currently has this problem.

As you can see the visit date was recorded

As well as the DOB, but the Age calculation did not happen

This is the exported data for this individual

And this is the paradata

Kind regards
Eugene

I didn’t say that. As a matter of fact I have posted already an example of inputs, which are valid (based on your description) and for which the expression that you are using results in an incorrect (invalid, wrong) age (because the result is negative). See the post #15 above:

This has an impact on the quality of data that you are collecting, but should not have an impact on whether the result of the calculation is recorded by Survey Solutions or not.

All I confirmed is that for the data contained in the sample file you’ve shared the problem of age not being calculated was not observed using a clean server and contemporary version of the Survey Solutions software and a limited number of attempts.

Now, here is what we do know for sure:

  • Your server is different.
  • Your version of Survey Solutions installed on that server is different.
  • Your sample is different (because the person shown in the screenshot with DOB=1984-05-06 was not part of the sample you’ve shared)

We don’t know for sure, but wee can probably impute that something else is going on:

  • The screenshot shows that the preloaded date of birth was 1984-05-06, but in the exported data the date comes out with a different format: 1984/05/06. Why? I don’t know, but this casts doubts on the consistency of the screenshots.
  • The questionnaire that was posted to the demo server contained a calculation of an additional variable var_mem_age_str:
    image
    this calculation is present in the paradata stream that I obtained in my experiments, but not present in the screenshot you’ve sent. Which could mean the questionnaire is different and it just happens so that the difference is essential, even if this is not obvious.

Based on the paradata screenshot the var_reg_mem_age is assigned an empty value. I interpret this as the case when an exception occurs during the evaluation of your expression. This may be caused by the presence of an invisible character in the DOB, which get’s truncated if you modify the answer. (strings are trimmed of the leading and trailing spaces), but until then gets your data misaligned as per hardwired positions in the string that you expect.

If you are 100% sure and conclude this is not the case, then I suggest the following:

a) upgrade to the newest version in which I wasn’t able to reproduce the issue. If it does not happen again, then we will know that there was a subtle bug in v21.05, which appeared for an unknown reason, and was equally quietly resolved, perhaps as part of numerous other changes and improvements. If it continues to happen again, then we will know that the version difference can be eliminated from the list of suspects above.

b) take a pen and paper and ‘replay’ on that paper all the changes to all the questions/variables according to the paradata stream before the empty assignment occurs (That is 2273 events if you start from the beginning, perhaps, for efficiency you can move from the end, just seeking the last posted values.) That should allow to recover reliably the values of all factors involved in your calculation formula.

c) find a volunteer in this forum (ideally an experienced user) that can reproduce the problem from scratch using your questionnaire/data/instructions on her server. Once you are successful with this, we can follow the same steps in our controlled environment to see what is happening.

d) rewrite the expression to not use the serialization/de-serialization features, which are vulnerable to culture settings. If the new version is free of this defect, then we will know it was somehow related to the original expression, but in what way - we don’t know.

Others may come up with more suggestions.

Dear Sergiy,

Thank you for your suggestions, We will update to v21.06.3 and take it from there.

Just to answer on your remarks in your previous post.

We handle negative ages appropriately through validations but at least there is a value present, even if it is negative. In some of our Interviews as I’ve shown through my previous posts there is no age value at all.

The reason for the difference in date format is because Microsoft Excel auto change this to the Date format yyyy/mm/dd. If I open the export file with a plain text editor like notepad, then the DOB is displayed as yyyy-mm-dd .

The screen shot was just an example of the first available Interview with similar issues that I located. This data I shared was captured in a different section of the same questionnaire that records members in new households but the age calculations is the same as the section on existing household members which is the variables that I shared with you in the questionnaire on the Demo server. The variables in the new household section is just named differently as outlined in red in my previous paradata screen shot

var_mem_age → var_reg_mem_age
var_mem_age_str → var_reg_mem_age_str

I would agree with you but why would the age be calculated correctly when we restart the Interview? The pre-loaded DOB is still the same (untouched) after we restart the Interview. Also I find it odd that all these Interviews with missing ages would only appear after we upgraded. Our pre-load scripts generations has not changed between the two versions so would we not have had invisible characters in the DOB on the previous version of Survey Solutions and if so why did the age calculation still work then?

I’ll work through your suggestions and try and narrow down the issue.

Kind regards
Eugene