Dear all,
I am currently preparing to conduct a choice experiment via survey solutions. Let’s say we have 20 binary single-select questions, each with a unique attachent for illustration (choice cards). Each respondent needs to answer 8 randomly selected questions out of these 20 in a random order. This means I need to present interviewees with a random selection of questions WITHOUT replacement.
I found this public example with random subsections rather appealing. However, the featured syntax works with replacement. This means by chance, respondents might be asked the same question twice.
The public example of Randomized question wording does not work either, because what varies is not the question but the attachment (the choice card).
Anyone ever faced a similar problem? How to properly integrate a choice experiment in survey solutions if we don’t want to use printouts for practical reasons? Any suggestions are highly appreciated.
Lena
Sorry if I used an incorrect term, I’ll try again: Drawing a random number more than once means that the same number can be drawn more than once. In other words, the same question can be asked more then once. What would be the correct syntax to draw a random number between 1-8 four times, each time removing that number from the list so that each number can be only drawn once, i.e. each question can’t be asked more than once?
Allow me to disagree with the “other word”: It all depends what exactly you do with that number to determine which question to ask. Here is an example:
Suppose you have a sequence of questions: Q1, Q2, Q3, Q4, Q5, Q6, Q7, Q8 and you draw random numbers which could be either zeroes or ones only, for example 0, 1, 0, 0, 0, 1, 1, 0.
Once having a random number you apply the following rule:
if that is a zero, you show the first question from the remaining ones in your list;
if that is a one, you show the last question from the remaining ones in your list.
So, for this random sequence: 0, 1, 0, 0, 0, 1, 1, 0
the sequence of questions will be: Q1, Q8, Q2, Q3, Q4, Q7, Q6, Q5,
and for this random sequence: 1, 1, 1, 1, 0, 0, 0, 1
the questions will be: Q8, Q7, Q6, Q5, Q1, Q2, Q3, Q4.
Note that if you are pulling more than 2 random numbers, of which each can only be a zero or a one, there are bound to be duplicates in the random sequence, but the subsequent application of the rule: taking “from the remaining ones” guarantees the same question will not be asked twice. Clearly, one can come up with multiple different strategies how to convert repetitive random values into distinct choices from a fixed choice set. The one described above is not particularly a good one, the questions closer to the beginning and to the end of the list have a higher probability of occurring in the beginning of your sequence (if I am not mistaken).
The C# language manuals go in lengths about different scenarios of the usage of randomness, but the relevant section for your case is probably “Retrieve a unique element from an array or collection”.
I also think that you are changing the question on the fly. First you’ve asked for 8 random cards from a set of 20. Now it is 4 random numbers between 1 and 8. And we have no idea what are the questions being asked, whether they are the same questions, but with different options, or same options, but different questions, or same questions and options and different cards, or what is on those cards? are they options? Single options? (Do you like this car Y/N?) or multiple options (Which of these cars would you prefer? single-select). If the latter, are these cards ‘pre-mixed’? or should the program mix them on the fly? The solution may be very much different depending on these parameters.
Be very specific about your ‘choice experiment’. Here is an example:
Dear Segiy,
Thanks a lot, I will check the indicated section of the manual.
As of the experiment, sorry for the confusion. Clearly we are in a learning process here, thanks for your patience.
The ultimate design is: Respondents have to answer two blocks of six single-select categorical question (1= Option A, 2=Option B, 3= opt out). Each of the questions comes with an attachment, the choice card. The options compared in the choice card are pre-mixed (done outside Survey Solutions with NGene). Neither the question nor the answer options vary, only the attachment does.
To my understanding, if I want to present the choice card as attachment (and not a as a printed hardcopy), I need to randomize the sequence of the 2*6 questions as I haven’t found a way to randomize the attachment only.
Actually randomizing the attachments is exactly the way to move forward here. A few years ago (in version 20.08) we’ve added a feature called “flexible images”, which is helpful exactly when you need to display different images based on different conditions that you determine dynamically (in your case randomly) within each interview.
I am not sure about which time zone you are in, but I can do a quick call on this Thursday (August 03, 2023) 10am (Washington DC time zone) to discuss. Let me know if this works.
Dear Sergiy, the presented example is just what I need. I tried with the info from the announcement. But I am struggling with the implementation as I have no programming background but can only modify existing code. Would be great if you could share a public example on this. Otherwise, of course a call would be very appreciated. Will you pm me contact details?
The following questionnaire “Choice Experiment” has been added to the collection of public example questionnaires to illustrate how such a choice experiment can be implemented in Survey Solutions.
Dear Sergiy, thanks so much for your help and this great public example. It worked very well for our purposes and I can just recommend it to everyone trying to implement choice experiments in survey solutions.
Best
Lena
