A practical guide to creating quizzes: part 3

Practical guidesIn part 1, we looked at the characteristics of online quizzes and explored how they could be used to assist or assess learning. In part 2, we explored the various question formats and the types of learning for which they are suited. In this part, we move on the writing of the questions, in particular the traps to avoid.
Question writers are faced with two tricky problems:

  1. They do not want to make the correct answers too obvious but, on the other hand, they don’t want to leave these answers open to challenge. So, they put an awful lot of care into how they phrase the right answers, to make them absolutely clear. In the process, they often give the answers away.
  2. They need to tempt the user with distractors (wrong answers) that are genuinely distracting. To be distracting, they must be plausible. The trouble is that plausible distractors – tempting but still unambiguously wrong – can be hard to come by, particularly when you’re testing knowledge of rules and principles. Still sure you want to write quiz questions?

We’ll explore both of these traps, with the help of some examples.

Obvious answers

In this first case, most learners will pick the fourth option. Why? Because it is the longest. That’s because the question writer will spend a disproportionate time making sure the correct answer is absolutely precise. As a result, it stands out. Also, the question writer was obviously struggling to come up with a plausible fourth option, which is why they threw in option 3 – an amusing throw-away, but certainly not a genuine distractor.

This second example is just carelessness. Only the word ‘elephant’ fits grammatically with the stem of the question.

Here’s another frequent mistake. The absolute answers (‘no men’ and ‘all men’) are clearly less likely than the softer ‘some men’. No learner is going to be fooled by these distractors.

Another lazy cop out is to use ‘none of the above’. It’s a way of telling the learner that you’re having trouble phrasing a correct answer. And option 3 is another throw-away.

Confusing questions

Learners don’t mind obvious answers, because it makes their job easier. But they certainly get annoyed if they are presented with questions that they don’t understand. Take these examples …

It’s bad enough having a negative in the question stem (“Which of the following is NOT …”), but a double negative such as “never unnecessary” makes the question really hard to fathom out.

There is no grammatical consistency in these options. Each one should be phrased in the same way and flow nicely from the question stem.

The problem here is that the learner could enter their answer in so many different ways: “CERN”, “CERN in Geneva”, “Switzerland”, even “Centre Européen de Recherches Nucléaires”. Text input questions are fine, but you must make it absolutely clear what sort of answer you are expecting, for example, “In what city was the research establishment where the World Wide Web was invented?” Even then you be prepared to accept both “Geneva” and “Genève”.

The problem here is that it is not clear whether you are looking for a single response or multiple responses. Experienced computer users will recognise that the use of check boxes implies you can pick any number of options, but that won’t be obvious to everyone. Better to say “You can choose more than one option.”
Chances are you won’t see your own obvious answers and confusing questions. You have no option really but to have your quiz questions tested by typical learners. Believe me, you will learn lots from what they have to tell you.
Coming in part 4: Quizzes as games