8 8.0 Creating Self-quizzes

Many adult learners have unhappy memories of taking tests. For many it has been a long time since they put pencil to paper for a formal evaluation. Giving them a good experience with the end of chapter self-quizzes is an important aspect of the job of the Committee Chair.

Open-ended questions where the student can write lists can be helpful. Students usually come up with some unusual items, and when discussed in class, they can add other items contributed by their classmates.

Well-designed examinations are very difficult to prepare.

Some examiners favour multiple-choice questions. These certainly are easy to mark but difficult for the student to demonstrate original thoughts. Anything that is in the least ambiguous can be easily misunderstood. It is this fact that makes examinations difficult to prepare, and ambiguity must be avoided at all costs. Students frequently ‘second guess’ the meaning of questions. However, well-constructed, multiple-choice questions can also test the student’s understanding.

8.1   Examination Questions

Before creating an examination, it is necessary to outline all of the examinable topics and subjects that are presented in the CPS-ECP material. Review your outline and select the most important items for examination questions. When all of your questions have been written, reviewed, and completed, group them according to major topics. Assign a weight or value to each topic. Then select questions from each group so that the selected questions have a combined total value of 100% according to your assigned weighting.

Ensure a consistent level of difficulty among the various weightings.

Good questions are essential to the learning process. Phrasing a question to produce a single correct answer is the greatest challenge of writing exam questions. These guidelines will often seem to be common sense, but review the following suggestions before you start to write your questions. Writing questions is a challenge because what seems clear to the writer may not be clear to the reader. Review questions critically and then submit them to committee members to get another opinion. When the examination questions have been completed, the exam is submitted to the Exam Review Committee. Refer to Section 2. 2.

A good examination gives students an opportunity to demonstrate the knowledge, concepts and skills they have acquired. Students should be able to demonstrate that they have done more than just memorize content and should be able to reason and apply what they have learned to new circumstances. The examination indicates whether students have met the required standards. Feedback of the exam results provides information about the course material, the instruction and the examination itself.

A good examination should be a learning process within itself. For example, homework questions should be designed to encourage the student to think critically about the topic under study. It is also useful to have a few homework questions cover work completed in previous lessons (even three or four weeks ago). This encourages constant revision.

Questions should concentrate on material the students reasonably should be expected to know without having to look up at a later occasion. Several types of questions are available for use. These include: multiple-choice, short-answer and calculations. Some rules apply to all of these questions.

• Be specific and concise.

• Use simple wording, proper grammar, punctuation, syntax and spelling.

• Break long statements into shorter sentences.

• Use the vocabulary that is used in the CPS-ECP material.

• Avoid negative questions and double negatives.

• Create questions from the course material and not from outside sources.

• Do not use material which is non-examinable (refer to Section 6.7).

8.2   Multiple-choice Questions

Multiple-choice questions consist of two parts; the question or stem and the possible responses or distractors. The student must select the proper answer from the distractors presented. When creating questions, you may want to frame a correct statement and then divide this into two parts. The first part can be used as the stem and the last part as the correct distractor. Two styles of multiple-choice are the question and incomplete statement.

In the question format the stem ends with a question mark. Each option is identified with a capital letter and does not have a period at the end of the option. The options are possible answers and not part of a sentence. One option will be the correct answer and the others are incorrect distractors.

Example:

Stem:

Line made from which of the following materials will float?

Options:
A.  dacron
B.  hemp
C.  nylon
D.  polypropylene

A stem which is in an incomplete format ends with a colon. Each option choice starts with a lower case letter, unless it is a proper name or is a term which would normally be capitalized. Each option ends with a period because it completes the sentence begun by the stem.

Example:

Stem:

The boat’s log and plot should be kept according to:

Options:
A.  daylight savings time.
B.  ship’s time.
C.  universal time.
D.  zone time.

To write an effective multiple-choice question, begin by writing the stem, followed by the correct answer, and then the distractors. Some suggestions for writing the stems are:

• Be specific about what you are asking.

• State the problem precisely and avoid ambiguity.

• Avoid ‘trick’ questions.

• Place most of the subject matter in the stem.

• Avoid words that ask for personal opinions rather than facts (best, least, most, might, could).

• Do not write stems that point directly to the answer. Avoid key words in the stem and the options that provide students with clues to the correct answer.

• Questions should not contain extraneous statements that will cause a well-informed student to read more into the question than you intended.

• Words which are used in the stem should not be repeated in the options.

Composing multiple-choice questions is challenging since they require a clear statement or question and one unambiguous correct answer.

It takes four good options to turn a stem into an effective multiple-choice question. Three options (the distractors) must be reasonable, but totally incorrect. The correct answer should not be obvious because the distractors are obviously wrong.

Use of  ‘all of the above’, ‘none of the above’, should be avoided and use of ‘choices a and b’, or ‘choices a and c’ etc. should be kept to a minimum.

Some suggestions for writing distractors are:

• Ensure a similarity between the alternate answers in grammatical structure, content, length, and mode of expression (the longest answer option is quite often the correct answer).

• Ensure that the tenses are consistent throughout.

• Arrange numerical answers in ascending or descending order.

• For options that end with numerals, don’t use a period. Students may confuse this with a decimal point and infer a degree of accuracy that you do not intend.

In preparing a set of questions, be conscious of the distribution of answers and avoid patterns. This can be avoided by writing the entire set of questions and answers and then placing the answers in alphabetical order. Correct responses should then to be distributed nearly equally among the questions. This practice also makes the exam paper more aesthetically pleasing. There should be a maximum of three identical responses in a row. In addition, be cautious of using questions in a series so that the response to one question provides a clue to the correct answer in another question.

8.3   Short-answer Questions

If you use short-answer questions, make sure that each question is clear and specific and relates to one of the major topics. Indicate the extent and depth of answer that you expect by providing the marks awarded to the question. In addition, on the answer sheet, supply a specific number of lines or space required to completely answer the question.

8.3.1  Calculations 

Questions involving computations should emphasize the student’s ability to solve the problem, not on mathematical knowledge. For example, it may be necessary to evaluate the student’s ability to perform an unusual mathematical procedure. Design a question to evaluate the student’s ability to perform the required task. All other questions should avoid the use of such skills and be confined to testing the principle involved. Take any data required directly from tables so that distances, course, bearings, etc. can be easily recorded within the standards of accuracy.

Keep multi-part computations to a minimum. Problems occur when a mistake is made on one part and carries over into the following questions. Reducing carry-over problems will ease the burden on the student and examiner. This can be achieved by having a break in the line of questioning and starting afresh. For example, plotting a course that has several route changes will be cumbersome if a mistake is made in the early parts of the problem, since all following results will be incorrect.

8.3.2  Review the Examination 

Review your prospective examination to evaluate the following questions:

• Is the list of topics fair to the student? Are there any ‘trick’ questions?

• Are there any questions based on non-examinable material? (Refer to Section 6.7.)

• Does it cover all the material the students should know?

• Should a picture or diagram be used in some cases to illustrate a point?