Validating test questions
I am a Power Shell fanatic currently working in Windows client management at Haworth based out of Holland, MI.
I tinker around and document my findings at tiberriver256.and you can find me on Twitter @tiberriver256.
Multiple choice test items are less susceptible to guessing than true/false questions, making them a more reliable means of assessment.
The reliability is enhanced when the number of MC items focused on a single learning objective is increased.
Because students can typically answer a multiple choice item much more quickly than an essay question, tests based on multiple choice items can typically focus on a relatively broad representation of course material, thus increasing the validity of the assessment.
The key to taking advantage of these strengths, however, is construction of good multiple choice items.
In the example below, the script can't create the folder if the user enters illegal characters for folder names.
The alternatives consist of one correct or best alternative, which is the answer, and incorrect or inferior alternatives, known as distractors. A stem that does not present a clear problem, however, may test students’ ability to draw inferences from vague descriptions rather serving as a more direct test of students’ achievement of the learning outcome. Students often have difficulty understanding items with negative phrasing (Rodriguez 1997).In constructing multiple choice items to test higher order thinking, it can also be helpful to design problems that require multilogical thinking, where multilogical thinking is defined as “thinking that requires knowledge of more than one fact to logically and systematically apply concepts to a …problem” (Morrison and Free, 2001, page 20).Finally, designing alternatives that require a high level of discrimination can also contribute to multiple choice items that test higher-order thinking.When writing multiple choice items to test higher-order thinking, design questions that focus on higher levels of cognition as defined by Bloom’s taxonomy.
A stem that presents a problem that requires application of course principles, analysis of a problem, or evaluation of alternatives is focused on higher-order thinking and thus tests students’ ability to do such thinking.In addition, the objective scoring associated with multiple choice test items frees them from problems with scorer inconsistency that can plague scoring of essay questions.Validity is the degree to which a test measures the learning outcomes it purports to measure.We can then validate that before executing any code that could possibly make unexpected changes to the system.