|
Article Excerpt
Feedback can be defined as knowledge of one's performance provided by an external agent (Ammons, 1956; Annet, 1969; Kluger & DeNisi, 1996). This definition includes the restricted definition of knowledge of results, or outcome feedback, which has generated most of the research.
Hypotheses attributing performance improvements to feedback have been largely a heritage from behaviourism. Based on the law of effect (Thorndike, 1913), positive feedback and reinforcement were equated, while negative feedback became identified with punishment. Thus considered, a beneficial effect of feedback on performance was generally expected: negative feedback would punish incorrect behaviour, while positive feedback would reinforce the correct behaviour. A summary of the results of various experiments on feedback interventions can be found in the review made by Ammons (1956), whose main conclusions were that feedback increases both learning and motivation.
In traditional achievement and aptitude testing, where the objective is not instructional but evaluative, feedback has been recommended mostly for its supposed motivational advantages (e.g. Kyllonen, 1991). From a methodological perspective, it has been hypothesized that item feedback can improve the validity of the scores by eliminating difficulty which is not relevant to the measured construct, such as that produced by anxiety and other motivational or affective factors (Rocklin, O'Donnell, & Holst, 1995; Wise, Roos, Plake, & Nebelsick-Gullet, 1994). Results, however, are contradictory (see Roos, Wise, & Plake, 1997), and the scarce empirical evidence on this matter can mainly be found in the context of research into self-adapted testing, in which the effects of item feedback itself are actually confounded with those of other extraneous variables (e.g. control over the stimuli, total testing time, difficulty of the items). In addition, it should be taken into account that, had tests been properly designed, the informative value for examinees of feedback intervention should be nil with respect to the content of the remaining items (i.e. items must be locally independent). Still, there is some useful information that can be extracted from the knowledge of performance: information related to one's calibration. It is typically assumed that examinees are capable of assessing their own state of knowledge; however, their subjective evaluation is not necessarily correct. if people's subjective judgments are diagnostic of the correctness of their answers, then they are calibrated; else, they are miscalibrated.
Whether information on calibration is (or is not) cognitively relevant to one's performance will depend on the expected consequences of errors on scores: for example, the omission of the answer to an item will be strategically prejudicial if errors have no negative consequences on the score, given that an omission cannot improve the examinee score, but a wild guess could. However, this variable is rarely taken into account, as implicitly recognized by the fact that it is usual to estimate ability parameters from the number of correct responses, without taking errors into account (or taking them into account only indirectly, as byproducts of guessing). As a result of technological requirements, it is also becoming usual not even to give the examinees the option to omit an answer, which might have educationally undesirable consequences (for a recent review, see Prieto & Delgado, 1999a).
Although, owing to the difficulty of modelling incorrect answers, cognitive psychometrical theories have been constructed on right answers, it is impossible to obviate the relevance of errors on accuracy We do not expect witnesses testifying in a court of law to guess when they are not sure about what to say, nor would we accept a wild guess from psychologists on the diagnosis of their patients: in many social contexts, errors have important negative consequences, and therefore omission will be the expected behaviour for people unsure of their knowledge. In the more controlled domain of research on memory processes, it has been shown that people monitor and control their performance by deciding which items to omit as a function of the amount of penalty for errors (e.g. Koriat & Goldsmith, 1996). However, the effect of self-monitoring on accuracy depends on the calibration of the participants, and it is well known that people's confidence judgments are not always diagnostic of the correctness of their answers (Ben-Simon, Budescu, & Nevo, 1997; Gigerenzer, Hoffrage, & Kleinbolting, 1991). if participants are unaware of the inadequacy of their answers (i.e. if they are miscalibrated), but still monitor and control their answers according to instructions (i.e. deciding which items to answer and which to omit), then their level of accuracy under conditions of strong penalty for errors may be little better than under no penalty conditions: for example, some examinees are overconfident and believe that their answers are correct when they are in fact wrong but, having no means to know that the answers they are giving are wrong, they will have nearly as many errors in one condition as in the other. There is some experimental evidence concerning the effect of instructions (and their associated scoring systems) on the accuracy of the participants in testing contexts: when comparing participants' scores calculated without any penalty per error, obtained under the instruction of always guessing when uncertain (called [S.sub.1], to follow Budescu & Bar-Hillel, 1993), with scores calculated with a penalty of 1 per error, obtained under the instruction of better to omit when uncertain given the strong penalty (54), results clearly indicate that, on average, participants monitor their omissiveness according to the expected consequences of wrong answers. This has an effect on the accuracy of their responses: significantly fewer errors are committed under S4 (Prieto & Delgado, 1999a).
Up to...
|
