9.4 EVALUATION ERRORS

In order to test effectively, the ACD requires not only a sound knowledge of the characteristics of evaluation, but also a firm understanding of the possible errors that can occur throughout the evaluation process. Errors in evaluation fall into several categories.

9.4.1 Personal Bias Error

Personal bias is indicated by a tendency of an ACD to rate candidates or a particular group of candidates the same.

9.4.2 Central Tendency Errors

Central tendency errors are indicated by a tendency to rate all or most candidates as average. The ACD feels that the performance of most candidates is not as good as it should be and therefore underscores a candidate’s good performance. On the other hand, the ACD is reluctant to cope with the possible emotional response of a candidate. It results in padded or inflated assessments of poor performance. This error may also occur because an ACD does not want to put effort into making a decision. An average mark is easier to make.

9.4.3 Generosity Errors

Generosity errors are indicated by a tendency to rate all individuals at the high end of the scale and are probably the most common type of personal bias. This could be caused by an ACD’s desire to be known as a nice person.

9.4.4 Severity Errors

In this case, all or most candidates are graded at the low end of the marking scale. The ACD may feel that the published standards are too low and score the test against their own set of standards. This type of ACD feels that few people can do as well as they can.

9.4.5 Halo Effect

This occurs when an ACD’s impression of a candidate is allowed to influence the assessment of performance. Halo error can result in rating an applicant too high or too low. One form of halo error is the error of leniency. Leniency has its source in an examiners likes, dislikes, opinions, prejudices, moods and political or community influence of people. For example, when testing a friend, acquaintance, or high profile individual, an ACD may give undeservedly high marks or, conversely the error of stereotype.

9.4.6 Stereotype

As with the error of leniency, the error of stereotype has its source in likes, dislikes, opinions, prejudices, etc. In this case however, an ACD may allow personal opinion or prejudice to influence the assessment of the candidate and award undeservedly low marks.

9.4.7 Logical Error

Logical error occurs when an ACD assumes that a high degree of ability in one area means a similar degree of competence in another. This is especially true if more than one item being assessed is similar or related. A good mark on one or two items does not mean the candidate is also qualified on all. The full competency check must be completed and marked.

9.4.8 Error of Narrow Criterion

This may occur when an ACD has more than one candidate to evaluate. The ACD may, under this condition, rate each applicant against the others within the group instead of against the standards. If the group to be tested is above average, a candidate who is of average ability may be awarded an undeservedly low mark. If the group of candidates to be tested is below average, then a candidate who performs the best within this group may be awarded a higher assessment than actually deserved.

9.4.9 Error of Delayed Grading

Should a delay occur in awarding the assessment for an item, there might be a tendency to award average marks due to the lack of information and/or poor recall. By not making an assessment immediately after the event, the ACD may award assessments based upon an overall impression of the competency check. This results in an erroneous assessment and a DCC Report that is of little value to the training system.

9.4.10 Standards Error

All the errors we have discussed result in a standards error. However, if an ACD is not thoroughly familiar with established performance criteria, as outlined in this manual, it is virtually impossible to conduct an evaluation to that standard. While these errors are presented here on paper in a clear and obvious way, under the actual conduct of a DCC, this is not always so. Normally it is a combination of two or more of the errors and clear and obvious is not an apparent trait. Therefore, the ACD must be aware of these errors and consciously prevent such errors from entering, in any degree, into the assessment of competency checks they conduct to ensure the validity of the marks they award.

Date modified: