9.4 EVALUATION ERRORS
- Record of Amendments
- Record of Bulletins
Part I – General Administration
- Chapter 1: Approved Check Dispatcher Program
- Chapter 2: ACD Delegation Policy and Authorities
- Chapter 3: ACD Qualifications & Authority Application Prodedures
- Chapter 4: Transport Canada Approval Procedures
- Chapter 5: ACD Delegation of Authority
- Chapter 6: Transport Canada ACD Oversight Program
- Chapter 7: Air Operator Responsibilities
- Chapter 8: Conduct of Competency Checks
- Chapter 9: Competency Check Assessments
- Chapter 10: Assessment Standards
- Chapter 11: Validity Periods and Extensions
Part II – Appendixes & Forms
Appendix A: ACD Application Form
Appendix B: ACD Delegation of Authority Form
Appendix C: Dispatcher's Competency Check Report
Appendix D: TC Inspector Request for an Annual ACD Monitor or DCC
Appendix E: Approved Check Dispatcher Monitor Report
Appendix F: ACD or Flight Dispatch Extension Request/Approval
Appendix G: Revocation of ACD Authorization
Appendix H: Recommendation for Competency Check
Appendix I: Operational Control Inspection Report
Appendix J: Sample Letter of Approval
Appendix K: Flight Dispatcher Training & Proficiency Record
Appendix L: Familiarization Flight Report
Appendix M: Competency Check Practices Guide
- Appendix A: ACD Application Form
In order to test effectively, the ACD requires not only a sound knowledge of the characteristics of evaluation, but also a firm understanding of the possible errors that can occur throughout the evaluation process. Errors in evaluation fall into several categories.
9.4.1 Personal Bias Error
Personal bias is indicated by a tendency of an ACD to rate candidates or a particular group of candidates the same.
9.4.2 Central Tendency Errors
Central tendency errors are indicated by a tendency to rate all or most candidates as average. The ACD feels that the performance of most candidates is not as good as it should be and therefore underscores a candidate’s good performance. On the other hand, the ACD is reluctant to cope with the possible emotional response of a candidate. It results in padded or inflated assessments of poor performance. This error may also occur because an ACD does not want to put effort into making a decision. An average mark is easier to make.
9.4.3 Generosity Errors
Generosity errors are indicated by a tendency to rate all individuals at the high end of the scale and are probably the most common type of personal bias. This could be caused by an ACD’s desire to be known as a nice person.
9.4.4 Severity Errors
In this case, all or most candidates are graded at the low end of the marking scale. The ACD may feel that the published standards are too low and score the test against their own set of standards. This type of ACD feels that few people can do as well as they can.
9.4.5 Halo Effect
This occurs when an ACD’s impression of a candidate is allowed to influence the assessment of performance. Halo error can result in rating an applicant too high or too low. One form of halo error is the error of leniency. Leniency has its source in an examiners likes, dislikes, opinions, prejudices, moods and political or community influence of people. For example, when testing a friend, acquaintance, or high profile individual, an ACD may give undeservedly high marks or, conversely the error of stereotype.
As with the error of leniency, the error of stereotype has its source in likes, dislikes, opinions, prejudices, etc. In this case however, an ACD may allow personal opinion or prejudice to influence the assessment of the candidate and award undeservedly low marks.
9.4.7 Logical Error
Logical error occurs when an ACD assumes that a high degree of ability in one area means a similar degree of competence in another. This is especially true if more than one item being assessed is similar or related. A good mark on one or two items does not mean the candidate is also qualified on all. The full competency check must be completed and marked.
9.4.8 Error of Narrow Criterion
This may occur when an ACD has more than one candidate to evaluate. The ACD may, under this condition, rate each applicant against the others within the group instead of against the standards. If the group to be tested is above average, a candidate who is of average ability may be awarded an undeservedly low mark. If the group of candidates to be tested is below average, then a candidate who performs the best within this group may be awarded a higher assessment than actually deserved.
9.4.9 Error of Delayed Grading
Should a delay occur in awarding the assessment for an item, there might be a tendency to award average marks due to the lack of information and/or poor recall. By not making an assessment immediately after the event, the ACD may award assessments based upon an overall impression of the competency check. This results in an erroneous assessment and a DCC Report that is of little value to the training system.
9.4.10 Standards Error
All the errors we have discussed result in a standards error. However, if an ACD is not thoroughly familiar with established performance criteria, as outlined in this manual, it is virtually impossible to conduct an evaluation to that standard. While these errors are presented here on paper in a clear and obvious way, under the actual conduct of a DCC, this is not always so. Normally it is a combination of two or more of the errors and clear and obvious is not an apparent trait. Therefore, the ACD must be aware of these errors and consciously prevent such errors from entering, in any degree, into the assessment of competency checks they conduct to ensure the validity of the marks they award.