9.3 CHARACTERISTICS OF EVALUATION
- Record of Amendments
- Record of Bulletins
Part I – General Administration
- Chapter 1: Approved Check Dispatcher Program
- Chapter 2: ACD Delegation Policy and Authorities
- Chapter 3: ACD Qualifications & Authority Application Prodedures
- Chapter 4: Transport Canada Approval Procedures
- Chapter 5: ACD Delegation of Authority
- Chapter 6: Transport Canada ACD Oversight Program
- Chapter 7: Air Operator Responsibilities
- Chapter 8: Conduct of Competency Checks
- Chapter 9: Competency Check Assessments
- Chapter 10: Assessment Standards
- Chapter 11: Validity Periods and Extensions
Part II – Appendixes & Forms
Appendix A: ACD Application Form
Appendix B: ACD Delegation of Authority Form
Appendix C: Dispatcher's Competency Check Report
Appendix D: TC Inspector Request for an Annual ACD Monitor or DCC
Appendix E: Approved Check Dispatcher Monitor Report
Appendix F: ACD or Flight Dispatch Extension Request/Approval
Appendix G: Revocation of ACD Authorization
Appendix H: Recommendation for Competency Check
Appendix I: Operational Control Inspection Report
Appendix J: Sample Letter of Approval
Appendix K: Flight Dispatcher Training & Proficiency Record
Appendix L: Familiarization Flight Report
Appendix M: Competency Check Practices Guide
- Appendix A: ACD Application Form
An evaluation may become useless if certain criteria are not respected. The following five characteristics, if used carefully when conducting a Competency Check, will result in an accurate and effective form of evaluation.
Reliability ensures consistent results. As applied to the competency check, this would mean that two identical performances should result in the same score. Human factors can have a significant effect on reliability. Some of these factors are:
Fatigue: insufficient sleep or rest prior to the competency check;
Emotions: work or home personal problems;
Health: cold or flu, etc.
Time of Day: very early in the morning, or late in the day;
Distractions: noise, interruptions, etc.
The ACD should be conscious of these factors and attempt to reduce as many variables as possible. He may recognize some of these factors as a reason for some lack of accuracy in the candidate’s performance. The ACD should also be aware that his/her ability to accurately assess the candidate’s performance could be affected by these same factors.
Another factor that may affect the reliability of an evaluation is to allow learning to take place during the check. It must be emphasized that testing for the purpose of certification must remain clearly removed from teaching. For example, if worded improperly, questions may lead the candidate to the correct answer. If given a second or third attempt, the candidate may finally answer adequately because of the repeated practice. For this reason, a scenario will not be repeated unless one of the following conditions applies:
Discontinuance: Discontinuance of a scenario for valid reasons such as emergency, mechanical, diversion, weather issue or other procedure necessary to modify the original plan;
Misunderstood Request: A legitimate instance when a candidate does not understand an ACD’s request to perform a specific task. A candidate’s failure to know the requirements of a specified request is not grounds for repeating a task.
Other Factors: Any condition where the ACD was distracted to the point that the candidate’s performance of the task (telephone & radio calls, interruption from another department, etc.) could not adequately be observed.
These provisions have been made in the interest of fairness and safety and do not mean that instruction, practice, or the repeating of an item or task unacceptably demonstrated, are permitted during the evaluation process.
Competency Checks are valid if they measure what they are supposed to measure and nothing else. Assessment of items must remain within the bounds of the appropriate DCC and the guidelines provided in this manual. The scope of the DCC must be such that when candidates pass, they have met the required standards for the issuance of the certificate or its renewal.
A DCC is comprehensive if it contains a sample of all course material and measures each area of skill and knowledge required to ensure the skill requirements are met.
During the conduct of a DCC, discrimination enables the ACD to detect different levels of achievement among candidates. Discrimination separates standard performance from above and below standard performance. A 1-4 marking scale is designed to reveal how candidates perform and allows for a greater degree of discrimination than one that simply distinguishes between pass and fail.
Objectivity ensures the ACD’s personal opinions will not affect the outcome or assessment of the competency check. Marks awarded must be made in accordance with the applicable DCC criteria. Despite the fact that ACDs must strive to remain objective, competency checks may be marked to some degree on a subjective basis when the ACD is an experienced dispatcher, has sound and adequate background knowledge of the evaluation process and has the expertise to accurately assess candidates without prejudice.