10.2 Rating Scale

10.2.1 Each air operator can develop their own rating scale(s) which will be described in the air operator’s approved Dispatch Manual. To provide an example of a possible rating system, a four-point scale will be described. This example should not be taken as limiting possible intervals to a four-point scale, nor should the terminology used herein be seen as limiting. The rating scale and associated criteria are included here to provide a familiar comparative reference with that contained in this manual. With appropriate scale construction and evaluator training, air operators may elect to define other scales that maximize the quality (sensitivity, reliability, validity) of the collected data. When applying the 4-point scale, an ACD will award the mark that best describes the weakest element(s) applicable to the candidate’s performance. Comments to support marks of 1 or 2 should link to a safety issue, technical performance criteria, approved techniques or procedures, and/or non-technical skills. The grades in an example four-point scale are described below. The table in Appendix M can also be used as a guide.

10.2.2 Above Standard or Excellent (4)

  1. Performance, procedure or task remains well within specified performance criteria. Non-technical skills contribute effectively towards the desired outcome. All potential threats received proper consideration. Errors, if any, are inconsequential or immediately corrected.

  2. In this example, a sequence must be rated Above Standard or Excellent (4) where:

    1. Organization is structured, precise and methodical;

    2. Technical knowledge generally exceeds the level required for safe and efficient operation;

    3. Very good cooperation skills ensure effective teamwork and coordination at all times;

    4. Leadership and managerial skills contribute to top team performance; 

    5. Behavior indicates continuous vigilance and highly accurate situational awareness;

    6. Decision-making skills provide for timely decisions using all available information that lead to the safest and most efficient outcome.

10.2.3 Standard (3)

  1. Minor deviations may occur from specified performance criteria while overall performance, procedure or task remains within prescribed limits. Non-technical skills are generally effective in assisting technical performance. Potential threats tend to generate due consideration. Errors, if any, are identified and corrected in a timely manner.

  2. In this example, a sequence must be rated Standard (3) where:

    1. Organization is positive and generally done in a methodical way;

    2. Technical knowledge meets the required level of competency for safe and efficient operation;

    3. Good cooperation skills assist in effective teamwork and coordination;

    4. Leadership and managerial skills contribute to good team performance;

    5. Behavior indicates that situational awareness is maintained;

    6. Decision-making skills provide for timely decisions aimed at safe and effective outcomes.

10.2.4 Basic Standard or Satisfactory (2)

  1. Deviations from specified performance criteria occur, which may include excursions from prescribed tolerances or a major error, but are recognized and corrected within an acceptable timeframe. Non-technical skills may be contributing to sub-standard technical performance. Consideration to relevant threats may not be fully adequate. Errors are poorly managed but do not jeopardize safety of flights.

  2. In this example, a sequence must be rated Basic Standard or Satisfactory (2) where:

    1. Organization is performed with limited proficiency and/or includes brief uncoordinated actions;

    2. Technical knowledge reveals limited competency and/or depth of knowledge with respect to applicable regulatory requirements, SOPs, and/or aircraft systems, limitations and performance characteristics;

    3. Cooperation skills undermine effective teamwork and coordination; 

    4. Leadership and managerial skills allow for deviations from procedures and/or poor team performance;

    5. Behavior indicates lapses in situational awareness that are identified and corrected by the team;

    6. Decision-making skills do not consistently generate decisions providing a safe and efficient outcome.

10.2.5 Below Standard or Unsatisfactory (1)

  1. Unacceptable deviations from specified performance criteria occur, which may include excursions from prescribed tolerances or procedures which are not recognized or corrected within an acceptable timeframe. Non-technical skills detract from overall technical performance. Mismanagement of potential threats and/or errors compromises safety of flights.

  2. In this example, a sequence must be rated Below Standard or Unsatisfactory (1) where: 

    1. Organization is uncoordinated, includes uncorrected or excessive deviations, or leads to an undesired situation;

    2. Technical knowledge reveals unacceptable levels of technical competency and/or depth of knowledge with respect to applicable regulatory requirements, SOPs, and/or aircraft systems, limitations and performance characteristics;

    3. Poor cooperation skills contribute to unresolved conflicts or lead to an unacceptable level of team coordination;

    4. Leadership and managerial skills are counterproductive and ineffective to good team performance unless continuously challenged or prompted by other team members;

    5. Behavior indicates lapses in situational awareness that are not identified and corrected by the team;

    6. Decision-making skills are inadequate and may lead to decisions jeopardizing safety of flights.

Date modified: