Advanced Qualification Program Evaluator Manual TP 14672

CHAPTER 10 – ASSESSMENT STANDARDS

10.1 General

10.1.1 The air operator must decide how, when, where, and who will assess a candidate’s proficiency on each terminal and supporting proficiency objective. This testing strategy is contained in the documents which comprise the air operator’s approved Program Audit Database (PADB).

10.1.2 A Qualification Standard is a job task proficiency objective (TPO or SPO) linked to an evaluation strategy. Qualification Standards define the requirements for mastery of the duty position. Demonstration that an individual has met the required standards will lead to certification. The Qualification Standards also identify what constitutes a failure and/or unsatisfactory performance.

10.1.3 In addition to addressing the testing/validation/evaluation methodology, the air operator must also specify the approach to be used in documenting the results of validations and evaluations.

10.1.4In addition to any grade sheet or electronic data collected by the air operator, the LOE will be documented on the Flight Test Report Pilot Proficiency Check, form 26-0249 (AppendixE), as required by paragraph 13.1.3. In addition, the OE will be documented on a suitable Online Evaluation (OE) Report.

10.1.5The ratings assigned during validations and evaluations are critical to the effectiveness of an AQP. This data is collected and analyzed by the air operator to verify student, instructor and evaluator proficiency. Data will also be collected and analyzed by the operator for:

  1. continued validation of the AQP;
  2. identification of requirements for curriculum changes; and
  3. program maintenance.

10.1.6Transport Canada will also conduct, on a regular basis, a review of AQP data submitted by the air operator.

10.2 Introduction to Rating Scales/Scoring

10.2.1 Each AQP must have a rating methodology for grading the performance of the proficiency objectives against the Qualification Standards. The measurement codes associated with performance events are typically ratings, repeat counts, and reason codes or skill categories.

10.2.2 Ratings are used to define different levels of performance. Rating codes are usually air operator specific. They can be chosen by the individual air operator to meet the specific requirements of their AQP; however, Transport Canada requires the use of something more sensitive to performance differences than a binary code (i.e. some rating method that provides more performance differentiation than pass/fail for individual items being evaluated).

10.2.3 For First-Look Manoeuvres (FLM), Manoeuvres Validation (MV), Line Operational Evaluation (LOE) and Online Evaluation (OE) a minimum four point grading scale shall be used. An example of a four point rating scale that discriminates among performance levels is provided below in section10.3.

10.2.4Each air operator should ensure that the grades established on the rating scale are clearly defined, meaningful to the instructor and evaluator, and easily used for performance assessment. Consistency among fleets and across different types of validations and evaluations (FLM, MV, LOE and OE) is important and generally desirable. However, rating scales may be slightly different when used for different purposes. For example, different rating scales may be used for validation/evaluation as compared to training.

10.2.5When applying any rating scale, evaluators should award the grade that best describes the weakest element(s) applicable to the candidate’s performance.

10.3 Example of a Four-Point Rating Scale

10.3.1 As described above, each air operator will develop their own rating scale(s) which will be described in the air operator’s approved PADB. To provide an example of a possible rating system, a four-point scale will be described. This example should not be taken as limiting possible intervals to a four-point scale, nor should the terminology used herein be seen as limiting. The rating scale and associated criteria are included here to provide a familiar comparative reference with that contained in the Approved Check Pilot Manual. With appropriate scale construction and instructor and evaluator training, air operators may elect to define other scales that maximize the quality (sensitivity, reliability, validity) of the collected data. The grades in an example four-point scale are described below.

10.3.2 Above Standard or Excellent (4)

  1. Performance remains well within the Qualification Standards and management skills are excellent.
  2. In this example, a sequence would be rated Above Standard or Excellent (4) where:
    1. performance is ideal under existing conditions,
    2. aircraft handling is smooth and precise,
    3. technical skills and knowledge exceed the required level of competency,
    4. behavior indicates continuous and highly accurate situational awareness,
    5. flight management skills are excellent,
    6. safety of flight is assured, risk is well mitigated.

10.3.3 Standard (3)

  1. Minor deviations occur from the qualification standards and performance remains within prescribed limits.
  2. In this example, a sequence would be rated Standard (3) where:
    1. performance meets the recognized standard yet may include deviations that do not detract from the overall performance,
    2. aircraft handling is positive and within specified limits,
    3. technical skills and knowledge meet the required level of competency,
    4. behavior indicates that situational awareness is maintained,
    5. flight management skills are effective,
    6. safety of flight is maintained, risk is acceptably mitigated.

10.3.4Basic Standard or Satisfactory (2)

  1. Deviations from the qualification standards occur, which may include momentary excursions beyond prescribed limits but these are recognized and corrected in a timely manner.
  2. In this example, a sequence would be rated Basic Standard or Satisfactory (2) where:
    1. technical skills and knowledge reveal limited technical proficiency and/or depth of knowledge,
    2. behavior indicates lapses in situational awareness that are identified and corrected by the crew,
    3. flight management skills are effective, but slightly below standard. Some items are addressed only when challenged or prompted by other crewmembers,
    4. safety of flight is not compromised, risk is poorly mitigated.

10.3.5Below Standard or Unsatisfactory (1)

  1. Unacceptable deviations from the qualification standards occur, which may include excursions beyond prescribed limits that are not recognized or corrected in a timely manner.
  2. In this example, a sequence would be rated Below Standard or Unsatisfactory (1) where:
    1. technical skills and knowledge reveal unacceptable levels of technical proficiency and/or depth of knowledge,
    2. behavior indicates lapses in situational awareness that are not identified or corrected by the crew,
    3. flight management skills are ineffective, unless continuously challenged or prompted by other crewmembers,
    4. safety of flight is compromised. Risk is unacceptably mitigated.

10.4Reasons for Assessment

10.4.1 Remarks or reason codes should be applied to support all mark awards that are substandard. For example, in the four-point scale described in 10.3, any awarded marks less than "Standard" would require a remark or reason code.

10.4.2 These remarks should be linked to the Qualification Standards and identify such things as a safety issue, a competency standard (manual flying skills or use of autoflight systems for example), CRM items (such as crew performance monitoring, decision-making, workload management, communication skills, situational awareness), knowledge of aircraft systems, or an approved technique or procedure. Linking sub-standard performance to a remark or reason code will facilitate data analysis and the implementation of adequate corrective actions.

10.5Elements of Assessment

10.5.1 In accordance with the Qualification Standards, assessments will ensure that proficiency in both technical and CRM aspects are addressed. Several of the elements that should be evaluated are discussed below. This list is not exhaustive. In addition to the generic information listed below, air operators should develop their own assessment guidance material and associated tools for evaluators. The air operator’s assessment guidance material should reflect the significant increase in emphasis on CRM, which is fundamental to all AQPs.

10.5.2 As a minimum, the following six elements will be evaluated:

  1. Performance
  2. Aircraft Handling
  3. Technical Skills and Knowledge
  4. Situational Awareness
  5. Flight Management Skills
  6. Safety of Flight

10.5.3 Performance:

  1. Overall error assessment
    1. no errors, or
    2. magnitude, significance, or consequence of errors
    3. risk of such errors during critical phases of flight

  2. Recognition of errors
    1. recognized
    2. unrecognized

  3. Error management
    1. promptness or delay in correcting errors
    2. not corrected

10.5.4Aircraft handling:

  1. Quality of handling
    1. smoothness and coordination of controls
    2. control input appropriate to the flight situation
    3. airmanship

  2. Accuracy
    1. use of approved technique or procedure
    2. performance relative to specified tolerances
    3. action taken when deviations occur
    4. magnitude of deviations

10.5.5Technical skills and knowledge:

  1. General
    1. practical use and understanding of aircraft systems and automation, data, charts, weather and physiological factors
    2. knowing what to do, how to do it and understanding why

  2. Expected level of competency
    1. appropriate to the requirements of the qualification sought
    2. competency that would get the job done safely and efficiently
    3. above average, average, or below average

10.5.6Situational Awareness:

  1. General
    1. resides in the candidate’s mind and can only be assessed by monitoring behavior

  2. Behavior
    1. actively monitors weather, aircraft systems, instruments, ATC communications
    2. avoids tunnel vision and fixation
    3. stays "ahead of the aircraft", stays "with the aircraft", gets "behind the aircraft"

  3. Identification and correction of errors
    1. omissions, slips and lapses
    2. detection and correction of errors.

10.5.7Flight Management Skills:

  1. Degree of effectiveness
    1. makes effective use of available resources
    2. anticipates problems far enough in advance
    3. uses effective decision-making processes
    4. maintains the ability to adapt during high workload situations by prioritizing and allocating tasks effectively
    5. avoids distractions during high workload situations
    6. establishes and maintains effective communication with all crew members as well as other persons and outside agencies
    7. uses effective leadership techniques

10.5.8Safety of Flight:

  1. Degree to which safety was maintained or jeopardized
    1. respect for published procedures and limits
    2. effectiveness of lookout during visual manoeuvres
    3. errors that are serious or have potentially grave consequences
    4. breach of regulations (intervention required)
    5. any situation where the examiner had to intervene to ensure the safety of the flight

10.6Tolerances

10.6.1 The tolerances for instrument flight sequences must be respected by all evaluators. Each candidate must demonstrate aircraft control to maintain:

  1. assigned headings during normal flight within ±10 degrees;
  2. tracking VOR/LOC/LOC BC/ILS/RNAV within ½ scale deflection;
  3. NDB bearings within ±10 degrees prior to final approach course (and prior to the final approach fix) and ±5 degrees on the final approach course (and after passing the final approach fix);
  4. altitude:
    1. during normal flight within ±100 feet,
    2. during approach and for minimum IFR altitudes associated with the intermediate and final segments (e.g., FAF, beacon crossing or step-down fixes) within + as required/-0 feet, and
    3. accurate altitude control is required at MDA,

  5. airspeed during normal flight within ±10 knots; and
  6. airspeed during take-off and approach within +10/ -5 knots.

10.6.2 These criteria assume no unusual circumstances or conditions and may require allowances for momentary variations. Such things as weather, turbulence, simulated malfunction and type of approach may modify the exact rating definition and tolerances to be applied during a particular sequence.

10.6.3 The competency of each pilot to fly instrument procedures, to the standards specified in 10.6.1, will be monitored during each validation and evaluation.

10.6.4If, during a LOE, a pilot fails to demonstrate an adequate level of competency in those sequences mandatory for instrument flying competency, the Type E Evaluator conducting that LOE shall suspend the pilot’s Instrument Rating as described in section9.9.

10.7Validations/Evaluations - General

10.7.1 To evaluate the overall technical proficiency, communications skills, leadership and situational awareness of pilots with respect to normal and abnormal procedures, evaluators must observe the performance of each crew closely. To evaluate specific items listed in the Qualification Standards, the applicable validation/evaluation shall be conducted in a manner that enables the pilots to demonstrate knowledge and skill with respect to such things as aircraft automation including FMS/RNAV programming, auto flight systems and flight mode awareness, pilot not flying (PNF) duties, crew coordination and pilot decision making.

10.7.2 When assessing normal procedures, the evaluator must ensure the crew demonstrates adequate knowledge of the company SOPs and aircraft systems to confirm their ability to use installed equipment properly. In addition, aircraft operation must be assessed with specific reference to those items requiring crew co-ordination and discipline.

10.7.3 The crew shall demonstrate the use of as many of the Operator’s approved Standard Operating Procedures and normal procedures as are necessary to confirm that the crew has the knowledge and ability to use installed equipment properly, including FMS, auto-pilot and hand flown manoeuvres as appropriate.

10.7.4Evaluators must adhere to the applicable script to ensure that all required sequences are covered in each validation and evaluation.

10.7.5As described in 10.1, the assessments made during validations and evaluations are made with respect to the air operator’s Qualification Standards. A discussion of generic standards, which should be reflected in the Qualification Standards, appears below in sections10.9 to10.39.

10.8Standardized Phases of Flight

10.8.1 For the purposes of flight checks conducted under Part VII of the Canadian Aviation Regulations, Transport Canada has adopted the Standardized Phases of Flight as specified in ATA iSpec2200, issued April2002. These Standardized Phases of Flight are also typically utilized in AQP, but should not be seen as limiting.

10.8.2 These phases of flight are identified as:

  1. flight planning,
  2. pre-flight,
  3. engine start/depart,
  4. taxi-out,
  5. take-off,
  6. rejected take-off,
  7. initial climb,
  8. en route climb,
  9. cruise,
  10. descent,
  11. approach,
  12. go-around,
  13. landing,
  14. taxi-in,
  15. arrival/engine shutdown,
  16. post-flight, and
  17. flight close.

10.8.3 Descriptions of each of the standardized phases of flight, the associated manoeuvres and sequences, and the common errors that occur during each phase are listed below.

10.9Flight Planning

10.9.1 Begins when the flight crew initiates the use of flight planning information facilities and becomes dedicated to a flight based upon a route and an aircraft; ends when the crew arrives at the aircraft for the purpose of the planned flight or the crew initiates a Flight Close phase.

10.9.2 The crew must demonstrate adequate knowledge of the company’s SOPs, AOM and AFM, including aircraft performance charts and weight and balance procedures to effectively plan a flight.

10.9.3 Assessment will, where applicable, be based on the candidate/crew’s ability to:

  1. select an appropriate route, altitude and alternate;
  2. obtain and correctly interpret applicable NOTAM information;
  3. calculate the estimated time en route and total fuel requirement based on factors such as power settings, operating altitude or flight level, wind, and fuel reserve requirements;
  4. calculate the aircraft weight and balance for the planned flight;
  5. determine that the required performance for the planned flight is within the aircraft’s capability and operating limitations;
  6. locate and apply information essential to the flight;
  7. complete, or participate in the completion of, a flight plan which
    1. reflects the conditions of the proposed flight; and
    2. is in accordance with procedures specified in the COM,

  8. demonstrate sufficient knowledge of the regulatory requirements relating to instrument flying specified in the regulations.

10.9.4Some common errors that may affect the assessment are as follows:

  1. lack of proper charts and manuals;
  2. inadequate knowledge of, or proficiency in, the interpretation of performance charts; or
  3. failure to check if fuel load is adequate for the intended flight.

10.10Pre-flight

10.10.1 Begins with flight crew arrival at an aircraft for the purpose of flight; ends when a decision is made to depart the parking position and/or start the engine(s). It may also end by the crew initiating a Post-flight phase.

10.10.2 Assessment will be based on the candidate/crew’s ability to:

  1. safely inspect and prepare the aircraft for engine start by ensuring that all checks and procedures are carried out according to the applicable AOM, AFM, COM and SOPs.

10.11 Engine Start/Depart

10.11.1 Begins when the flight crew take action to have the aircraft moved from the parked position and/or take switch action to energize the engine(s); ends when the aircraft begins to move forward under its own power or the crew initiates an Arrival/Engine Shutdown phase.

10.11.2 Assessment will be based on the candidate/crew’s ability to:

  1. carry out the appropriate checks and procedures specified in the applicable AOM, AFM, COM and SOPs.
  2. identify and respond to abnormal or emergency situations in accordance with procedures specified in the applicable aircraft checklist, QRH, AOM, AFM, COM and SOPs.

10.12 Taxi-out

10.12.1 Begins when the crew moves the aircraft forward under its own power; ends when thrust is increased for the purpose of take-off or the crew initiates a Taxi-in phase.

10.12.2 Assessment will be based on the candidate/crew’s ability to:

  1. avoid any activity that would compromise lookout on the ramp or taxiway;
  2. limit radio procedures and conversation from outside and within the aircraft to ensure compliance with ATC direction or clearance (i.e., judicious use of company frequencies, cockpit chatter, etc.).
  3. adhere to company runway incursion avoidance procedures; and
  4. set up and check the aircraft systems, radios and instruments in accordance with prevailing departure procedures and weather.

    Note:Any aircraft system required due to weather, navigational requirements or crew composition shall be checked and set for take-off, i.e., weather radar, de icing equipment, heaters, on board navigation equipment, auto pilot, auto-throttles, FMS, etc.

10.13 Flight Planning, Pre-flight, Engine Start/Depart and Taxi-out

10.13.1 Flight planning, pre-flight, engine start/depart and taxi-out are completed as a crew exercise and, for validation/evaluation purposes, need only be demonstrated once when the captain and first officer perform the duties of their assigned seat position.

10.13.2 Inspection of the aircraft, required de icing procedures and aircraft documents must be in accordance with the applicable AOM, AFM, COM and SOPs. The Pilot in Command must ensure adequate ramp safety for start, push back/power back, and taxi.

10.13.3 Engine checks, if applicable, shall be conducted by each crew according to the applicable AOM, AFM, COM and SOPs.

10.14Take-off

10.14.1 Begins when the crew increases the thrust for the purpose of lift-off; ends when an Initial Climb is established (35 feet above runway elevation) or the crew initiates a Rejected Take-off phase.

10.14.2 Each pilot must perform the take-off exercises as detailed in the appropriate script. Each crew need only conduct a complete take-off briefing once. Discussing specific safety items, or changes to the original departure, constitute an acceptable briefing for subsequent take-offs.

10.14.3 Assessment will be based on the candidate/crew’s ability to:

  1. ensure that published cockpit procedures and correct airspeeds are observed during ground roll and lift off;
  2. rotate the aircraft smoothly to the correct pitch angle;
  3. attain a satisfactory rate of climb and the required airspeed in a reasonable period of time;
  4. handle the engines smoothly and positively and establish and monitor the correct power setting.

10.14.4Some common errors that may be observed and affect the assessment of the sequence are as follows:

  1. checks not complete, or out of sequence;
  2. use of incorrect speeds or power settings;
  3. incorrect take-off technique;
  4. mishandling of throttles or thrust levers;
  5. loss of directional control, or using incorrect control input to correct adverse yaw during the take-off roll;
  6. exceeding engine or airframe limitations;
  7. rotation before, or lift off at an airspeed less than, VMCA or VR; or
  8. an incorrect or incomplete check resulting in a vital action being missed.

10.15Rejected Take-off

10.15.1 Begins when the crew decides to reduce thrust for the purpose of stopping the aircraft prior to the end of the Take-off phase; ends when the aircraft is taxied off the runway for a Taxi-in phase or when the aircraft is stopped and engines shut down.

10.15.2 Rejected take-offs will be conducted in simulators only. For validations conducted in an aircraft, the candidate will verbally respond to a scenario briefed by the evaluator. The response will outline the actions of the PF and PNF as appropriate.

10.15.3 For validations and evaluations conducted in a simulator, a rejected take-off shall be completed by each crewmember as appropriate to their assigned seat position.

10.15.4Assessment will be based on the candidate/crew’s ability to:

  1. communicate effectively between themselves, with cabin crew and ATC,
  2. maintain control of the aircraft during deceleration and stop the aircraft on the runway surface or over-run in compliance with the applicable AOM, AFM, COM and SOPs.

10.15.5Some common errors that may be observed and affect the assessment of the sequence are as follows:

  1. failure to alert the crew with the appropriate call, if applicable, e.g., "Reject" or "Stop";
  2. failure to maximize use of brakes and/or improper handling of stopping devices;
  3. failure to alert ATC to emergency, and request assistance;
  4. failure to advise cabin crew of type of emergency and initiate appropriate evacuation procedures (if any);
  5. failure to complete emergency checks and/or power plant(s) shutdown if required;
  6. failure to recognize the need to initiate a rejected take-off prior to V1;
  7. failure to maintain control of the aircraft or stop within the confines of the runway; or
  8. endangering the safety of passengers and crew and/or rescue personnel through improper handling of the emergency condition.

10.16Initial Climb

10.16.1 Begins at 35 feet above the runway elevation; ends after the speed and configuration are established at a defined manoeuvering altitude or to continue the climb for the purpose of cruise. It may also end by the crew initiating an Approach phase.

10.16.2 Assessment will be based on the candidate/crew’s ability to:

  1. select and use the appropriate communications and navigation systems associated with the proposed departure phase,
  2. perform the aircraft checklist items relative to the phase of flight,
  3. intercept, in a timely manner, all tracks, radials, and bearings appropriate to the procedure, route, or clearance,
  4. correctly adhere to departure and noise abatement procedures, and
  5. maintain proper aircraft control and flight within operating configurations and limitations.

10.17En-route Climb

10.17.1 Begins when the crew establishes the aircraft at a defined speed and configuration enabling the aircraft to increase altitude for the purpose of cruise; ends with the aircraft established at a predetermined constant initial cruise altitude at a defined speed or by the crew initiating a Descent phase.

10.17.2 Assessment will be based on the candidate/crew’s ability to:

  1. select and use the appropriate communications and navigation systems associated with the proposed departure phase;
  2. perform the aircraft checklist items relative to the phase of flight;
  3. intercept, in a timely manner, all tracks, radials, and bearings appropriate to the procedure, route, or clearance;
  4. correctly adhere to departure, noise abatement and transition procedures; and
  5. maintain proper aircraft control and flight within operating configurations and limitations.

10.18Cruise

10.18.1 Begins when the crew establishes the aircraft at a defined speed and predetermined constant initial cruise altitude and proceeds in the direction of a destination; ends with the beginning of Descent for the purpose of an approach or by the crew initiating an En Route Climb phase.

Note: For the purposes of validations, steep turn and stall manoeuvres will be included in this phase of flight where applicable, as well as the holding procedure.

10.19Steep Turns

10.19.1 If required, the candidate’s ability to maintain bank angle, altitude and airspeed should be checked in one or more 45° bank turns through at least 180°. He/she should be allowed to stabilize the aircraft at the required altitude and airspeed before starting the turn(s).

10.19.2 Some common errors that may be observed and affect the assessment of the sequence are as follows:

  1. failure to maintain bank angle;
  2. failure to maintain airspeed; or
  3. failure to maintain altitude.

10.20Approach to the Stall/Stall Procedures

10.20.1 If required, approach to the stall/stall procedures are carried out on validations to ensure the candidate is familiar with the stall warning devices and airframe response to the onset of the stall condition. Care must be exercised to ensure that limitations imposed by the AFM are not exceeded in the event an approach to the stall is made with warning devices deactivated (if authorized in the flight manual). The exercise may be carried out with the aircraft in either the take-off, clean or landing configuration.

10.20.2 Some common errors that may affect the assessment of the exercise are as follows:

  1. incorrect application of power;
  2. allowing the nose to come up prior to safety speed being attained during recovery resulting in secondary stall or stall warning;
  3. not recovering lost altitude when safety speed attained;
  4. a significant altitude loss; or
  5. incorrect recovery procedure or aircraft configuration.

10.21 Holding

10.21.1 Each pilot shall conduct a holding procedure consisting of entry, the hold and exit as appropriate to the aircraft type. For FMS equipped aircraft, each pilot must demonstrate the ability to program a hold and to clear it, but at the discretion of the evaluator, only one hold is required to be flown. Flying the hold for the second crewmember is not required.

10.21.2 Assessment will be based on the candidate/crew’s ability to:

  1. recognize arrival at the holding fix and initiate entry into the holding pattern;
  2. use a suitable entry procedure as specified in the Instrument Procedures Manual;
  3. report entering the hold;
  4. use the proper timing criteria, where applicable;
  5. comply with leg lengths when a DME distance is specified;
  6. assess and use proper wind correction procedures;
  7. maintain a deviation of not more than 10 degrees from the designated track or course or within ½ scale deflection of the course deviation indicator, as applicable;
  8. maintain airspeed within plus or minus 10 knots of declared airspeed;
  9. maintain altitude within plus or minus 100 feet; and
  10. maintain proper aircraft control and flight within operating configurations and limitations while in the hold.

10.21.3 Some common errors that may affect the assessment of the sequence are as follows:

  1. failure to obtain a current altimeter setting and to set and cross check the altimeters according to company SOPs;
  2. failure to obtain an expected approach time (EAT);
  3. failure to adjust power settings according to the company SOPs;
  4. poor tracking or incorrect allowance for wind;
  5. failure to enter a holding pattern using standard IFR procedures;
  6. failure to fly the holding pattern as prescribed;
  7. allowing the aircraft to exceed an assigned airspeed or altitude limitation;
  8. violating the accepted and acknowledged ATC clearance;
  9. inability to correctly program and execute the hold procedure with the FMS;
  10. unable to effectively clear the hold from the FMS or to depart the holding pattern;
  11. failure to select the correct auto-flight modes for lateral navigation and airspeed control; or
  12. failure to comply with an ATC instruction.

10.22 Descent

10.22.1 Begins when the crew departs the cruise altitude for the purpose of an approach at a particular destination; ends when the crew initiates changes in aircraft configuration and/or speeds to facilitate a landing on a particular runway. It also may end by the crew initiating an En Route Climb or Cruise phase.

10.23 En Route Climb, Cruise, Descent

10.23.1 Each pilot shall demonstrate enroute climb, cruise and descent manoeuvres.

10.23.2 Assessment will be based on the candidate/crew’s ability to:

  1. adhere to any clearance, whether actual or simulated, and understand and follow the procedures in SIDs, STARs and published transitions, as well as noise abatement procedures;
  2. demonstrate proper use of navigational equipment including the FMS when applicable.

10.23.3 Some common errors that may be observed and affect the rating of the sequences are as follows:

  1. not familiar with, or failure to follow, a SID, STAR or transition;
  2. failure to adhere to noise abatement procedures;
  3. incorrect selection of radio aids or failure to properly identify facilities;
  4. altitude, heading or airspeed allowed to deviate due to pre-occupation or poor cockpit management of workload;
  5. an attempt made to follow a procedure that would violate an accepted and acknowledged ATC clearance or instruction, or endanger the aircraft;
  6. departure or arrival not correctly programmed or failure to monitor the flight guidance modes;
  7. inability to program and fly an altitude crossing restriction or lateral offset;
  8. failure to select and display FMS pages according to company SOPs; or
  9. inability to correctly program the FMS for a change of destination or to activate the alternate flight plan.

10.24Approach

10.24.1 Begins when the crew initiates changes in aircraft configuration and/or speeds enabling the aircraft to manoeuvre for the purpose of landing on a particular runway; ends when the aircraft is in the landing configuration and the crew is dedicated to land on a specific runway. It may also end by the crew initiating an Initial Climb or Go-around phase.

10.25Instrument Approaches

10.25.1 Each pilot must complete the requisite number and type of instrument approaches as detailed in the applicable validation and evaluation scripts. Each crew must conduct a managed and non-managed (or VNAV) approach if applicable to the aircraft type. One approach must be made with a simulated engine failure.

10.25.2 Each crew must demonstrate one Category II or Category III approach if authorized in an Air Operator Certificate (AOC). Where an air operator is authorized both CAT II and CAT III, both types of approaches shall be conducted during the Qualification Course (QC) Manoeuvres Procedures Validation (MPV).

Warning: Conducting an autoland on CAT I ILS facilities can cause unpredictable aircraft performance, especially during visual weather conditions where the ILS signal protection is not maintained.

Warning: CAT II and CAT III approaches shall only be conducted on facilities that support that operation.

10.25.3 Evaluators will pay particular attention to the briefing, when operating in a multiple crew environment, to ensure it is in accordance with the Operator’s SOPs or covers a review of the:

  1. type of approach to be conducted;
  2. missed approach procedure; and
  3. landing configuration.

10.25.5Assessment of the candidate/crew’s ability to organize and share the cockpit workload, in respect to crew resource management, is accomplished by ensuring adherence to company SOPs.

10.25.6Some errors common to all Instrument Approaches that may affect the assessment of the exercise or sequence are as follows:

  1. not following published transitions when cleared to do so;
  2. not using the correct radials or tracks;
  3. incorrect selection of radio aids or failure to properly identify facilities;
  4. descent below procedure turn altitude;
  5. no altimeter correction for cold weather temperatures;
  6. unable to properly program the FMS/RNAV for the type of approach;
  7. not sure when to leave last assigned altitude for transition, initial, or procedure turn altitude when cleared for the approach;
  8. not monitoring raw data for the approach when appropriate;
  9. failure to conduct a navigation instrument accuracy check if required;
  10. failure to respect step down fixes;
  11. improper flight director (FD) mode selected for type of approach;
  12. slow to make corrections or change modes when tracking;
  13. not monitoring all required approach aids;
  14. loss of separation with other aircraft due to incorrect interpretation of, or failure to follow, an ATC clearance or instruction, or a published approach procedure;
  15. crew duties, including monitoring and verbal call-outs, not in accordance with company SOPs;
  16. commencing a missed approach either too early or too late because of poor speed control, wind effect, navigation or timing;
  17. aircraft not in a position to land due to lateral or vertical misalignment or too high an airspeed at DH, MDA or on turning final from a circling procedure;
  18. failure to initiate a go-around in accordance with the published aircraft and company procedures;
  19. configuring the aircraft inappropriately for the phase of flight; or
  20. manoeuvering the aircraft inappropriately for the phase of flight.

10.26NDB Approach

10.26.1 Assessment will be based on the candidate/crew’s ability to:

  1. select and comply with the NDB instrument approach procedure to be performed;
  2. select, tune, identify, confirm, and monitor the operational status of ground and aircraft navigation equipment to be used for the approach procedure;
  3. establish the appropriate aircraft configuration and airspeed and complete the aircraft checklist items for that phase of flight;
  4. prior to final approach course, maintain altitude within plus or minus 100 feet, heading and bearing within plus or minus 10 degrees;
  5. on the final approach course, maintain a deviation of not more than 5 degrees from the designated track or course;
  6. maintain airspeed within plus or minus 10 knots of the declared approach speed;
  7. descend to and maintain the MDA and accurately track to the MAP or to minimum visibility so as to permit completion of the visual portion of the approach with minimal manoeuvering; and
  8. initiate the missed approach procedure, if the required visual references for the intended runway are not obtained at the MAP.

10.27VOR/LOC/LOC BC

10.27.1 Assessment will be based on the candidate/crew’s ability to:

  1. select and comply with the VOR/ LOC/ LOC BC instrument approach procedure to be performed;
  2. select, tune, identify, and confirm the operational status of ground and aircraft navigation equipment to be used for the approach procedure;
  3. establish the appropriate aircraft configuration and airspeed and complete the aircraft checklist items for that phase of flight;
  4. prior to final approach course, maintain altitude within plus or minus 100 feet, heading within plus or minus 10 degrees;
  5. on the final approach course, maintain VOR/ LOC/ LOC BC within ½ scale deflection of the course deviation indicator;
  6. maintain airspeed within plus or minus 10 knots of the declared approach speed;
  7. descend to and maintain the Minimum Descent Altitude (MDA) and accurately track to the Missed Approach Point (MAP) or to minimum visibility so as to permit completion of the visual portion of the approach with minimal manoeuvering; and
  8. initiate the missed approach procedure, if the required visual references for the intended runway are not obtained at the MAP.

10.28Common Errors – Non-Precision Approaches

10.28.1 Some common errors on Non-Precision Approaches that may be observed and affect the rating of the exercise are as follows:

  1. failure to establish a drift angle on the inbound track;
  2. arriving over the FAF on final too high and/or fast, including accepting an ATC assigned airspeed that leads to de-stabilizing the aircraft approach;
  3. reaching MDA too late;
  4. failure to establish the correct MAP;
  5. inability to program and fly a managed or VNAV approach as appropriate to the aircraft type; or
  6. aircraft incorrectly configured at FAF.

10.29ILS Approach

10.29.1 Assessment will be based on the candidate/crew’s ability to:

  1. select and comply with the ILS instrument approach procedure to be performed;
  2. select, tune, identify, and confirm the operational status of ground and aircraft navigation equipment to be used for the approach procedure;
  3. establish the appropriate aircraft configuration and airspeed and complete the aircraft checklist items for that phase of flight;
  4. prior to final approach course, maintain altitude within plus or minus 100 feet and heading or course within plus or minus 10 degrees;
  5. on final approach course, allow no more than ½ scale deflection of the localizer and/or glideslope indications;
  6. maintain airspeed within plus or minus 10 knots of the declared approach speed;
  7. descend to the DH so as to permit completion of the visual portion of the approach with minimal manoeuvering; and
  8. initiate the missed approach procedure upon reaching the DH, when the required visual references for the intended runway are not obtained.

10.30GPS/RNAV Approach

10.30.1 Assessment will be based on the candidate/crew’s ability to:

  1. select and comply with the GPS instrument approach procedure to be performed;
  2. retrieve the GPS approach from the database, conduct a Receiver Autonomous Integrity Monitoring (RAIM) check or a multi-sensor RNAV check and verify the approach waypoints used for the approach procedure;
  3. establish the appropriate aircraft configuration and airspeed and complete the aircraft checklist items for that phase of flight;
  4. prior to final approach course, maintain altitude within plus or minus 100feet, heading plus or minus 10 degrees;
  5. on final approach course, maintain GPS track bar within ½ scale deflection;
  6. maintain airspeed within plus or minus 10knots of the declared approach speed;
  7. descend to and maintain the MDA and accurately track to the Missed Approach Waypoint (MAWP) or to minimum visibility so as to permit completion of the visual portion of the approach with minimal manoeuvering; and
  8. initiate the missed approach procedure, when the required visual references for the intended runway are not obtained at the MAWP.

10.31 Common Errors – Precision Approaches

10.31.1 Some common errors on Precision Approaches that may be observed and affect the assessment of the sequence are as follows:

  1. slow to react to ATC instructions or to instrument deviations, resulting in poor tracking of the localizer or glide slope;
  2. aircraft not stabilized and at the correct airspeed on the final approach and upon reaching DH;
  3. failure to monitor aircraft and ground equipment required for the approach; or
  4. using incorrect company procedures for the conduct of CategoryI, II or III approaches.

10.32 Circling Approaches

10.32.1 A circling approach shall not be conducted in weather conditions less than the minimum published in the CAP. If the candidate should lose sight of the intended runway of landing, he/she shall commence a missed approach in accordance with published procedures. If conducted in a simulator, the evaluator should question the crew on what procedure they plan to follow in order to conduct the circling approach.

10.32.2 Assessment will be based on the candidate/crew’s ability to:

  1. select and comply with the appropriate circling approach procedure considering the manoeuvering capabilities of the aircraft;
  2. confirm the direction of traffic and adhere to all restrictions and instructions issued by ATC or the check pilot; and
  3. stay within the visibility criteria and not descend below circling MDA until in a position from which a descent to a normal landing is assured.

10.32.3 Some common errors that may affect the assessment of this sequence are as follows:

  1. no briefing on the type of circling approach to be used;
  2. not designating which pilot will fly the circling approach;
  3. failure to monitor and inform the pilot flying of deviations in airspeed or altitude;
  4. exceeding 30° of bank or poor final alignment with the runway;
  5. gross upward deviations in altitude or circling below circling altitude; or
  6. not maintaining correct airspeed or failure to align aircraft with runway to effect a safe landing.

10.33 Go-Around

10.33.1 Begins when the crew aborts the descent to the planned landing runway during the Approach phase; ends after speed and configuration are established at a defined manoeuvering altitude or to continue the climb for the purpose of cruise.

Note: For the purposes of MV/LOE, one missed approach or one rejected landing is required per the schedules listed in CASS 725.106. These events both fall under the Go-around phase of flight.

10.33.2 A missed approach may be carried out at any time from intercepting final approach to touch down on the runway. The published missed approach profile must be followed except where it is modified by ATC.

10.33.3 Rejected landings may be carried out at any time after the instrument portion of the approach is complete, the runway is in sight and the aircraft is configured and has started its final descent to landing.

10.33.4Assessment will be based on the candidate/crew’s ability to:

  1. promptly initiate the missed approach;
  2. report beginning the missed approach procedure;
  3. comply with the published or alternate missed approach procedure;
  4. report anytime the aircraft is unable to comply with a clearance, restriction, or climb gradient;
  5. follow the checklist items appropriate to the go-around procedure;
  6. request a clearance to the alternate airport, clearance limit, or as directed by the check pilot; and
  7. maintain the recommended airspeed within plus or minus 10 knots; heading, track, or bearing within plus or minus 10 degrees; and altitude within plus or minus 100 feet during the missed approach procedure.

10.33.5Some common errors that may affect the assessment of this sequence are as follows:

  1. not utilizing adequate power/thrust settings and attitude to achieve a satisfactory climb profile;
  2. not following the published profile or ATC clearance;
  3. manoeuvering the aircraft inappropriately for the phase of flight;
  4. failure to ensure that required checks are completed;
  5. improper programming of FMS;
  6. not establishing or monitoring the missed approach guidance mode;
  7. missed approach altitude not set for auto flight system; or
  8. delayed or forgotten aircraft checks.

10.34Landing

10.34.1 Begins when the aircraft is in the landing configuration and the crew is dedicated to touch down on a specific runway; ends when the speed permits the aircraft to be manoeuvered by means of taxiing for the purpose of arriving at a parking area. It may also end by the crew initiating a Go-around phase.

10.34.2 Landings and approaches to landings must be conducted according to the applicable AOM, AFM, COM and SOPs. The actual landing and rollout must be assessed by the evaluator particularly when the candidates have undertaken a Level C or D training program.

10.34.3 Assessment will be based on the candidate/crew’s ability to:

  1. execute a landing from an approach MDA or DH when the required visual references for the intended runway are obtained;
  2. take action respecting NOTAMs, wind shear, wake turbulence, runway surface, braking conditions, and other operational considerations; and/or
  3. take into consideration weather factors such as turbulence, wind shear, wind, and visibility.

10.34.4Some common errors that may affect the assessment of this sequence are as follows:

  1. initiating the flare too early or too late;
  2. excessive body angle or roll on touch down;
  3. late or incorrect de-rotation rate;
  4. over controlling on short final;
  5. manoeuvering the aircraft inappropriately for the phase of flight;
  6. poor or no cross wind correction;
  7. improper use, or selection, of auto-brake;
  8. attempted landing without completing required checks; or
  9. failure to track the runway on rollout.

10.35Taxi-in

10.35.1 Begins when the crew begins to manoeuvre the aircraft under its own power to an arrival area for the purpose of parking; ends when the aircraft ceases moving under its own power with a commitment to shut down the engine(s). It may also end by the crew initiating a Taxi-out phase.

10.36Arrival/Engine Shutdown

10.36.1 Begins when the crew ceases to move the aircraft under its own power and a commitment is made to shutdown the engine(s); ends with a dedication to shutting down ancillary systems for the purpose of securing the aircraft. It may also end by the crew initiating an Engine Start/Depart phase.

10.37Post-flight

10.37.1 Begins when the crew commences the shutdown of ancillary systems of the aircraft for the purpose of leaving the flight deck; ends when the cockpit and cabin crew leaves the aircraft. It may also end by the crew initiating a Pre-flight phase.

10.38Flight Close

10.38.1 Begins when the crew initiates a message to the flight following authorities that the aircraft is secure, and the crew is finished with the duties of the past flight; ends when the crew has completed these duties or begins to plan for another flight by initiating a Flight Planning phase.

10.39Abnormal/Emergency

10.39.1 Abnormal procedures should be of sufficient complexity to allow each crewmember to demonstrate the handling of primary and secondary failures and paper checklist procedures appropriate to the aircraft type. In addition to the required engine failures, normally a minimum of two different systems malfunctions for each pilot is required to adequately demonstrate knowledge and ability.

10.39.2 Multiple, unrelated failures that have a cumulative effect on the operation of the aircraft must not be planned as part of a validation or evaluation scenario. For example, a configuration problem combined with a power plant failure have a cumulative effect requiring excessive work during the final approach and should not be simulated. Conversely, an emergency descent followed by a configuration problem or engine failure does not have a cumulative effect on workload during a single phase of flight and may be planned.

10.39.3 The evaluator shall not correct any unrelated malfunctions that are a result of crew actions.

10.39.4Assessment will be based on the candidate/crew’s ability to:

  1. demonstrate adequate knowledge to diagnose malfunctions of aircraft components or systems in a reasonable time and to take corrective action on those critical emergencies designated as memory checks in the applicable AOM, AFM, COM and SOPs without reference to a checklist or manual;
  2. demonstrate an understanding of alternate components, systems, procedures and any restrictions to continued flight predicated on their use and be able to develop a course of action that makes allowance for any further degradation in the aircraft airworthiness status; and
  3. demonstrate knowledge and discipline in the use of an electronic checklist, if applicable, and various alerting systems.

10.39.5Some common errors that may affect the assessment of this sequence are as follows:

  1. inability to identify a malfunction or incorrect diagnosis of the malfunction;
  2. inadequate knowledge of the procedures required to deal with an emergency, or failure to carry out vital actions in an acceptable time period;
  3. loss of situational awareness during the completion of required checklists or procedures;
  4. failure to correctly carry out secondary actions to determine limitations imposed by the emergency on the remaining systems;
  5. checks/procedures not in accordance with the applicable AOM, AFM, COM and SOPs;
  6. failure to carry out a vital action thereby jeopardizing the safety of the aircraft;
  7. exceeding aircraft or engine limitations; or
  8. improper electronic checklist and alerting system crew discipline.
Date modified: