Advanced Qualification Program Evaluator Manual TP 14672

CHAPTER 6 – MONITORING REQUIREMENTS FOR EVALUATORS

6.1 General

6.1.1 Notwithstanding the policies described in this chapter, TC reserves the discretion to conduct any LOE and any MV on any individual when deemed necessary. That being said, a Type E Evaluator can conduct another evaluator's LOE. Where possible, TC recommends that QAEs conduct the LOEs of supervisory or management pilots as well as Type E Evaluators, as a means to enhance quality assurance of the program.

6.1.2 Where an evaluator is authorized to conduct validations and evaluations on more than one aircraft type, the aircraft type on which the LOE referred to in paragraph 6.1.1 is conducted shall be at the discretion of the evaluator conducting the LOE. If there are large differences in the characteristics of the aircraft types for which AQP Evaluator Delegation of Authority is held, or if the evaluator conducting the LOE has any concerns pertaining to the evaluator's ability to operate an aircraft type, a LOE may be required on each aircraft type.

6.2 Type E Evaluator Recurrent Monitor Requirements

6.2.1 Subject to paragraph 6.2.5, TC Inspectors shall conduct an annual monitor on Type E Evaluators conducting a LOE. This monitor will be valid until the first day of the thirteenth month following the month in which the monitor was completed.

6.2.2 Where a Type E Evaluator monitor is renewed within the last 90 days of its validity period, its validity period is extended by 12 months.

6.2.3 The Issuing Authority may extend the validity period of a Type E Evaluator monitor by up to 60 days.

6.2.4Where the validity period of a Type E Evaluator monitor has been extended pursuant to paragraph 6.2.3 and the Type E Evaluator monitor is renewed after the initial expiry date, its validity is extended by 12 months calculated from the date the monitor was conducted.

6.2.5Where a Type E Evaluator is authorized to conduct validations and evaluations on more than one aircraft type, the aircraft type on which the monitor required by paragraph 6.2.1 is conducted shall be at the discretion of the Issuing Authority. If there are significant differences in the characteristics of the aircraft types for which AQP Evaluator authority is held, or if the Issuing Authority has any concerns pertaining to the evaluator's ability to conduct validations and evaluations on any aircraft type, a recurrent AQP evaluator monitor may be required on each aircraft type.

6.3 Type E Evaluator Monitor Procedures

6.3.1 The TC Inspector and Type E Evaluator shall meet prior to the LOE to establish the sequence of procedures to be demonstrated and to delineate the extent of the TC Inspector's input.

6.3.2 During an AQP evaluator monitor, the TC Inspector shall ensure that:

  1. the evaluator's report and the data collected is complete, accurate and in accordance with the air operator's approved AQP;
  2. where applicable, the evaluator's administrative procedures for the issuance of a type and/or instrument rating are in conformance with requirements specified in the Authorized Persons Training Program for AQP Evaluators;
  3. the evaluator covers the required event sets as per the script;
  4. the evaluation is conducted in a manner that is fair and in conformance with the standards and procedures described in this manual and in accordance with the air operator's Approved AQP; and
  5. the evaluator is acting within the limits of his/her authority and the air operator's approved AQP.

Note:Requirements of this paragraph are also checked during inspections and audits.

6.3.3 On completion of the simulator portion of the LOE, the TC Inspector and Type E Evaluator shall meet privately to reach agreement on the results of the Evaluation and the items to be covered in the debriefing. Where a disagreement exists between the assessments of the TC Inspector and Type E Evaluator, the TC Inspector's assessment shall take precedence and shall be used in the debriefing.

6.3.4After each AQP evaluator monitor, TC Inspectors shall complete an AQP Evaluator Monitor Report. A copy of this form can be found in AppendixD.

6.3.5TC Inspectors shall ensure that a copy of the AQP Evaluator Monitor Report is provided to the air operator and a copy placed on the evaluator's TC regional file.

6.3.6The Issuing Authority will ensure that the evaluator's electronic files located within NACIS are updated with the latest AQP Evaluator Monitor Report date.

6.3.7During recurrent AQP evaluator monitors, the TC Inspector may also review the air operator's utilization of evaluators.

6.3.8Where a Type E Evaluator fails to demonstrate the required level of competency during the monitor, the evaluator's monitor shall be deemed to have lapsed. Type E Evaluator privileges will be suspended until remedial training, as determined by the Issuing Authority, is completed and a subsequent monitor successfully passed.

6.4Type V Evaluator Recurrent Monitor Requirements

6.4.1 Subject to paragraph 6.4.5, TC inspectors shall conduct an annual monitor on Type V Evaluators conducting a MV. This monitor will be valid until the first day of the thirteenth month following the month in which the monitor was completed.

6.4.2 Where a Type V Evaluator monitor is renewed within the last 90days of its validity period, its validity period is extended by 12 months.

6.4.3 The Issuing Authority may extend the validity period of a TypeV Evaluator monitor by up to 60days.

6.4.4Where the validity period of a TypeV Evaluator monitor has been extended pursuant to paragraph 6.4.3 and the Type V Evaluator monitor is renewed after the initial expiry date, its validity is extended by 12 months calculated from the date the monitor was conducted.

6.4.5Where a TypeV Evaluator is authorized to conduct validations on more than one aircraft type, the aircraft type on which the monitor required by paragraph 6.4.1 is conducted shall be at the discretion of the Issuing Authority. If there are significant differences in the characteristics of the aircraft types for which AQP Evaluator Delegation of Authority is held, or if the Issuing Authority has any concerns pertaining to the evaluator's ability to conduct validations on any aircraft type, a recurrent AQP evaluator monitor may be required on each aircraft type.

6.5Type V Evaluator Monitor Procedures

6.5.1 The TC inspector and Type V Evaluator shall meet prior to the MV to establish the sequence of procedures to be demonstrated and to delineate the extent of the inspector's input.

6.5.2 During an AQP evaluator monitor, the TC inspector shall ensure that:

  1. the evaluator's report and the data collected is complete, accurate and meaningful;
  2. the evaluator covers the required manoeuvres and sequences as per the appropriate script;
  3. the validation is conducted in a manner that is fair and in conformance with the standards and procedures described in this manual and in Company SOPs; and
  4. the evaluator is acting within the limits of his/her authority.

Note:Requirements of this paragraph are also checked during inspections and audits.

6.5.3 Upon completion of the simulator portion of the MV, the TC inspector and Type V Evaluator shall meet privately to reach agreement on the results of the validation and the items to be covered in the debriefing. Where a disagreement exists between the assessments of the TC inspector and Type V Evaluator, the TC inspector's assessment shall take precedence and be used in the debriefing.

6.5.4After each AQP evaluator monitor, the TC inspector shall complete an AQP Evaluator Monitor Report. A copy of this form can be found in AppendixD.

6.5.5The TC inspector shall ensure that a copy of the AQP Evaluator Monitor Report is provided to the air operator and a copy sent to Transport Canada to be placed on the evaluator's TC regional file.

6.5.6The Issuing Authority will ensure that the evaluator's electronic files located within NACIS are updated with the latest AQP Evaluator Monitor Report date.

6.5.7Where a Type V Evaluator fails to demonstrate the required level of competency during the monitor, the evaluator's monitor shall be deemed to have lapsed. The air operator must advise Transport Canada of the failure and the intended remedial training. Type V Evaluator privileges cannot be exercised until remedial training has been completed and a subsequent monitor by a Transport Canada Inspector is successfully passed.

6.6Type O Evaluator Recurrent Monitor Requirements

6.6.1 Subject to paragraph 6.6.5, Quality Assurance Evaluators (QAEs) shall conduct a biennial monitor on Type O Evaluators conducting an OE. This monitor will be valid until the first day of the twenty-fifth month following the month in which the monitor was completed.

Note: The QAE who performs the duties described in 6.6.1 must hold a valid Type E or Type O AQP Evaluator Delegation of Authority

6.6.2 Where a Type O Evaluator monitor is renewed within the last 90 days of its validity period, its validity period is extended by 12 months.

6.6.3 The Issuing Authority may extend the validity period of a Type O Evaluator monitor by up to 60 days.

6.6.4Where the validity period of a Type O Evaluator monitor has been extended pursuant to paragraph 6.6.3 and the Type O Evaluator monitor is renewed after the initial expiry date, its validity is extended by 12 months calculated from the date the monitor was conducted.

6.6.5Where a Type O Evaluator is authorized to conduct evaluations on more than one aircraft type, the aircraft type on which the monitor required by paragraph 6.6.1 is conducted shall be at the discretion of the Issuing Authority. If there are significant differences in the characteristics of the aircraft types for which AQP Evaluator Delegation of Authority is held, or if the Issuing Authority has any concerns pertaining to the evaluator's ability to conduct evaluations on any aircraft type, a recurrent AQP evaluator monitor may be required on each aircraft type.

6.7Type O Evaluator Monitor Procedures

6.7.1 The QAE and Type O Evaluator shall meet prior to the OE to establish the sequence of procedures to be demonstrated and to delineate the extent of the QAE's input.

6.7.2 During an AQP evaluator monitor, the QAE shall ensure that:

  1. the evaluator's report and the data collected is complete, accurate and meaningful;
  2. the evaluator covers the required assessment items as per the appropriate OE strategy;
  3. the evaluation is conducted in a manner that is fair and in conformance with the standards and procedures described in this manual and in Company SOPs; and
  4. the evaluator is acting within the limits of his/her authority.

Note: Requirements of this paragraph are also checked during inspections and audits.

6.7.3 Upon completion of the flight portion of the OE, the QAE and Type O Evaluator shall meet privately to reach agreement on the results of the evaluation and the items to be covered in the debriefing. Where a disagreement exists between the assessments of the QAE and Type O Evaluator, the QAE's assessment shall take precedence and be used in the debriefing.

6.7.4After each AQP evaluator monitor, the QAE shall complete the appropriate monitor report form used by the air operator.

6.7.5The QAE shall ensure that a copy of that report is provided to the air operator and a copy sent to Transport Canada to be placed on the evaluator's TC regional file.

6.7.6The Issuing Authority will ensure that the evaluator's electronic files located within NACIS are updated with the latest report date.

6.7.7Where a Type O Evaluator fails to demonstrate the required level of competency during the monitor, the evaluator's monitor shall be deemed to have lapsed. The air operator must advise Transport Canada of the failure and the intended remedial training. Type O Evaluator privileges will be suspended until remedial training has been completed and a subsequent monitor by a QAE or Transport Canada Inspector is successfully passed.

6.8Approved AQP Evaluator Recurrent Training Program

6.8.1 All AQP Evaluators are required to attend an Approved Annual AQP Evaluator Recurrent Academic Training Course as outlined in the air operator's Evaluator Curriculum. This academic training program will include training on the use of Inter Rater Reliability (IRR) or Referent Rater Reliability (RRR).

6.8.2 The Approved Annual AQP Evaluator Recurrent Academic Training Course will be valid until the first day of the thirteenth month following the month in which the course was completed.

6.8.3 When the Approved Annual AQP Evaluator Recurrent Academic Training Course is renewed within the last 90 days of its validity period, its validity period is extended by 12 months.

6.8.4The Issuing Authority may extend the validity period of the Approved Annual AQP Evaluator Recurrent Academic Training Course by up to 60 days.

6.8.5Where the validity period of the Approved Annual AQP Evaluator Recurrent Academic Training Course has been extended pursuant to paragraph

6.8.4 and the course was completed after the initial expiry date, its validity is extended by 12 months calculated from the date that the course was conducted.

6.8.6Type E Evaluators are required to complete training annually on the duties and responsibilities of an "Authorized Person". The Authorized Persons training will be valid until the first day of the thirteenth month following the month in which the training was completed.

6.8.7A list of candidates attending the academic training program shall be forwarded to the air operator's POI for tracking purposes (entry into NACIS).

Table 6-1: Continuing Qualification of Evaluators
TypeE Evaluator
  • TC Inspectors shall conduct an annual monitor on Type E Evaluators conducting a LOE.
  • Annual AQP Evaluator Recurrent Academic Training Course including Inter Rater Reliability (IRR) or Referent Rater Reliability (RRR).
TypeV Evaluator
  • TC Inspectors shall conduct annual monitors on Type V Evaluators conducting a MV.
  • Annual AQP Evaluator Recurrent Academic Training Course including Inter Rater Reliability (IRR) or Referent Rater Reliability (RRR).
TypeO Evaluator
  • Quality Assurance Evaluators (QAEs) shall conduct a biennial monitor on Type O Evaluators conducting an OE.
  • Annual AQP Evaluator Recurrent Academic Training Course including Inter Rater Reliability (IRR) or Referent Rater Reliability (RRR).
Date modified: