Appendix C - AQP DOCUMENTATION CHECKLIST AND REVIEW JOB AID TABLES

From: Transport Canada

Commercial & Business Aviation

This AQP tool contains seven job aids for both TC and the certificate holder to use as simplified checklists for the development and review of the documentation requirements of AQP. Other document configurations may be appropriate to a specific certificate holder. If a certificate holder adopts a different document configuration, other than the one suggested in the attachment to Policy Letter 169, then the applicant should provide clear and specific guidance as to the location of the information for each of these document topics. Certificate holders should include specific references where information can be found which addresses each item. The comment section may be used to record any remarks relative to the review and approval status of the document.

There are six document types and one annual report required for each AQP certificate holder. Each document will have its own job aid:
  1. Application – One per AQP air operator
  2. Job Task Analysis – One for each trainee type and one for each trainee type’s instructor/evaluators
  3. Qualification Standards – One for each trainee type and one for each trainee type’s instructor/evaluators
  4. Instructional Systems Development Methodology – One per AQP air operator
  5. Curriculum Outline – One per curriculum/make, model, series, variant and instructor/evaluator
  6. Implementation and Operations Plan (I&O Plan) – One per AQP air operator

    NOTE:  Each of the above documents must remain current throughout the life of the AQP. Each of the documents must utilize a revision control process.
Annual AQP Report – Certificate holders will monitor the status of all AQP curricula and the performance/proficiency Data Base and will summarize their findings annually in a report to TC. Although there is no established format for the report, the associated job aid highlights areas that should be addressed.

 

Application, Phase I.  The purpose of the application is to establish the applicant’s methodology for developing an AQP for all of its fleets, their instructors and evaluators. The application is submitted once and is updated as information in the application changes (e.g., a change in the transition schedule, adding new aircraft, etc.). In order to establish the applicant’s intent and approach for developing an AQP, the application should thoroughly discuss the following topics numbered 1 – 9 in this job aid.

 

 

1  Statement of Intent Y N Comments
  a. Does the Statement of Intent specify the applicant’s intent to develop, implement, and operate an AQP?      
  b. Does the Statement of Intent address all fleets?      
  c. Does the Statement of Intent address how and to what extent the AQP will be operated and maintained?      
  d. Does the Statement of Intent address how CRM will be integrated and measured?      
  e. Does the Statement of Intent address how LOS concepts will be integrated into both training and evaluation?      
2 The Applicant’s Staff Organization Y N Comments
  a. AQP Coordination: Is a person identified that will be the focal point for the applicant’s AQP development and contact with TC?      
  b. Subject Matter Expertise (SME): Are current and qualified individuals identified by name or position, who have varying levels of expertise that fairly represent the population of professionals the AQP will address?      
  c. Document and Curriculum Development: Are individual(s) identified by name or position, who interface with the AQP coordinator and the SMEs to develop the requisite AQP process, curriculum, and instructor/evaluator documents?      
  d. Document Management: Is an individual identified, who ensures AQP document control and congruence with TC approvals?      
  e. Computer Specialist/Database Management: Is an individual identified who will develop and manage the performance/proficiency data acquisition and analysis system?      
  f. In addition, will the computer specialist/database manager be used for other computer-related issues related to the facilitation of an AQP, such as electronic document review?      
3 Data Collection, Submission and Analysis Reporting Y N Comments
  a. Does the applicant acknowledge their understanding and acceptance of the AQP performance data requirements by stating the intended purpose for the collection, management, analysis, and reporting of AQP training/evaluation data for each curriculum?      
  b. Does the applicant defer to the I&O plan for describing the process and methodology for AQP data collection and analysis?      
  c. Does the applicant acknowledge that an electronic data management system will be developed prior to entering phase III of any AQP curriculum?      
  d. Does the applicant acknowledge that requirement for submitting de-identified data to TC no later than 2 months after collecting the data?      
  e. Does the applicant acknowledge the requirement of a more stringent collection and analysis of the data than that submitted to TC?      
  f. Does the applicant describe the purpose of the data analysis and how it will be used?      
  g. Does the applicant acknowledge the requirement for submitting an annual AQP report summarizing their data analysis and any resulting changes that ensued in their AQP program?      
4 Supporting Documents and Manuals Y N Comments
  a. Do the supporting documents and manuals list each make, model, and series aircraft or variant?      
  b. Has the applicant provided TC with the following documents or manuals:      
 

1.  A current listing of company and manufacturers manuals that govern company operations?

     
 

2.  General descriptive summary of each aircraft type, including aircraft configuration and the performance baseline?

     
 

3.  Flight Crew Operations Manual (FCOM)?

     
 

4.  Company Operations Manual (COM)?

     
 

5.  Manufacturer’s Aircraft Flight Manual (AFM)?

     
 

6.  Master Equipment List (MEL/CDL)?

     
  c. Does the applicant have available the training and qualification recommendations in the Flight Standards Board reports where applicable?      
5 Operating Environment Description Y N Comments
  a. Does the applicant describe the operating environment, including the general meteorological and geographic factors expected to be encountered during operations?      
  b. Does this description include the weather norms and extremes expected to be encountered in operations?      
  c. Does this description include normal, abnormal, and emergency equipment operation in geographic areas that require special procedures (e.g., engine failures in mountainous terrain)?      
  d. Does this description include terminal and enroute operating areas such as controlled and uncontrolled airfields?      
6 Trainee Demographics Y N Comments
  a. Does the applicant provide a general summary of trainee experience and entry level by aircraft make, model, series or variant?      
  b. Does the applicant identify entry requirements for ground and flight instructors and evaluators?      
  c. Does the applicant group students in terms of previous experience (e.g. with high, low and mean experience included)?      
  d. Does the applicant identify the current and anticipated need for replacement crewmembers by duty position? This information is necessary to determine priority on curriculum development.      
7 Training Equipment – Description and Location Y N Comments
  a. Does the applicant identify the training equipment to be used, its location, and identify the organization (vendor or applicant) responsible for its security and maintenance?      
  b. Does the applicant identify flight simulators and/or flight-training devices by make, model, serial number, and/or TC identification number?      
8 Facilities Description Y N Comments
  a. Does the applicant describe the location, general type of facility, classrooms, training aids, course software, and other resources to be used to support AQP training?      
9 Master AQP Transition Schedule (MATS) Y N Comments
  a. Does the MATS include all aircraft, aircrew, instructors and evaluators that the applicant intends to transition to AQP?      
  b. Is the MATS complete? A partial MATS is not acceptable.      
  c. Does the MATS address how currently qualified personnel may transition between traditional recurrent training & continuing qualification curricula?     A simple spreadsheet, provided by the applicant, may be helpful to highlight the crew pairing requirements during the first year of AQP operations.
  d. Does the MATS address how personnel who have completed initial, transition or upgrade curricula may enter a continuing qualification curriculum?      
  e. Does the MATS address personnel who have completed a traditional basic indoctrination curriculum, but have not completed an initial, transition or upgrade curriculum?      
  f. Does the MATS address personnel who are current instructors or evaluators and how they may transition to AQP via a differences course?      
  g. Does the MATS address the incremental implementation of the curricula as opposed to all at once?      
  h. Does the MATS provide the time frame necessary to withdraw from AQP if it becomes necessary to revert to the applicable CARS Part VII training program?      

 

 

Job Task Analysis (JTA),Phase II.  A JTA is the method or procedure used to reduce a unit of work to its base components. The JTA provides a detailed, sequential listing of tasks, subtasks, and elements with the knowledge, skill, and attitude characteristics (KSAs) that clearly define and completely describe the job. An applicant will provide a JTA for each make, model, and series aircraft (or variant). These may be submitted as individual listings, or a single-higher level listing with appendices for each aircraft, showing its unique lower-level features. As a complete document, the JTA has four components: a job task list, a learning analysis (KSA), identified crew positions, and references.

 

 

1 JTA Structure Y N Comments
  a. Does the JTA provide a general introduction explaining the development of the task listing and the subsequent task analysis and how it is to be used to form the basis for the qualification standards, and the AQP curricula upon which they are built?      
  b. Is the JTA organized using a hierarchical system with the flight phases at the top level, tasks at the next level, component subtasks at the next level, elements at the next level?      
  c. Is the JTA complete with tasks, subtasks, elements, and crew positions?      
  d. Has the applicant completed full development of the JTA to the element level?      
  e. In the JTA, are applicable knowledge, skills, CRM markers, and (where desired) attitudes applied at the element level?      
2 Flight Crew Training Requirements Y N Comments
  Does the JTA incorporate all knowledge and skill requirements currently specified in the regulations?        
  a.  Aircraft Systems’ Subjects      
 

Aircraft General

     
 

Equipment and Furnishings

     
 

Emergency Equipment

     
 

Powerplants

     
 

Electrical

     
 

Pneumatic

     
 

Air conditioning and Pressurization

     
 

APU

     
 

Hydraulics

     
 

Landing Gear and Brakes

     
 

Flight Controls

     
 

Fuel

     
 

Communications Equipment

     
 

Flight Instruments

     
 

Navigation Equipment

     
 

Autoflight

     
 

Warning and Detection Systems (to include TCAS/ACAS, GPWS/TAWS, and Weather Radar)

     
 

Fire and Overheat Protection

     
 

Oxygen

     
 

Aircraft Performance & Limitations

     
 

MEL/CDL

     
  b.  System-Operations Integration Training      
 

Pre-flight Visual Inspection

     
 

Pre-start Checklist and Procedures

     
 

Powerplant Start

     
 

Taxi to include lowest visibility allowed by OPS SPECS

     
 

Pre-Takeoff Checks and Procedures

     
 

Normal Takeoff

     
 

Crosswind Takeoff

     
 

Instrument Takeoff (Low Visibility)

     
 

Powerplant Failure on Takeoff
(at or near V1)

     
 

Powerplant Failure After V2

     
 

Rejected Takeoff

     
 

Area Departure

     
 

Cruise Procedures

     
 

Holding

     
 

Area Arrival

     
 

RNAV, RNP, SAAAR

     
 

Normal ILS

     
 

Engine-out ILS

     
 

Autopilot-Coupled ILS – CAT II/III

     
 

Non-precision Approach

     
 

ILS missed Approach

     
 

Second Missed Approach

     
 

Precision Radar Monitored Approaches/Missed

     
 

Circling Approach

     
 

No-flap Landing

     
 

Crosswind Landing

     
 

Landing With Engine Out

     
 

Landing From Circling Approach

     
 

Rejected Landing

     
 

Landing With 50% Power Loss

     
 

Approaches to Stalls

     
 

Steep Turns

     
 

Powerplant Failure

     
 

Windshear Training

     
 

Mis-Trim Situations / Trim Runaway

     
 

Selected Events - Unusual Attitudes

     
 

TCAS and GPWS - Escape

     
 

Normal and Abnormal Procedures

     
 

Emergency Procedures

     
  c.  Company Operations Manual Content      
 

Company Policy or Procedures (dispatch and flight release requirements)

     
 

Regulations, Operations Specifications, and Standards Operating Procedures (SOP)

     
 

Weather Requirements (seasonal changes, flight into various geographic locations and temperature-related requirements)

     
 

Hazardous Material

     
 

Security

     
 

Special Operations (special airports, special approaches and departures)

     
 

Emergency crew-assigned duties and procedures

     
 

Operation of emergency equipment/systems

     
 

Operation of ditching/evacuation equipment/systems

     
 

Crew Resource Management (CRM)

     
 

Emergency Situation Training – Rapid Depressurization, Fire (in flight/on ground), and Smoke Control Procedures

     
 

Assistance of Persons to Exits during an Emergency

     
 

Illness, Injury, or Other Abnormal Situations Involving Passengers or Crew (use of medical kit)

     
 

Flight Physiology (i.e., Hypoxia, Respiration, etc.)

     
 

Use of Checklist (SOP)

     
 

Cockpit Familiarization

     
 

Preflight Planning and FMS

     
 

In-flight Planning: LNAV, VNAV, RNAV and GPS

     
 

Required Navigation Procedures

     
 

Navigation Systems Integration

     
 

Autoflight and Flight Director Integration

     
 

Use of Radar/CRTs

     
 

TCAS/ACAS

     
 

GPWS/TAWS

     
 

Communication Systems Integration (ACARS/FMS/CPDLC)

     
 

Surface Movement Guidance Systems (SMGS)

     
 

Runway Incursion Prevention Strategy

     
 

Stabilized Approach Strategy

     
 

Precision Radar Monitoring (PRM) Procedures

     
 

Land and Hold Short (LAHSO) Procedures / Simultaneous Intersecting Runway operations (SIRO)

     
 

CAT II/III

     

 

 

Qualification Standards (QS), Phase II.  The qualification standards document has four parts:  The prologue, a regulatory comparison, the evaluation/remediation methodology, and the specific terminal proficiency objectives (TPO)/supporting proficiency objectives (SPO) qualification standard.
  1. Prologue:   This is an introductory section that explains the methodology, format, and terminology of the document.
     
  2. Regulatory Requirements Comparison: The qualification standards document must also include regulatory comparison information. The comparison must meet the requirement of AQP regulatory guidance, which states that the AQP program must indicate specifically the requirements of CARS Parts IV, VI and VII, as applicable, that would be replaced by an AQP curriculum. The comparison should be comprehensive and understandable so that a reader can discern the scope and appropriateness of the training.
     
  3. Testing/Validation/Evaluation & Remediation Methodology: This section is a detailed plan describing the point in the curriculum when a test, validation, or evaluation will be applied. It must identify what constitutes a failure and/or unsatisfactory performance. This section will also describe the remediation strategy to be used to correct unsatisfactory performance and special tracking provisions.
     
  4. The Qualification Standard: The qualification standard is constructed by applying a performance statement, conditions, and standards to a task or subtask, thereby creating a TPO or an SPO.

 

 

1 Qualification Standards Prologue Y N Comments
  a.  Does the prologue discuss the methodology used to develop the qualification standards document?      
  b.  Does the methodology explain how any aspect, from curriculum outline to lesson elements or grade sheet items, is to be traceable to an item in the qualification standard?      
  c.  Does the prologue discuss the format (structure) that is used for the qualification standards?      
  d.  Does the prologue define terms used for the qualification standards?      
2 Regulatory Comparison Y N Comments
  a.  Does the regulatory comparison specify the requirements of the applicable flight check standards and CARS Parts IV, VI and VII that would be replaced by an AQP curriculum and how they will be addressed?      
  b.  Are departures from those requirements identified and justified?      
  c.  Are any standards specifications used that differ from flight check standards?      
3 Testing/Validation/Evaluation & Remediation Methodology Y N Comments
  a.  Does the applicant describe where to ascertain how, when, where, and who will assess a student’s proficiency on each terminal and supporting objective?      
  b.  Does this section identify the points in the curriculum where the testing, validation, or evaluation will be applied?      
 
  • Systems Knowledge Validation
      
 
  • Procedures Validation
      
 
  • Manoeuvres Validation
      
 
  • LOE
      
 
  • OE
      
  c.  Does the applicant clearly define the different strategies that will be used to test, validate, or evaluate performance?      
 
  • First Look
      
 
  • Train To Proficiency
      
 
  • Systems Knowledge Validation
      
 
  • Procedures Validation
      
 
  • Manoeuvres Validation
      
 
  • LOE
      
 
  • Initial OE
      
 
  • OE
      
  d.  Does this section describe how the criticality and currency ratings translate into testing strategies for TPOs and SPOs in the continuing qualification cycle?      
  e.  Does this section describe how a TPO with several SPOs may be alternatively sampled over multiple evaluation periods or continuing qualification cycles? e.g., TPO (non-Precision Approaches) and corresponding SPOs (NDB, VOR, BC, etc.)      
  f.  Does the applicant specify and clearly describe the rating scale that will be used by instructors/evaluators to score performance?      
  g.  Do the rating scale definitions clearly discriminate performance levels? Are they clear?      
  h.  Does the applicant identify what constitutes a failure and/or unsatisfactory performance for each validation/evaluation point?      
  i.  Does the applicant specify the strategy for remedying unsatisfactory performance?      
  j.  Does this remediation strategy detail when and what may be repeated and whether or not additional training is warranted?      
  k.  Does the applicant describe the methodology that will be used to remediate unsuccessful testing, validation, or evaluation sessions?      
  l.  Does the applicant describe the level of training devices, simulators, or aircraft to be used to evaluate the proficiency objective at each point in the curriculum?      
  m.  Does the remediation strategy specify when no more training will be offered to the individual and the resulting actions such as “Referred to Director of Training,” “returned to previous position,” etc.?      
  n.  Does the remediation strategy describe the criteria for placing an individual on special tracking?      
  o.  Does the remediation strategy describe the strategy that will be used for an individual on special tracking?      
  p.  Does the remediation strategy describe what must take place for an individual to be removed from special tracking?      
4 Qualification Standards Y N Comments
  a.  Does each individual qualification standard contain the following:      
 
  • A header identifying the air operator and the document?
      
 
  • Page revision control dates and revision numbers?
      
 
  • Consecutive page numbers?
      
 
  • Phase of Operations: Number and title from task listing?
      
 
  • Qualification Standard Title: Either TPOs or SPOs?
      
 
  • Task or Subtask: Number and title from task listing?
      
 
  • Crew duty positions?
      
 
  • Criticality/Currency Rating: From the task factors analysis?
      
 
  • Curriculum:  This field identifies the curriculum(s) in which the task will be trained and evaluated.
      
 
  • Evaluation Strategy: The evaluation point for this particular qualification standard: e.g., train to proficiency, procedures validation, manoeuvres validation, LOE, or OE?
      
 
  • Media: The specific media in which training and/or evaluation will be conducted? (For qualification, the media is the lowest media used for final evaluation. For continuing qualification, the media is the range of media used for training.)
      
 
  • Performance Statement: An expanded statement of expected behaviour, which, when executed, will complete the work required for a specific portion of a job?
      
 
  • Conditions operational and environmental? Are the specific conditions to be used for the qualification curriculum specified?
      
 
  • Contingencies:  Are the specific contingencies to be used for the qualification curriculum specified?
      
 
  • Manoeuvre Standards: Are they specific and do they correspond to the standards listed in the flight check standards?
      
 
  • Procedural Standards: They may be specific or general. If they are specific, do they correspond to the standards listed in the manuals listed in the reference block? If general, do they reference information in a document or manual to chapter or section (page numbers are not required)?
      
 
  • References:  Do they identify the primary references from which performance statements and associated standards were derived? Do they cite documents by title and where applicable, chapter or section (page numbers are not required)?
      
 

b.  Are there any operations specifications requirements other than those listed above?

     

 

 

The Instructional Systems Development Methodology.  This document describes the approach to be used by applicant air operators to develop and maintain all Advanced Qualification Program (AQP) curricula. This document is divided into two sections. The first section, Development Procedures, describes the applicant’s approach for using the Job Task Analyses and Qualification Standards as baseline documents to construct their general training curricula across all AQP courses. The second section, Line Operational Simulation Methodology, describes the approach for developing line operational simulation (LOS) scenarios.

 

 

1 Development Procedures Y N Comments
  a.  Is the procedure for allocating TPOs and SPOs to lessons, selecting media and methods, and developing the curricula described?      
  b.  Does the applicant describe how enabling objectives (EO) are developed to support their higher-level objectives?      
  c.  Does the applicant describe how learning and evaluation activities are developed to support these objectives?      
  d.  Does the applicant describe how instructional media and methods are assigned to objectives?      
  e.  Does the applicant describe how objectives are clustered and sequenced into lessons, modules, segments, and curricula?      
  f.  Does the applicant describe how an audit trail will be maintained to link proficiency objectives, lesson activities/content, and test items?      
2 Line Operational Simulation (LOS) Methodology Y N Comments
  a.  Does the applicant describe how the typical scenario is constructed?      
  b.  Does the applicant describe how each event set relates to a phase of operation?      
  c.  Does the applicant describe how each event set consists of a series of proficiency objectives that include both technical and CRM activities?      
  d.  Does the applicant describe the use of event conditions, triggers, and distracters, as well as supporting events?      
  e.  Does the applicant identify possible sources of incidents that will elicit the behaviour required by the proficiency objectives selected for the scenario?      
  f.  Does the applicant define the basic success criteria for the LOS, and each event set within it?      
  g.  Does the applicant describe the scenario development process?      
 
  • Drafting - who will do the work?
      
 
  • Use of grade sheets?
      
 
  • Testing - who will be involved?
      
 
  • Training instructors or evaluators to administer specific LOS scenarios?
      

 

 

The curriculum outline. This is a listing of course content. It should be arranged from curriculum into segments, segments into modules, modules into lessons, and lessons into elements. Each part of the curriculum outline must clearly indicate the subject matter to be taught and correspond directly to the hierarchical system of the task analysis. A curriculum outline provides the basis for the curriculum footprint, which is a high level graphical overview of the curriculum content depicting the training and evaluation activities and the proposed hours for each day of the curriculum.

 

 

1 Curriculum Outline Y N Comments
  a.  Does the certificate holder have qualification and continuing qualification curricula for each duty position in each make, model, and series/variant of aircraft?      
  b.  Does the certificate holder have separate qualification and continuing qualification curricula for the instructors and evaluators?      
  c.  Does the certificate holder have any special curricula (transition, upgrade, requalification or refresher)?      
  d.  Is each curriculum constructed in the following order: curriculum, segment, module, lesson, and lesson element?      
  e.  Does the curriculum outline provide a level of detail that will allow the AQP applicant to make changes to the syllabus without submitting a new document for each syllabus change?      
  f.  Does the curriculum outline include the following:      
 
  • Operator’s name?
      
 
  • Type of aircraft?
      
 
  • Duty position(s)?
      
 
  • Title of curriculum and/or curriculum segment?
      
 
  • A listing of numbered (coded) objectives organized into lessons, modules, and segments?
     Numbers (codes) must allow TC to track objectives back to qualification standards and Job Task Analysis
 
  • An outline of each training module within each curriculum segment?
     Each module should contain sufficient detail to ensure that the main features of the principal elements or events will be addressed during instruction.
 
  • The checking and qualification modules of the qualification curriculum segment used to determine successful course completion, including any regulatory qualification requirements for crewmembers to serve in CARS Part VII operations (such as initial operating experience, OE, route and aerodrome familiarization)?
      
  g.  Does the curriculum outline indicate that it is part of the revision control system by page format?      
  h.  Do the curriculum outlines provide a hierarchical link (proficiency objectives) between the qualification standards and a curriculum?      
  i.  Does each part of the curriculum outline clearly indicate the subject matter to be taught and correspond directly to the hierarchical numbering system of the task analysis?      
2 Curriculum Footprint Y N Comments
  a.  Does the curriculum footprint describe the training and evaluation activities conducted each day of the curriculum?      
  b.  Does the curriculum footprint include the planned hours?      

 

 

Implementation and Operations Plan (I&O). This document is a milestone schedule detailing the transition to an AQP for flight crewmembers, instructors and evaluators, and a blueprint describing provisions for maintenance, administration, data management, and continuing quality control of curricula. The I&O Plan can be sectioned into two parts. The first part spells out how the operator proposes to implement the AQP. Included in this proposal is the schedule for the phase III training evaluation to include instructor/evaluator training and small group tryouts. It should also include provisos for evaluating the effectiveness of performance measurement tools, and provisions for evaluating facilities, courseware, and equipment before starting the plans for the small group tryouts. The second part explains how the certificate holder intends to operate the AQP in phases IV and V. Included in this section are strategies for maintaining the program, crew pairing policy, first-look administration, and instructor/evaluator requirements. The operations plan should also describe in detail the data management plan. This plan includes a statement of understanding addressing the collection and analysis of performance/proficiency data and a description of the performance/proficiency database (PPDB), the data management collection process, and the TC data submission, analysis, and reporting requirements.

 

 

1 Implementation – Phase III Y N Comments
  a.  Does this section include schedules for the implementation of each of the AQP curricula?      
  b.  Do the schedules correlate to the MATS?      
  c.  Do the schedules include dates for training the instructors/evaluators?      
  d.  Does the instructor/evaluator training include difference training for those previously qualified (if applicable)?      
  e.  Does this section include provisions for evaluating facilities, courseware, and equipment before starting the phase III training?      
  f.  Does this section describe a plan for evaluating and observing the instructors and evaluators during the phase III small group tryout?      
  g.  Does the instructor/evaluator training include provisions for evaluating the effectiveness of performance measurements such as the application of the rating scale, use of grade sheets and student feedback?      
  h.  Does this section indicate that the applicant will request no-jeopardy credits for the small group tryout course graduates in a separate letter addressed to TC?      
2 Operations Phases IV & V – AQP Maintenance Y N Comments
  a. Does this section describe the methodology for maintaining control of the AQP approval documents?      
  b.  Does this methodology include a procedure for providing document copies to the POI after receiving the approval stamp from the POI?      
  c. Does this section describe the strategy to be employed for curriculum maintenance and update?      
  d. Does the strategy for curriculum maintenance and update include plans for acquiring and measuring data for tracking curricula?      
  e. Does the strategy for curriculum maintenance and update identify the person(s) responsible for making changes in the AQP?      
  f. Does this section describe the strategy for monitoring and responding to demographic changes?      
  g. Does this section describe using training and evaluation feedback to maintain and improve the AQP?      
  h. Does this section describe how student and instructor feedback will be obtained?      
  i. Does this section describe any plans to upgrade training equipment?      
3 Operations Phases IV & V – First Look Manoeuvres Administration Y N Comments
  a. Does the applicant define first-look manoeuvres, their purpose, and the strategy that will be used to administer them?      
  b. Does this strategy indicate who will administer the first-look manoeuvres and at what point in the curriculum?      
  c. Does this strategy state that first-look will not be briefed prior to the first execution of these items?      
  d. Does this section describe how first-look manoeuvres are selected?      
  e. Does this section describe how the first-look manoeuvres would be updated?      
  f. Does the applicant describe how first-look manoeuvres will be analyzed to determine trends of degraded proficiency?      
4 Operations Phases IV & V – LOFT/LOE Crew Scheduling and Pairing Policy Y N Comments
  a. Does this section describe the circumstances that would require a seat substitute?      
  b. Does this section describe the decision rules that would apply in obtaining a seat substitute?      
  c. Do the decision rules that would apply in obtaining a seat substitute ensure that in all cases, the seat substitute must be task familiar with the duty position?      
  d. Does this section describe at what point in the curriculum that a seat substitute would be used?      
  e. Does the applicant acknowledge that all occurrences of seat position substitution including the qualification of the seat substitute must be reported?      
5 Operations Phases IV & V – Instructor/Evaluator Requirements Y N Comments
  a. Does the applicant identify by title each instructor or evaluator position?      
  b. Does the applicant describe the job function(s) that each instructor or evaluator is authorized to perform?      
6 Data Plan Y N Comments
  a. Does the data plan have a prologue that establishes the intended purpose and methods for the collection, management, analysis, and reporting of AQP training/evaluation data for each curriculum?      
  b. Does the prologue specify how the data plan will be maintained and updated?      
  c. Does the prologue acknowledge the air operator’s responsibility to collect and analyze more data than is required to be submitted to TC in order to adequately identify performance trends and make changes to factors that impact crewmember performance?      
  d.  Does the data plan address the methods (e.g., grade sheets, computer-input screens, etc.) used to collect performance/proficiency data for all curricula?      
  e. Does the data plan address data input quality control, security, and usability?      
  f.  Does the data plan address data management as the means and strategy the AQP air operator intends to employ to store, access, and assimilate the AQP performance/proficiency data that is collected?      
  g. Does the data plan address the type of software the data management system employs (e.g., relational database, spreadsheet etc.), the organization of the information in the electronic medium (e.g., database definition, database table relationships, spreadsheet description, etc.) and a description of the user interface to this data management system?      
  h. Does the data plan address the type of analysis it will employ to facilitate the AQP performance information needs of the air operator and TC? This discussion of the data analysis must address how each type of AQP data will be analyzed.      
  i. Does the data plan address TC data submission requirements including format and frequency?      
  j. Does the data plan address the type of data format it will employ for the reports, (e.g., tabular reports, graphs)?      
  k. Does the data plan address the frequency of the reports, both internal and TC?      
  l. Does the data plan identify the internal air operator personnel that will receive the reports?      
  m. Does the data plan include copies of all forms used for data acquisition and grading?      
  n. Does the data plan include a database description of data field types and a graphical depiction of the database table relationships?      
  o. Does the data plan address a quality assurance strategy for ensuring data integrity? Does this strategy include instructor and evaluator grading calibration?      

 

 

Annual AQP Report – Phases IV & V.  AQP requires that each AQP certificate holder prepare an annual AQP report. This report is based on the certificate holder’s analysis of the data that is collected during training and at strategic points (validation/evaluation gates) in each curriculum and maintained in the performance/proficiency database (PPDB). AQP requires data collection and analysis in order to establish and maintain quality control of curricula for flight crewmembers, instructors, and evaluators. The annual AQP report should summarize the lessons learned and adjustments made to the curriculum(s) during the reporting period. The report should also include projected or proposed changes to the curriculum(s) based on the certificate holder’s current analysis. The actual adjustments made to the AQP are reflected in revisions to the approved AQP documents.  The report should be submitted to TC no later than 60 days past the end of the report period. The reporting period is usually based on the authorization date for a particular curriculum in either phase IV or V. During AQP development, particularly for multiple fleet operators, with different authorization dates for multiple curricula, the reporting period may be modifiable as agreed upon by TC and the certificate holder. Copies of the report should be distributed the principal operations inspector (POI) at least 2 weeks prior to the annual AQP review meeting.

 

 

  Annual AQP Report Y N Comments
  a. Is there a prologue or introduction to the report that addresses how each type of AQP data was analyzed? This prologue should corroborate the information in the I&O Plan.      
  b. Does the report discuss data reliability and consistency?      
  c.  Does the report summarize the internal quality assurance reports as addressed in the I&O Plan?      
  d. Does the report validate the effectiveness of the AQP with supporting evidence of successful training and evaluation?        
  e. Is the report supported by TC analysis of the submitted data?      
  f. Does the report identify any trends, problem areas, and potential deficiencies that could result in decreased proficiency?      
  g. Does the report include a description of corrective measures taken and any resulting changes to curricula?      
  h. Does the report include any projected corrective measures to be taken and provide rationale for these changes?      
  i. Does the report indicate a need for changes to the AQP maintenance strategy as described in the I&O Plan?      
  j. Does the report indicate a need for changes to the certificate holders data plan as described in the I&O Plan?      
  k. Does the report provide an analytical comparison of data between equivalent periods in preceding years?      
  l. Does the report identify any future operational changes that will affect the AQP (operational changes or trainee demographics)?      
  m. Does the report analyze training and evaluation feedback as part of the collected data to determine the effectiveness of the training program?      
  n. Does the report analyze on-time completion rates for training curricula and initial operating experience?      
  o. Does the report analyze special tracking rates?      
  p. Does the report analyze instructor/evaluator (I/E) rater reliability training results?      
  q. Does the report analyze instructor comments as part of the collected data to determine the effectiveness of the training program?      
  r. Does the report analyze first look data?      
  s. Does the report analyze manoeuvres data?      
  t. Does the report analyze LOE data by technical topics, and CRM elements?      
  u. Does the report analyze OE data (exclusive of initial OE)?      
  v. Does the report address progress towards phases III, IV, & V in other fleets (as applicable)?      
  w. Does the report address seat substitution rates?      
  x. Does the report address recordkeeping?      
  y. Does the report address adherence to the I&O Plan?      
  z. Does the report address the validity and usefulness of the qualification standards?      
  aa. Does the report address internal audit or TC surveillance findings?      
  bb. Does the report address related safety programs implemented by the air operator (i.e., flight data monitoring / operational safety auditing)?      

 

 

 

Report a problem with this page

Date modified: