Advisory Circular (AC) No. 107-001

Advisory Circulars logo

Guidance on Safety Management Systems Development

Table of Contents

9.0 COMPONENT 5 - QUALITY ASSURANCE PROGRAM

  1. A quality assurance program (QAP) defines and establishes an organization's quality policy and objectives. It also allows an organization to document and implement the procedures needed to attain these goals. A properly implemented QAP ensures that procedures are carried out consistently, that problems can be identified and resolved, and that the organization can continuously review and improve its procedures, products and services. It is a mechanism for maintaining and improving the quality of products or services so that, according to the Standards Council of Canada, they consistently meet or exceed the organization's implied or stated needs and fulfill their quality objectives (The Standards Council of Canada).
  2. An effective quality assurance system should encompass the following elements:

    1. Well designed and documented procedures for product and process control
    2. Inspection and testing methods
    3. Monitoring of equipment including calibration and measurement
    4. Internal and external audits
    5. Monitoring of corrective and preventive action(s), and
    6. The use of appropriate statistical analysis, when required

9.1 Quality Assurance General

In a SMS, the quality assurance program elements can be applied to an understanding of the human and organizational issues that can impact safety. In the same way that a QAP measures quality and monitors compliance, the same methods are used to measure safety within the organization. In the SMS context, this means quality assurance of the SMS, as well as quality assurance to ensure compliance to the CARs, Standards and procedures utilised by the organization.

9.2 PDCA

  1. Quality assurance is based on the principle of the continuous improvement cycle. In much the same way that SMS facilitates continuous improvements in safety, quality assurance ensures process control and regulatory compliance through constant verification and upgrading of the system. These objectives are achieved through the application of similar tools: internal and independent audits, strict document controls and on-going monitoring of corrective actions.
  2. As discussed in Chapter 1, most modern management systems follow the Plan, Do, Check, and Act (PDCA) cycle of continuous improvement. In this model, all of the individual processes in an organization are planned (PLAN), performed as planned (DO), reviewed to ensure use and effectiveness (CHECK), and modified as necessary to ensure that they are safe, effective and efficient (ACT).
  3. Simply stated, the Quality Assurance Program provides the CHECK component of PDCA and ensures that the ACT portion of the cycle achieves the desired results.

9.3 Focus on Process

It has been said that «the emphasis with assuring quality must focus first on process because a stable, repeatable process is one in which quality can be an emergent property.» This emphasizes the importance of focusing on process and on the need to ensure that processes are documented. The reason we need to do this is that in order to verify the effectiveness of a process, it must be used; in order to improve a process, we must be assured that the process we are improving was in fact the process that was originally being used. Remember, you cannot improve a process unless that process has been documented. So, what is meant by process? Process is the sequence of steps taken to arrive at a given output, and in the context used here, is the output from planning

9.4 Operational and System QA

  1. Operational and System QA are two distinct activities and are basic requirements of the Canadian Aviation Regulations. Operational QA verifies that all activities are being conducted in accordance with regulatory and organizational requirements documented in the appropriate approved manual, while System QA evaluates the overall effectiveness of the organization's SMS and the interaction of the individual processes within the organization.
  2. From TC's perspective, the purpose of Operational QA is similar to that of inspections and audits currently conducted by TC inspectors, specifically to provide assurance that the certificate holder is operating in compliance with regulatory requirements by following the processes documented in the appropriate manual. Subtle but very important differences are that the organization's operational QA will also look at non-regulatory activities and in addition, assess the presence, effectiveness and efficiency of existing processes and make recommendation for improvements. Follow-up of process changes resulting from corrective actions will also be a responsibility of the organization's QA.
  3. As mentioned above, System QA assesses the overall effectiveness of the SMS and from a regulatory standpoint, an organization is required to review or audit their SMS periodically and for cause. This System QA will typically be provided by a third party, or at a minimum, by personnel other that those assigned regular QA responsibilities. The reason for this is that QA, as a major component of the SMS, will be subject to scrutiny during this review in the same manner as all other SMS component/elements, and you can't have the auditors «auditing themselves». To maintain objectivity, persons not directly involved in the day-to-day operation of the SMS must conduct this activity.
  4. As the Operational QA capability of an organization matures, it is planned that TC will gradually back away from conducting operational level inspections and audits and focus more on assessing the overall effectiveness of the SMS. This activity will be similar to System QA and will be accomplished in accordance with guidance provided in TC's documentation relating to oversight.
  5. You will find that the distinction between Operational QA and System QA will lessen where internal audits begin to focus more on process than simply on results. This is especially true if SMS components/elements have been truly integrated into the existing management system.

9.5 Audits

  1. The use of audit functions, to verify compliance and standardization, is an integral part of the quality assurance system. An initial audit, covering all technical activities, should be conducted, followed by a recurring cycle of further internal audits. Detailed records of audit findings, including issues of compliance and non-compliance, corrective actions and follow-up inspections should be kept. The cyclical period for recurrent audits is not fixed (at this time) although it is generally accepted that all areas of the organization should be evaluated within each three-year period. The results of the audit should be communicated throughout the organization.
  2. Depending on the size of the organization, these functions may be performed by individuals within the organization or assigned to external agents. Wherever practical, having regard to the size of the organization, these functions should be undertaken by persons who are not responsible for, and have not been involved in, the certification or performance of the tasks and functions being audited. In this way, the quality assurance function remains neutral and is independent from the operational aspects of the organization.

9.6 Establishing an Internal Audit Program

  1. The first step in establishing your internal audit (evaluation) program is to develop the policy and procedures under which the program will operate. This policy, which will reside in the approved manual, or if developed, in an SMS Policy Manual that is cross-referenced from the approved manual, is the «higher-level» guidance that describes the QA program in general terms and is normally linked to regulatory requirements. Items included will typically be the commitment to having a QA program, a general description of the program including its purpose, position descriptions including qualifications and training, reporting responsibilities, declaration of the recurrent audit cycle, and reference to a procedures document that will exist outside of the approved manual. The reason for this is that audit procedures will be dynamic and are likely to change as the program itself is subjected to the PDCA cycle of continuous improvement, and you don't want to include this type of material in a document that requires TC approval each time you make changes.
  2. The procedures document will focus on the specific processes that will be used by QA personnel as they conduct their QA activities. There is ample reference material to guide you in the development of these processes. One source is the TC Inspection and Audit Manual and any of the reference materials such as the Aircraft Maintenance and Manufacturing Inspection and Audit Manual, Commercial and Business Aviation Inspection and Audit Manual or the National Aerodrome Safety Database (NASD). These sources will help you to identify audit specialty areas, prepare checklists, determine audit procedures and define the format and reporting requirements for audit findings and audit reports. Pay particular attention to the development of checklists during this phase of program development, as this is the principle means of identifying the processes that personnel are expected to follow (and will be audited to) for any given activity.
  3. There is also a wealth of information on quality auditing available from the International Organization for Standardization (ISO). Valuable information can also be obtained from the International Air Transport Association (IATA), specifically information pertaining to the Operation Safety Audit (IOSA).

9.7 Process versus Results Auditing

As previously stated, the distinction between Operational QA and System QA begins to lessen where audits focus more on process than on results. If you are auditing by specialty area (ex, training programs, operational control system, technical dispatch, defect rectification and control, etc.), and if audit checklists have been developed with reference to documented processes that include SMS component/element processes where applicable (training programs for example), then you are doing process auditing. Process auditing involves looking at an entire process including inputs and outputs and related requirements to determine a) if personnel are doing what they are supposed to be doing, and b) if by so doing, that the desired results are being achieved. This will also provide the opportunity to identify the absence of documented processes.

9.8 Checklists

  1. Audit checklists should be employed to identify all of the technical functions controlled by the approved manual. These should be sufficiently detailed to ensure that all of the technical functions performed by the organization are covered. Accordingly, the extent and complexity of these checklists will vary from organization to organization.
  2. In the case of a quality audit on an organization's SMS, the checklist should provide a detailed account of the following areas:

    1. Safety policy
    2. Safety standards
    3. Safety culture
    4. Contractor's safety organization
    5. Structure of safety accountabilities
    6. Hazard management arrangements
    7. Safety assessment, and
    8. Safety monitoring.
  3. Examples of detailed audit checklists are provided in TC's Inspection and Audit Manual and Maintenance and Manufacturing, Commercial and Business Aviation companion documents and NASD.

9.9 On-Going Monitoring

The on going monitoring of all systems and the application of corrective actions are functions of the quality assurance system. Continuous improvement can only occur when the organization displays constant vigilance regarding the effectiveness of its technical operations and its corrective actions. Indeed, without on-going monitoring of corrective actions, there is no way of telling whether the problem has been corrected and the safety objective met. Similarly, there is no way of measuring if a system is fulfilling its purpose with maximum efficiency.

9.10 QA Personnel

The quality of the QA program will, in the end, be determined by the quality of the personnel who do the QA work. You will want to ensure that personnel have the knowledge, experience and personal suitability to undertake QA tasks and that they have been provided with audit training such as the TC Audit Procedures Course or with industry courses such as the ISO Lead Auditors Course, the Canadian Standards Association or the International Air Transport Association (IATA) Audit Course.

9.11 Existing Systems

There are many existing quality assurance standards. The most appropriate system for your organization will depend upon the size and complexity of your operation. It should be tailored to meet your specific requirements. As with all components of the SMS it can be as simple or complex as you want and should be monitored to ensure it remains appropriate.

9.12 Role of QA

The role of QA can be summarized as follows:

  1. Identifies the processes that personnel are expected to follow for a given activity;
  2. Verifies that personnel are following the required processes;
  3. Validates the processes by ensuring that the desired outcomes are achieved;
  4. Identifies undocumented processes and processes that are ineffective and/or inefficient;
  5. Follows-up on processes that have been changed (corrective actions) to ensure that they are being used and are effective; and
  6. Provides senior management with the documentary evidence of the above activities.

9.13 How do you know if your SMS is working?

Component 5 - Quality Assurance Yes/No
A quality assurance program is established and maintained, and the program is under the management of an appropriate person  
There exists an operationally independent audit function with the authority required to carry out an effective internal evaluation program  
The organization conducts reviews and audits of its processes, its procedures, analyses, inspection and training  
The organization has a system to monitor for completeness the internal reporting process and the corrective action completion  
The quality assurance system covers all functions defined within the certificate(s)  
There are defined audit scope, criteria, frequency and methods  
A selection/training process to ensure the objectivity and competence of auditors as well as the impartiality of the audit process  
There is a procedure to record verification of action(s) taken and the reporting of verification results  
The organization performs a periodic Management review of safety critical functions and relevant safety or quality issues that arise from the internal evaluation program  
There is a procedure for reporting audit results and maintaining records  
There is a procedure outlining requirements for timely corrective and preventive action in response to audit results  

Previous Page

Next page

Date modified: