Chapter 5: Safety Management Systems
2. State of Rail Safety in Canada
4. Regulatory Framework
5. Safety Management Systems
6. Information Collection, Analysis and Dissemination
7. Proximity Issues
8. Environmental Protection and Response
9. Operational Issues
10. Scientific and Technological Innovation
12. Building Relationships
To order a copy
The Railway Safety Act (RSA) requires railways to implement and maintain a safety management system (SMS), which is defined as a formal framework for integrating safety into day-to-day railway operations. SMS is a modern, flexible and efficient regulatory approach that aims to improve rail safety in Canada. Throughout the consultative process, the Panel heard many opinions about SMS.
This chapter outlines the history of, and rationale for, the SMS approach, assesses its implementation, and addresses the importance of an effective safety culture, oversight and risk assessments to SMS.
5.1 The SMS Concept
The concept of safety management systems grew from an evolution in thinking about safety practices and the causes of accidents during the 1990s. The original Railway Safety Act did not set out requirements for safety management systems. Rather, SMS Regulations were added as part of the 1999 amendments to the Act.
Traditionally, in rail and in other safety-critical industries, safety had been pursued through compliance with prescriptive rules and regulations. In the 1990s, however, advancements in safety research demonstrated that organizations could be compliant with prescriptive regulations, yet still be unsafe. More specifically, compliance did not necessarily mean effectively managing risks.
At the time, researchers and safety managers were also working to understand human behaviour in the context of accidents. In 1990, James Reason presented a now well-known model of accident causation (the Swiss Cheese model) that explained how human beings contribute to the breakdown of complex, interactive and well-guarded systems, such as rail transportation.
Figure 5.1: Reason's Model of Accident Causation
According to Reason, most accidents can be traced to one or more of four types of failure: organizational influences, unsafe supervision, preconditions for unsafe acts, and the unsafe acts themselves. These can be characterized as latent (underlying) or active conditions.
In the "Swiss Cheese model," defences against failure within an organization can be considered as a series of barriers, which are represented as slices of swiss cheese. The holes in the cheese slices represent individual weaknesses or even breaches in individual parts of the system, which continually vary in size and position in any of the slices. The system as a whole produces a failure when a series of holes in each of the slices momentarily lines up, allowing what Reason describes as "a trajectory of accident opportunity," so that a hazard passes through all of the holes in all of the defences, leading to a failure.1
Reason's model provides an understanding of how humans contribute to the breakdown of complex systems. Most importantly, the model demonstrates that the whole system must be considered when evaluating safety performance. With this new understanding of accident causation, it became clear that the traditional prescriptive approach to regulatory oversight alone was insufficient for preventing accidents.
Concurrently, transportation regulators realized that as traffic volumes increased, the total number of accidents would increase, even if the accident rate remained the same (i.e., the number of accidents per level of activity). Under an exclusively prescriptive regulatory approach, this would have required a significant injection of resources for regulatory oversight, simply to maintain or further reduce the total number of accidents. Regulators recognized, as well, that projected shortages of technical personnel in the industry would make it difficult to recruit the staff necessary to sustain a traditional regulatory oversight model.
Also in the 1990s, the Government of Canada was evolving from the owner-operator of major portions of the transportation system to the regulator and policy-maker. Increasingly, safety depended upon a partnering approach - with industry responsibilities focussed on the safety of the operations and the regulator focussed on a safe national transportation system. As the railway industry continued to grow and evolve, there was an even more pressing need to apply modern safety practices.
Of course, risk had always been a part of transportation systems, and those charged with managing safety began to conceptualize a system where hazards were identified and assessed, and the resulting risks were then managed proactively. Lessons learned from accidents, incidents and day-to-day operations would be injected into the system, thus leading to "continuous safety improvement."
While this evolution in thinking about safety, accident causation, and regulatory oversight was occurring, the 1994 review of the Railway Safety Act occurred. It was during that review that the concept of safety management systems for railways was born and, indeed, came to be seen as a way of regulating more effectively. As a result, amendments to the Railway Safety Act were introduced in 1999 that added requirements for railway companies to develop and implement safety management systems.
- Improved decision making
- Learning about operations
- Improved safety performance
- Customized mitigation strategies
- Possibly exceeding safety standards set by regulation
- Improved public and customer confidence
- Increased competitive advantage
- Demonstrated due diligence
- Potential for reduced regulatory oversight
- Enhanced relationships and collaboration
- Improved economic performance
The key for railway companies was to become more proactive, to refine their abilities to identify hazards, and to assess and mitigate risks. The need for companies to build a safety consciousness into their day-to-day operations was of paramount importance. This represented a shift from the traditional reactive approach of considering what had happened in a post-accident environment. As railway companies adopted the SMS concept, they began to fully realize the benefits that can be derived.
For example, companies can profit from improved decision making on safety-related issues and can learn more about their operations through the higher level systems perspective that the SMS approach offers.
They can achieve improved safety performance and customize mitigation strategies to their own operations, which is especially important in the case of smaller operators and short lines. This means mitigation can actually exceed standards set by regulation. In the end, higher public and customer confidence result.
Companies also benefit from an increased competitive advantage and can demonstrate that they are constantly taking safety into consideration in their decision making. There is significant potential for reduced regulatory oversight, and improved relationships, partnerships, and collaboration.
A strong SMS can lead to economic benefits because safety and economic performance are linked. There are direct and indirect cost savings when accidents are prevented, because accident clean-up is costly and shutdowns cause lost revenues. In short, safety is good for business.
Evidence of the economic rationale for SMS is provided by companies that report spending fewer resources because they solve problems much earlier and avoid costly abnormal operations. In fact, in aviation, some companies report significantly improved economic performance because the implementation of safety management systems has helped to avoid costly abnormal operations (e.g., late flights, passenger compensation), which are associated with accidents and incidents.
When the railway SMS Regulations came into force on March 31, 2001, they were the first of their kind in the federal Canadian transportation sector. They were created with significant industry input and placed the responsibility for managing the safety of operations on the railways themselves. They were not intended to replace existing regulations, rules or standards, but to develop a more comprehensive way of managing safety by complementing the existing regulatory framework.
Under the SMS Regulations, railway companies must implement and maintain a safety management system plan that includes a safety policy with annual safety targets and initiatives to meet those targets. There must be clear responsibility for safety at all levels of a company and a means to involve employees in safety management. Systems for identifying and showing compliance with applicable rules and regulations are required. A process for identifying hazards, and assessing and mitigating risks must be in place. Processes and procedures for accident reporting and investigation are also required. Methods to ensure that employees are appropriately trained must exist. Procedures for data collection and analysis and periodic internal safety audits are required. Finally, there are requirements for monitoring corrective actions and consolidating documentation.
This framework must be in place to achieve a systems approach to managing safety - one that embodies taking action before accidents occur.
5.2 Implementation of Safety Management Systems
It has been nearly seven years since railways have been required to implement SMS. While progress has certainly been made, in the Panel's opinion, the implementation of SMS across the rail transportation system and by the regulator has been inconsistent. The Panel expected that, after so many years, both the regulator and the industry would have made more progress.
During public consultations, the Panel received many submissions that focussed on SMS implementation and related topics, such as safety culture, appropriate oversight and risk management. Those with the most to say about SMS were the railways (Class 1s, short line, passenger, and commuter), Transport Canada, and union representatives. Railway employees largely had less to say because many told us they were unaware of SMS or had not been trained in its objectives.
While much progress has been made, most employees have only a cursory awareness of [the] existence [of SMS] and what it means to them.
CAW-TCA Canada, Submission, page 12.
Generally, stakeholders thought that SMS was the right approach, but many submissions tended to support the view that improvement is needed before SMS could be considered to be fully implemented.
We were also made aware of misunderstandings about the intent of SMS. Some stakeholders were under the impression that SMS would replace regulations but the Panel understands that SMS was never intended to be de-regulation or industry self-regulation. Rather, an effective SMS depends on both the industry and the regulator working to better manage the risks inherent in the system and to improve safety performance on a continuing basis.
Independent research commissioned by the Panel also found issues with the implementation of SMS across the country. Maturity of SMS plans varies widely across companies, with progress being remarkable in some companies and uneven in others. The weakest component in SMS plans appears to be in the management of human and organizational factors, rather than in respect of technical or equipment aspects.
The integration of SMS into Transport Canada's regulatory oversight program for rail safety has been inconsistent. In the Panel's opinion, clear direction and support are required from national headquarters to overcome inconsistent approaches to delivery throughout Transport Canada's five regions. Nonetheless, research confirms that SMS offers a significant advantage over traditional, exclusively prescriptive, regulatory models,2 but there remains disagreement about the extent to which SMS has been successfully implemented across the system.
The Panel supports the safety management system approach and recommends that both the railway companies and Transport Canada focus their efforts to improve its implementation.
We now turn our attention to understanding how implementation can be improved in the areas of safety culture, oversight and risk assessments.
5.3 Safety Culture
The cornerstone of a truly functioning SMS is an effective safety culture. The Panel views such a culture as one in which safety values are firmly entrenched in the minds of managers and employees at all operational levels, and respected on a daily basis in the performance of their duties. It is demonstrated by the decisions, actions and behaviour of individuals.
Reason's Elements of an Effective Safety Culture
- A just culture (with an atmosphere of trust and clear understanding of acceptable and unacceptable behaviour);
- A reporting culture (where people report their errors and near-misses);
- A flexible culture (which adapts to changing demands);
- A learning culture (which implements the reforms needed to make the system safer); and
- An informed culture (which has current knowledge).
An effective safety culture is one where past experience is not taken as a guarantee of future success and organizations are designed to be resilient in the face of unplanned events. Open communication and fresh perspectives are encouraged, and managers and employees at all levels are involved. New and ongoing practices and procedures are regularly compared, reviewed and improved. Human error is treated as a possible indication of broader, organizational influences.
There is an investment in safety, and the regulator and industry work together towards continuous improvement.
The success of a safety management system depends on effective communication and information sharing at all levels in an organization - from senior managers to frontline workers. The Panel believes that there is a vital role for railway employees and their representatives to play in implementing successful safety management systems.
Safety Management Systems must be built from the ground up, dedicated to detecting hazards and controlling them.
United Steelworkers Submission, View From The Track, page 16.
Employees can be a company's prime source of information for the identification of hazards and assessment of mitigation strategies. The Panel heard from many railway employees who felt neither involved nor informed about their company's safety management system. Rather, employees often described their organizational culture in such a way that the Panel could not reconcile it with an effective safety culture.
[Changing culture] is a journey; the progress we have made is still fragile. There are wide disparities within CP on acceptance and use of this approach and the various "tools" that have been introduced. And there is much more work to do. But generally, we are trying to move from a culture that blames the individual who ultimately makes the final error in the chain of accident causation, to one where we ask system-based questions such as: What defenses failed? How did they fail? How can the system be made more resistant?
Faye Ackermans, General Manager, Corporate Safety and Regulatory Affairs, CP, Statement to U.S. House of Representatives, Committee on Transportation and Infrastructure (October 25, 2007), pages 5-6.
SMS requires drastic cultural change for both the regulator and the regulated. The Panel recognizes that culture change is a long-term endeavour and no easy task. It requires the simultaneous building of new values with the destruction of old ones. It can be easier to change practices, with the associated values and culture changes eventually following naturally. Additionally, in relaying their experience with SMS, representatives of some transportation companies told us that, in some cases, building the culture necessary for effective implementation of SMS meant sweeping changes at their management levels.
5.3.1 Culture Change in the Railway Industry
Among major rail companies, VIA Rail has a respected SMS system and entrenched safety culture. In part, this is because it is a passenger-carrying railway and the market demands safe transportation, but the Panel also noted that VIA takes safety management seriously by making it important to everyone in the company.
In the Panel's opinion, CP is making great strides in adopting the kind of safety culture required for a successful safety management system. We were very impressed with CP's approach to occupational health and safety committees and the role that these committees play in safety management. In particular, the Panel applauds the engagement of a health and safety committee member in various CP accident and incident investigation protocols. These are steps in the right direction.
CN also made a positive first step in appointing a Chief Safety Officer in April 2007. The Chief Safety Officer requires the complete support of the senior management team to succeed, and all of the management group will need to be actively involved in inculcating the values and beliefs of an effective safety culture. In the Panel's opinion, CN's current day-to-day management of safety must evolve to the healthy safety culture necessary for a successful safety management system. With some exceptions, employees recounted a culture based on fear and discipline.
Based on what we heard throughout the Review process, there appears to be a serious disconnect between CN's stated objectives and what is occurring at employee levels. CN manages safety through an "antecedent, behaviour and consequences" process, which the Panel feels is constructed as a traditional rule and discipline model.
While rules certainly have had a positive impact on safety, rules alone may no longer be the most appropriate approach, given the modern understanding of accident causation. As noted earlier, a company can be in total compliance with prescriptive regulations, yet not necessarily be safe.
Further, current thinking about safety has evolved beyond designing safe processes and automating the human element necessary within these processes through rules.
Accidents were … analyzed up to the point where it became clear that someone had broken a rule (at which point discipline was appropriate) or that there was no rule for this eventuality (in which case a new one was made). In this way rulebooks continually grew and never diminished. … Ultimately, we get a rule for everything and safety is seen as something [that] requires no thinking any longer, but simply good training, a prodigious memory, a large safety manual or computer to refer to, and an iron discipline. Management does not need to do any more thinking or planning, because it is all fixed in the rule system.3
We also heard that a strict, rules-based system lays blame on employees for errors or failures, but fails to sufficiently recognize the management influences or organizational situations that may be contributing to those errors or failures. The Panel agrees.
CN's attitude towards safety seems to be "blame and punish" instead of "educate and correct." … Frequently, employees involved in accidents … are simply blamed for errors without follow up or root cause investigation. They are then punished without any other corrective action taken on the part of the railway to prevent reoccurrences.
Sylvia LeBlanc, Submission, page 1.
This is not to say that there is no need for rules or discipline for "intentional bad behaviour," wilful negligence or criminal activity in the rail industry. There certainly is such a need. A real or perceived over-reliance on discipline as the consequence of most actions is problematic in an effective safety management system. The Panel sees such an over-reliance as a culture where strict adherence to rules is achieved primarily through discipline or a threat of potential discipline. Disciplinary cultures have a tendency to instil fear, and to stifle employee participation and reporting. A significant mistrust of management develops. People stop communicating - and that can have a detrimental impact on safety.
In the Panel's opinion, over-reliance on discipline does nothing to support healthy management-employee relationships so vital to an effective safety management system. Such relationships must be built on openness and trust and this is difficult or impossible to instil in an environment where employees are constantly fearful of disciplinary action.
It is noteworthy that Air Transat has implemented a reporting system that balances open (though not anonymous) reporting of risks with appropriate discipline. This system is based on a formal understanding between management and employee representatives that provides immunity from corporate disciplinary measures (though not from regulatory or legal penalties) for those who report safety-related information. It has resulted in Air Transat employees feeling comfortable to report risks without fear of being disciplined, and it links directly to safety benefits.4
At VIA, as at most railways, there are certain "cardinal rule" violations where discipline is necessary, but VIA also has processes in place aimed at building openness and trust between managers and employees. For instance, employees are observed at regular cycles, and corrective coaching takes place immediately when errors are observed.
The Air Transat and VIA examples demonstrate that it is possible to have an effective safety management system based on a balance between openness and reporting, and appropriate discipline.
Recognizing that railways are at different stages of implementing SMS and, notwithstanding the challenges posed by effecting the culture shift needed to derive maximum benefit from an SMS, the Panel feels that this culture shift is the cornerstone to implementing truly effective and efficient safety management systems. Resources and commitment will be required to implement such a cultural change.
Transport Canada, Rail Safety Directorate and the railway industry must take specific measures to attain an effective safety culture.
5.3.2 Employee Involvement in Occupational Health and Safety
As we noted earlier in the report, safety in rail transportation is not governed by the RSA alone. Human Resources and Social Development Canada (HRSDC) administers the health and safety of workers in federally regulated workplaces, including railways under federal jurisdiction, under Part II of the Canada Labour Code (CLC-II). For on-board employees, this responsibility is delegated to Transport Canada, Rail Safety. HRSDC maintains responsibility for so-called "off-board" employees, such as those performing track maintenance and car and locomotive repairs.
We understand that the working relationship between Transport Canada and HRSDC in general is very good - that communications are effective, and responsibilities and accountabilities are clear. It is also essential that the local occupational health and safety committees for railway employees, required under the Labour Code, function effectively and share information and feedback that contributes to overall railway safety.
As discussed in Chapter 4, the occupational health and safety provisions of CLC-II require every employer to establish a workplace health and safety committee for each workplace that has 20 or more employees. The committees are responsible for health and safety matters that apply to those individual workplaces. Management and employees participate in the committees, and in unionized workplaces, representation of employees is through the unions involved. The Code also requires employers to appoint a health and safety representative for each workplace with fewer than 20 employees. Companies that directly employ 300 or more are also required to establish a policy health and safety committee, which has a broader policy, planning and monitoring mandate.
We were made aware of very active health and safety committees in several of the larger railway companies, and we had the opportunity to meet company and union committee members in different parts of Canada. It is clear that, like all collaborative mechanisms, when these committees are functioning well - when their members are engaged and committed, when training is adequate, when attendance is regular, and when management is responsive - they are extremely valuable for sharing information about safety practices and concerns, and in providing feedback to management and employees. The committees can be very effective in providing a formal mechanism for identifying concerns and for establishing a time frame within which to directly respond and resolve problems. They are an outstanding tool for managing safety, involving employees and building an effective safety culture. They should be an essential element of a safety management system.
Workplace health and safety committees, and the policy committees in larger companies, should involve employees in identifying hazards and assessing and mitigating risks in their own workplaces. This is not to suggest that the effective use of health and safety committees can satisfy all aspects of a company's SMS - after all, as we note throughout this report, the framework for railway safety contemplated under the RSA is broader than individual workplaces. Nevertheless, the structure that the committees provide, and the relationships that are developed within it, can contribute to an overall spirit of collaboration and an atmosphere of mutual trust and respect. These help to create an effective safety culture essential for implementation of safety management systems.
The industry must take every appropriate measure to ensure the effectiveness of local occupational health and safety committees. Specifically, they should involve employees in identifying hazards, and assessing and mitigating risks as part of safety management.
5.3.3 An Evaluation Tool for "Safety Culture"
A practice to determine where a company (and, indeed, the regulator) stands in terms of implementing an effective safety culture may be to use a measuring tool that categorizes where a company is situated along a continuum to full implementation of SMS. One such model,5 developed for the aviation industry, contains certain components that could be applied to the rail industry.
At one end of that continuum is a company that complies with minimum safety standards and views compliance as a cost of doing business. That company minimizes compliance expenditures and operates from a short-term perspective, addressing problems only after it has been caught in violation. The regulator must engage in significant surveillance and enforcement activities.
Next in the continuum is a company that views safety solely as compliance with current safety standards. Such a company has internal inspection and audit processes, as well as a system of reward and punishment. There is an assumption that compliance translates into safety, but such a company has not yet realized that compliance alone will not necessarily prevent an accident from happening. Intervention is still required from the regulator, though the approach may be more educational in nature.
At the third stage along the continuum is a company that sees safety as risk management and recognizes that compliance alone cannot guarantee safety. This company is anticipatory and identifies the potential for hazards before they occur. The regulatory approach must evolve from compliance inspections to system audits.
At the next stage is a company that views safety as an opportunity. This company leverages its safety management capability to its economic benefit. It has a longer-term outlook and proactively seeks to include safety in its business and operational decision-making processes. The regulator's role is primarily one of monitoring the company's safety performance.
Finally, at the advanced end of the continuum, is a company that has fully integrated safety into its business practices. Safety is reflected in core values and built into the business model. Again, the regulator's role is one of monitoring.
This safety culture continuum demonstrates that the shift to an effective safety culture is an evolution. Transport Canada could help companies to identify where they fall along this continuum.
The Panel recognizes that changing culture is not easy to achieve, but feels that this is the foundation upon which effective railway safety management systems will be built.
5.3.4 Culture Change in Transport Canada
Culture change is also required on the part of the regulator. Transport Canada recognizes that it is facing its own challenges in this respect. The department's recent publication (April 2007) entitled Moving Forward: Changing the safety and security culture, identifies one of the key challenges as demonstrating the impact of safety management on performance.
In the Panel's opinion, and as illustrated by the continuum, a shift in thinking will also be needed by the entire Transport Canada, Rail Safety organization. To effectively manage an SMS oversight model, the regulator will need to recognize the industry's primary responsibility for safe operations. Transport Canada's regulatory oversight program must be designed while bearing in mind where the greatest risks lie in the rail system. Success will need to be measured based on safety performance results, rather than simply the number of regulatory interventions.
Developing the capability to provide effective oversight of safety management systems, and investing the appropriate human and financial resources to ensure its success will also need to be addressed.
Training and development of Transport Canada employees must support the culture shift needed for effective and efficient oversight of SMS in industry. Traditionally, Transport Canada railway safety inspectors were trained in forensic investigative techniques to monitor compliance with existing rules and regulations. This training was appropriate for a time when the focus was on investigations to measure compliance and non-compliance, but this focus has shifted to a systems-based audit approach, required under SMS.
In the traditional model, inspectors were used to dealing directly with their peers in industry. Under SMS, "inspectors are called upon to intervene at a more strategic level and are required to interact with system managers whose motivations, contingencies, views, frame of reference, and language may be completely new to them."6
Unfortunately, despite the culture change necessary at Transport Canada since the inception of SMS, it is the Panel's opinion that the resources provided are inadequate to inculcate the culture and skill sets required to effectively manage and oversee SMS in industry.
We are concerned, for instance, that Transport Canada's administration of the Railway Safety Act SMS Regulations and audit program are treated as an "add-on" by the department, and have not been well integrated with the existing regulatory oversight program. Instead, traditional functional groups continue to operate separate from SMS-focussed inspectors and program groups. It is important for Transport Canada, Rail Safety Directorate to design its organization to support its oversight of railway SMS plans as its central regulatory oversight activity.
Transport Canada, Rail Safety Directorate should be organized so as to better integrate safety management systems as the key focus of its oversight activities.
Transport Canada also needs to accelerate the transition from inspections to audits. As discussed later in the chapter, there are several changes that Transport Canada needs to bring about to improve its audit regime. These improvements will lead to a more appropriate safety culture. This will require new resources, skills and training for Transport Canada personnel.
The Panel recognizes that inspections and audits are two very distinct functions, each requiring unique skill sets. We are of the opinion, however, that training an individual to perform both of these functions would accelerate the culture change required to oversee an SMS approach. The Panel also feels that changing the label associated solely with an inspection regime would be a step in the right direction.
In order to better reflect the fact that the current railway safety inspector (RSI ) performs both inspections and audits, the title should be changed to Railway Safety Officer.
5.4 Oversight of Railway Safety Management Systems
A key for making SMS work is an appropriate oversight system. The philosophy in the Railway Safety Act that makes railway companies responsible for ensuring the safety of their own operations means that the regulator assures compliance through performance- based oversight, rather than prescriptive enforcement. Under a prescriptive regulatory model, the regulator inspects industry with the view to identifying non-compliance with the rules and regulations. The oversight system required under SMS is fundamentally different from this approach. It requires ongoing monitoring and periodic audit of safety performance (though it does not replace inspections, and enforcement actions when warranted).
Gary Moser and Doug Lewis, Fraser River Valley, British Columbia, May 2007
Before a company's SMS is audited, however, it must first be submitted to Transport Canada. New railway companies are required to obtain a Certificate of Fitness from the Canadian Transportation Agency in order to begin operations, and as part of this process, a potential operator is informed about SMS requirements. The operator must then submit its SMS plan to Transport Canada prior to start-up. Railways with existing SMS plans must also submit annual targets and updates to the department.
Transport Canada reviews these SMS plans but does not approve them, although the Minister has the authority to order changes to a company's plan under section 32.3 (1). Essentially, this means that a plan is "reviewed for potential to comply with the regulated requirement, and not to assess whether it is either appropriate, nor whether it will be effective."7
Once a company has its SMS plan in place, it is subject to periodic audits of the plan by Transport Canada. Information gathered during traditional inspections is useful for the audit process. For instance, audit findings can lead to corrective actions, or a need to learn more through inspections. Additionally, inspections can be used to confirm audit findings.
There is also an expectation under the SMS requirements that the companies themselves inspect and audit their own systems, making those results available to the regulator as part of audit or inspection processes. Conversely, positive audit results can mean a company will be subject to fewer inspections since safety risks are deemed to be lower.
Transport Canada's audit program was designed to periodically evaluate the effectiveness of a company's safety management system and whether or not implementation objectives were being realized. These "global" audits were cast at a higher level, and were performed on a three-year cycle. More recently, Transport Canada, Rail Safety has shifted its approach to conducting more "focussed" audits, where the scope is dependent on existing or potential compliance and safety issues. The Panel considers this approach to be promising and one that is moving in the right direction.
Nonetheless, Transport Canada's SMS audits remain essentially focussed on process. The Rail Safety Directorate's oversight program remains fundamentally prescriptive, and this continued focus on adherence leads to a tactical, inspection-oriented approach. Furthermore, audits tend to focus on technical and environmental factors, such as equipment reliability. Weaknesses exist in effectively auditing human and organizational elements. Accidents and incidents result from a combination of factors - human, organizational, technical and environmental - and there needs to be an understanding that strategies for mitigating risks must be developed at a systems level.
As already discussed, when auditing a company's SMS, Transport Canada generally seeks evidence of compliance with the regulated SMS requirements, rather than information regarding the performance of either the SMS or the company. In the absence of performance goals, Transport Canada does not evaluate a company's SMS plan to determine whether the safety management is appropriate, effective or results in continuous improvement. It is, therefore, important that SMS audits include information on safety performance, and not just on processes.
Safety measurement based on performance goals is an important element of a well-functioning safety management system. As discussed in Chapter 6, information should be available to Transport Canada so that results can be measured with proactive indicators, rather than with reactive indicators (i.e., accidents, incidents, fatalities, injuries, etc.).
Using a performance-based audit system, a company could provide the indicators that "explicitly demonstrate that it is fully knowledgeable of the technical, operational, environmental, human and management hazards to which it is exposed; that it has the mechanisms to comprehensively and systematically manage these hazards proactively; and that there are systems in place to continuously evaluate the effectiveness of the company's risk management activities. A performance-based approach extends beyond mere compliance with safety standards."8
The principles of safety management and performance-based oversight are adaptable and can be applied differently depending on the nature of the organization. We understand that Transport Canada supports safety management systems that are appropriate to the size, scope and complexity of different organizations. Using a performance-based approach, companies, both large and small, can design their mitigation strategies based on their operations, so long as they are able to demonstrate to the regulator that those strategies do indeed lead to the desired and intended results.
At this time, to our knowledge, no SMS guidance has been specifically designed for small railway companies. Practical guidelines for smaller operations have been developed in other industries and may be adaptable to the railway industry. If developed for the short line industry, such guidelines could significantly facilitate its implementation of SMS.
The customized approach inherent in SMS allows smaller short line companies to present risk-based plans that may differ considerably from those of larger companies. Despite concerns about the ability of short lines to devote the resources needed to develop SMS, in our experience, short line railways were very interested in implementing effective safety management systems and had taken steps to do so.
Canadian short line railways also vary considerably in size and complexity of their operations. Size alone is not the best measure of risk - there are some very small operations that carry a significant number of passengers or that operate in environments that differ from larger railways only in scale. For these reasons, separate regulations applying to short line railways are not recommended.
Transport Canada should focus its safety management systems audits to emphasize the assessment of the safety performance of railway companies.
Well-functioning SMS audits that focus on safety performance would allow Transport Canada to better manage its own oversight activities by requiring railway companies to demonstrate that they appropriately measure and manage the safety risks associated with their individual operations. Currently, Transport Canada generally seeks evidence of compliance with the regulated SMS requirements, but efforts must continue to adopt a "systems approach" to determining company safety performance. By shifting its audit approach, the regulator would be better positioned to assess not only the effectiveness and appropriateness of a company's SMS plan but also its overall safety performance. In a performance-based regulatory program, however, the railway company must also be able to demonstrate to the regulator that it proactively manages safety and the way in which it does so.9
Transport Canada's performance-based audits need to meet public service audit standards. This is key to providing companies with flexibility to manage safety according to their size and operations, including short lines and smaller operators.
We understand that the Rail Safety Directorate has developed procedures, guidelines and tools for SMS audits, though it is unclear whether these are in use or effective. The Panel understands that the Office of the Auditor General and the Treasury Board Secretariat also have documentation of widely recognized audit practices that are suitable for use within a regulatory framework. The Rail Safety Directorate should ensure that its own audit standards, in cooperation with Transport Canada regions, the railway companies and other stakeholders, meet the professional standards of public sector audits.
The standards should include the methodology governing the planning and conduct of compliance oversight activities, the reporting/evaluation of results and the resolution of observed instances of non-compliance. The standards and criteria should be published, and draft audit findings should be shared with the company being audited to validate the findings, and allow for constructive response, development of corrective measures, and eventual implementation of recommendations.
We would also point out that Transport Canada audit standards would apply to the SMS audits conducted by Transport Canada, not to internal or financial audits carried out by companies themselves.
Transport Canada, Rail Safety Directorate should ensure that audits of railway companies' safety management systems meet the professional standards of public sector audits.
5.5 Risk Assessments
Once hazards and potential risks are identified, risk assessments then allow an organization to evaluate and plan for the mitigation of risks. They can be employed at various organizational levels. To be effective, risk assessments should be proactive, explicit, transparent, adaptable, credible and employed consistently.
At Transport Canada, risk assessments should continue to be used in risk-based planning of regulatory oversight. This approach is necessary for the efficient and effective use of resources since it would allow the department to focus its oversight action on companies or industry segments where the greatest risks exist.
Transport Canada, Rail Safety needs to develop an internal, analytical function to better plan and risk manage its oversight activities. The Panel believes that this is a necessary first step. The idea is developed further in Chapter 6.
Using this internal analytical capacity would allow Transport Canada to categorize railway companies, and identify those that had well-functioning safety management systems and were able to demonstrate their results. These companies would be subject to less intensive oversight. This would allow the department to focus its energy on companies with less robust SMS plans, meaning oversight would be carried out on organizations with the highest risks.
In June 2007, the Rail Safety Directorate adopted a new Integrated Oversight Model. When fully implemented, the model will allow the directorate to plan and prioritize its activities based on risk using data from a database currently under development - the Rail Safety Integrated Gateway. This model is certainly positive and implies that Transport Canada knows where it must go. Continued focus and effort are necessary, however, to ensure the department follows through to fully functional implementation of these initiatives.
With respect to risk assessments in industry, there can be disagreement between Transport Canada and the industry about when risk assessments are necessary. The railways tend to employ risk assessments when a change in operations is contemplated. From the Panel's experience, there are not many examples of risk assessments conducted on ongoing operations. Rather, risk assessments tend to be event-based and focus on technical aspects of operations. The identification and assessment of hazards and risks relating to human and organizational factors may be forgotten. As a result, mitigation strategies will not take into account the overall context within which problems occur.
Risk assessments should be conducted regularly for ongoing operations. They should not be reserved solely for when changes are being introduced. The industry is in need of guidelines for conducting risk assessments that provide direction for identifying and managing system hazards in human and organizational factors. Transport Canada and the industry should work together to achieve this.
Risk assessments are key to effective performance-based safety management systems. Currently, the Panel feels there is much room for improvement. System-level safety risk assessments would develop a safety profile of an entire company's operations. Safety profiles would then guide internal mitigation strategies and help regulatory bodies determine appropriate regulatory interventions.
In this chapter, we outlined how the implementation of SMS can be improved. Successful implementation will require collaborative efforts on the part of both the railway industry and the regulator.
Unlike legislation governing other industries, the performance goal of SMS is not currently articulated in the RSA, and it should be. Safety management systems should demonstrate how companies continuously manage their safety risks to a level as low as reasonably practicable. By including this objective expressly in the legislation, railways would be required to demonstrate that they systematically identify hazards and manage risks to achieve the best possible safety performance.
Transport Canada seems to consider that a railway is compliant with SMS requirements if the railway demonstrates that the processes and management systems outlined in the SMS Regulations exist. The shift to a performance-based SMS oversight approach would consider the process less, but look more at the results and outcomes of the processes. In other words, Transport Canada would look less at how a company got to an end result and more at whether it achieved results, and what those results mean.
The underlying premise of an SMS is that hazards are identifiable and the associated risks can be managed proactively. The Panel believes that the onus needs to be squarely on the railway companies to implement safety management systems, and to demonstrate their effectiveness to the regulator, rather than the regulator demonstrating that safety management systems are ineffective.
There seems to be consensus between industry and Transport Canada that performance- based concepts are necessary. There is disagreement, however, regarding how the industry demonstrates that it manages the safety of its operations. Additionally, a well-functioning, performance-based regulatory program is based on a willingness of both industry and the regulator to work together collaboratively. It is worthwhile noting that the implementation of SMS has been affected by less than ideal relationships. This is why we are recommending that implementation of SMS can be improved by the industry and the regulator working together in several key areas.
Transport Canada and industry should work together to develop the tools to assist railway companies in improving their safety management systems, including:
- proactive safety performance measures;
- identification of the company data needed to support these measures;
- measurement of safety culture;
- guidance on company safety-risk profiles and risk assessments of ongoing activities;
- user-friendly safety management system tools for small railway companies;
- evaluation techniques to supplement existing audits and inspections; and
- a means of involving railway employees at all levels and, where possible, through health and safety committees and representatives.
If implemented, these ideas would go a long way to building the strong relationships needed for effective safety management systems across the rail industry.
1 See, for example, James Reason, Human Error (Cambridge University Press, 1990); other references provided in Terry Kelly, SMS Aviation Safety Inc., An Examination of the Regulated Requirement for Canadian Railway Safety Management Systems (August 2007), Appendix B.
2 See Kelly, SMS Aviation Safety, Safety Management Systems, op. cit.
3 Andrew Hale, "Rail Safety Management: The Challenge of the New Millennium," Safety Science Monitor (Volume 4, Issue 1, 2000), pages 7-8.
4 Meeting with senior managers of Air Transat, September 10, 2007.
5 Bryce Fisher, "Regulators Must Oversee Companies and People that Reflect the Entire Safety Spectrum," ICAO Journal (Volume 60, Number 4, July/ August 2005).
7 Kelly, SMS Aviation Safety, Safety Management Systems, op. cit., section 2.3.2, footnote 3.
8 Ibid., section 4.3.1.
9 Ibid., sections 4.3.4-5.
- Date modified: