EHS Today - September 2016
By: Terry L. Mathis
Printable Version
Applying analytics to safety data is being touted as the perfect way to determine the right leading indicators for safety. Compared to blind choices and wild guesses, the use of statistical analysis certainly has some advantages. And, admittedly, pursuing the wrong leading indicators can be expensive and dangerous. Modern analytics has progressed significantly with the computerization of mathematical processes and use of massive sets of data. Using this technology to help find a starting place for determining leading indicators makes a lot of sense. That said, analytics is not necessarily the silver bullet or magic pill either. The early use of analytics to determine leading indicators potentially has at least five serious limitations:
- Garbage In, Garbage Out – The data being used to seek out leading indicators is often taken from accident investigations, near-miss reports, behavioral observations, site and process audits and similar sources. All of these are subject to massive subjectivity on the part of the person gathering the data and vary tremendously due to different methodologies, such as root-cause analysis, and differing observation and audit processes. The level of analysis and description differs greatly on accident and near-miss reports. Items on audit lists and behaviors on observation checklists can be different site to site. Physical conditions can be different and audits of them can have very little similarity. The old premise of data analysis is the quality of the output is never any better than the quality of the input.
- Correlation versus Causation – In truth, data analysis can determine correlation but cannot definitively determine causation. The fact that outcomes are generally better when certain things happen upstream is interesting and often a good starting place. But it has been proven many times that imitating the success of another person, site or company is no guarantee of success for the imitator. Marshall Goldsmith pointed out the futility of imitation in his book, What Got You Here Won't Get You There. Looking for the cause of success or failure still relies on the lagging indicators we are trying to end our reliance on. Truth is, some sites are successful in spite of the way they manage and practice safety. Finding correlations between their methods and their outcomes is of limited value.
- Hawthorne Effect – The famous Hawthorne experiments determined a correlation. They determined that focusing attention on something can temporarily improve performance. The problem is, when the attention begins to diminish, so does the improved performance. This is problematic for data analysis. A correlation between a new safety program, training session, management directive or almost any other activity focusing attention on safety could show up in "big data" as a corollary. Based on that correlation, the conclusion could be made this is a significant leading indicator of safety when it was, in fact, a temporary phenomenon. Seeing results from such efforts in the past, many organizations launched on a veritable parade of new programs one after the other to keep up the safety performance. They found in the end that these did not produce permanent change and actually lost impact after too many new "programs of the month" started to wear out the attention span of workers. It can be difficult to differentiate between permanent and temporary correlations if there a series of temporary one all labeled as safety programs.
- Assuming Site Sameness – The "big data" folks were quick to point out much of the data they used was either hidden or tucked away at the site level rather than gathered enterprise-wide. In their zeal to get enough data points to attain statistical significance, they could easily overlook the significant differences between these sites. Assuming an organization is simply the sum of its sites is like assuming a culture is simply the sum of the people in it. As a business and safety consultant with decades of experience, I have seldom seen a cookie-cutter approach work at multiple sites. Every site has at least small differences that can have a big impact on what works and what doesn't. Assuming the leading indicator determined from analysis of data from multiple sites will be a good fit for each site is far from scientific or pragmatic. Sites with vastly different approaches to safety have produced good lagging indicators. Changing one site to match another, just because the overall data suggests trends, can actually undo highly customized safety efforts that work well for that particular site or culture.
- Stopping at Two-dimensional Thinking – While the development of leading indicators can take safety metrics from one-dimensional (results metrics) to two-dimensional (cause-and-effect metrics), the real world in which safety is practiced is still three-dimensional. The Balanced Scorecard (multi-level leading indicators), which is in wide use by many organizations to measure their overall performance, seems a better approach. The early attempts to identify leading indicators for safety tended to focus on activities such as training, meetings, participation in safety audits and observations, etc. Then the attention turned to worker behaviors, then to safety culture. While none of these is sufficient alone, they all make up a process flow for how many organizations approach safety. Safety activities drive safety culture which drives safety performance which, in turn, drives safety results. Looking at leading indicators in sequence, rather than just as a set of individual or unrelated metrics, is much more descriptive of the reality of most safety processes. So far, big data is just looking for lone data points that correlate with results rather than looking for a sequence of corollaries.
While the application of new technologies to safety always has the potential for improving results, it is important to keep these theoretical tools in a reasonably pragmatic framework. Subject-matter experts should work closely with safety professionals who understand the workplace realities. Analytics is still a science. For it to be transformed into a technology that can benefit safety will require a realization of its limitations as well as its power.
Terry Mathis, Founder and retired CEO of ProAct Safety, has served as a consultant and advisor for top organizations the world over. A respected strategist and thought leader in the industry, Terry has authored five books, numerous articles and blogs, and is known for his dynamic and engaging presentations. EHS Today has named him one of the '50 People Who Most Influenced EHS' four consecutive times. Business leaders and safety professionals seek Terry's practical insight and unique ability to introduce new perspectives that lead to real change. Terry can be reached at info@proactsafety.com or 800-395-1347.
