Posted by: Jenny Laurello
Dashboard, Key performance indicators, KPI, Quality
Why you shouldn’t measure everything (or at least have it on your dashboard):
While the phrase “you can’t improve what you can’t (or don’t) measure” is undeniably true, it leads many organizations to try to measure everything. It is this impulse that leads to the proliferation of massive reports (that nobody reads) and overpopulated dashboards that in fact complicate decision making and therefore lead to very little (if any) actual improvement. It is impossible for a health care organization to improve everything at the same time. That is exactly why true health care transformation is a long-term endeavor, and why analytics must remain focused on answering the right questions, and providing the right answers, to achieve change.
Analytics is the lens through which the processes, workflows and business operations of a health care organization should be viewed to guide decision-making and improvement activities. To be effective at facilitating quality and performance improvement, every aspect of the health care business intelligence (BI) tool set (be it analytical tools, dashboards, reporting, etc) should be directed at identifying what needs to improve, and identifying if a change has occurred.
Identifying what needs to improve:
The quality goals of a Health Care Organization (HCO), either organization-wide or local to a unit or department, are a critical input when designing analytics solutions that provide guidance for selecting quality and performance improvement activities. For example, if a goal is to achieve national and international benchmarks for the treatment of stroke within the Emergency Department, then the analytics performed need to focus on achieving this goal. This is done, in part, by selecting relevant quality indicators (such as the “door to CT scan” time interval) and setting appropriate targets. (Click for an article on selecting key performance indicators, or KPIs.)
Once the appropriate quality indicators have been defined, individual processes that fall within the focus of improvement need to be analyzed to identify what must change at the process level. The indicator “door to CT scan” time interval undoubtedly is comprised of several steps (including, but not limited to: arrival at hospital, triage, registration, CT order, transfer to CT machine, perform CT and waiting for results). Each of these steps in the process likely can contribute valuable information to understanding how “door to CT time” can be improved.
Analytics developed to identify what needs to improve will typically include an analysis of the organization’s past performance. Understanding previous performance is important because it helps to clarify not only the magnitude of change necessary, but may even suggest which improvement approach (such as Lean or Six Sigma) may be most suited to address the challenge of improvement. (Click for an article on how analytics and quality improvement methodologies must be integrated to achieve success.)
In addition to providing a retrospective look at performance, analytics are very helpful at providing a real-time (or near-real-time) analysis of performance. The best real-time systems not only provide dashboards that compare current performance versus targets, but include “agents” that automatically monitor incoming data and provide alerts (for example, via email, pager, or SMS) when actual performance is deviating from desired goals. The real-time alerts allow corrective action to be taken immediately (whether it involves further refining a process or simply coaching inexperienced team members on new workflows). These opportunities for a “quick fix” may be lost if performance issues are uncovered days or weeks later in a report, or not clearly highlighted on a performance dashboard.
Identifying if a change has occurred:
As mentioned above, analytical applications are very useful within health care to monitor progress toward achieving quality and performance improvement goals. To accomplish this, however, analytical solutions must be able to detect if and when a change has occurred. This is especially important when quality improvement teams are actively engaged on projects and need to monitor the effects of their changes to processes and workflows.
Despite an abundance of data that many HCOs have at their disposal, detecting a change may not be as simple as calculating an average interval time and plotting a run chart. Data analysis approaches (i.e., statistics), and the visualization of results, must be selected carefully to ensure that any impact of a change in process or workflow that occurs is detectable and apparent. There are many reasons that a change in process may not register in the data. Several such reasons (that relate to the selection and analysis of indicators) are:
- The change in process has no significant impact on performance (in which case, further work is necessary to design improved processes and workflows to achieve the desired change);
- The indicators selected are not representative of the processes being changed (which means that the indicators need to be adjusted);
- The analysis or visualization is not sensitive enough and/or appropriate.
Regarding the point on analysis and visualization, process changes may manifest in more subtle changes to data before becoming “obvious.” For example, rather than simply reporting an average (which can be greatly affected by outliers), it is usually more informative to analyze data in multiple ways. For example, the median and interquartile ranges add additional information regarding the distribution and variation in the data. Visualizing performance by plotting histograms and boxplots better highlights the intricacies of the data than a simple bar graph of averages.
It is vital to have enough confidence in your analysis to trust that if no change is showing in the data, that in fact no change has occurred and not simply missed because your analytics are off-target. The reverse is true, as well – you don’t want fluctuations in the data (as a result of seasonal or other systematic changes) to be interpreted as a change due to improvement activities.
Analytics are a very powerful tool for use in improving health care quality and operational efficiency. With all the data available now, however, it is possible to create myriad dashboards that essentially don’t provide any useful information at all. When engaged in health care improvement activities, analytical solutions ultimately must identify what processes need to change, and clearly demonstrate that a change has (or hasn’t) occurred. Only this will provide the information required to achieve true change in health care.
Trevor Strome MSc, PMP, is the Informatics Lead for the Winnipeg Regional Health Authority, and is Assistant Professor at the Department of Emergency Medicine, Faculty of Medicine, University of Manitoba. You can visit Trevor’s blog at http://healthcareanalytics.info, or contact him by email at email@example.com.