Manage Learn to apply best practices and optimize your operations.

How to create a clinical data warehouse

Creating a clinical data warehouse isn't hard -- if you plan a strategy, develop a team, confirm everyone's readiness, determine project impact, set its direction and execute.

Few things are as challenging as creating a clinical data warehouse. The complexity lies in the diversity of the requirements as well as the sophisticated visualizations. Setting up the data is complicated enough, and you will need a partner from your clinical side to assist. But when you start to consider how to best present these data, the challenges can feel overwhelming.

This article will outline best practices for creating a clinical data warehouse. While these recommendations apply to any type of data warehouse, there are specific considerations given to the clinical aspect. 

In all, six things must be considered when building a clinical data warehouse. You can remember this through the acronym of STRIDE, which will help you hit your stride. All of these must occur in order to have a successful project. 

Strategy: You must plan your data warehouse project. It's incredibly important to understand what value the project will bring to the organization and how you intend to execute the project. This high-level strategic plan will also help you gain executive sponsorship. Winning executives over for this type of project can be challenging, but it's critically important if you want to ensure funding and long-term support. 

Team: In this case, for clinical data warehouses, you will need a team of people to truly be successful. While that point is true for most data warehouses, the need for a fully integrated and diverse team is most important here because it's unlikely that your IT team will have the depth of clinical knowledge needed, while your clinicians will probably lack the depth of technical expertise to completely finish the job. Bringing these two groups together will move the project forward.

In best practices for non-clinical data warehouses, "T" stands for Technology and Architecture. This is important too, but, truth be told, technology should not be a key criteria for decision making. The team structure is important for all data warehouse projects, but it is particularly critical for success in clinical data warehouses, and the project will certainly fail if enough focus isn't given to who creates the clinical data warehouse. (We will still address architecture in two different sections, as we break it down between the impact the architecture must make and its ability to execute.)

Readiness: Preparing the team for this work will be critical. To confirm their readiness, the team should create a checklist of key considerations that need to be made at each stage of the project. For example, as you get started, you need to have a fully vetted strategy. You need to have a committed and named team, not just "someone from clinical." You will likely need some type of budgetary support as well. The list can go on. This readiness step is part of the planning, but it will take on different perspectives as the project moves through different phases.

Impact: When planning a clinical data warehouse, you have to consider its impact and total payoff. Building a data warehouse is certainly interesting work. In order to get long-term support, however, you will have to demonstrate the return on investment for your organization. As you plan, think about the gaps that the clinical data warehouse can fill. For example, perhaps you have a case management department that today uses paper files in addition to electronic health record (EHR) systems to track patient populations.

A clinical data warehouse could automate that process, creating groups and sub-groups that your clinical team could track to determine efficacy of treatment modalities. If you have a large enough population set, you could do that based on criteria such as co-morbidities. This would not only save FTE time, but it could also help determine what treatments are best delivered to whom, when and how. Make a list of such impact statements so everyone can easily see the business value of the data warehouse for the entire organization.

Clinical data warehouse architecture should support a low-level patient, provider and encounter view.

The important thing about determining impact is to ensure that you can architect for it. Clinical data warehouse architecture, then, should support a low-level patient, provider and encounter view.

Direction: You will need a team leader who can see all the benefits of the finished product from the beginning. The team lead has to act as the head cheerleader to ensure that the project continues to stay funded and supported. The key role for the person providing direction is to ensure that the team stays on task, measure the progress against the strategy for the overall project and keep marketing the project internally. Depending on the size of the effort, this may be a full-time job, so plan accordingly. 

Execution: This is where the rubber meets the road. Let's say you already have a strong strategy, plan, team and leadership. Now you have to get the work done. Armed with your impact statements, your next step is to ensure success by finalizing your architecture plans. Think of the impact statements as your blueprint for success. The execution of those blueprints creates the house you get to live in. In data warehouse terms, this is referred to as the conceptual and physical models.

In addition to ensuring that you can execute against your impact statements, you have to ensure that the team is prepared to work together. This can be particularly challenging if your team is assigned this project in addition to other workloads. Make sure that communication is abundant and clear. Ensure that everyone knows what they are responsible for and when.

Finally, don't forget that, with any team, you will have the forming, storming, norming and performing stages of team development to go through. Consider that as you proceed. It's normal to not reach the performing stage until two or three impact statements have been delivered. In other words, just because you have struggled to get a couple things under your belt, you shouldn't underestimate the impact that team development can have. 

Fail early, fail often

Don't be afraid of failure. It's bound to happen. The great thing about failure is what you learn from it. The bad thing about failure is that if you wait to do it, it can have big repercussions to your project. So, fail early and fail often.

The best way to do this in the context of data warehousing is to utilize agile methodologies to deliver content quickly. It doesn't have to be perfect, but putting something tangible in front of your audience early will reduce re-work cycles. If you spend 20 hours building something that your end user modifies drastically, no big deal. On the other hand, if you spend 120 hours building something that your end user modifies drastically, that can be a project killer. 

Begin with the end in mind

Famously said by Stephen Covey, this statement applies very well to any business intelligence effort, and probably more so for a clinical application of data warehouses: Consider your end users. Most clinicians understand data in a very sophisticated way, but they do not want to understand software. Your goal for any clinical data warehouse, then, should be intuitive visualization of key data elements. Rely on your clinical team to help you document and visualize what's important for them to see.

Most business intelligence systems offer a sophisticated array of data visualizations, often referred to as widgets. Beyond your standard odometer or stop-light, highly data-oriented audiences will want to see things such as heat maps and control charts. Working through these things early on -- that is, beginning with the end in mind -- will help you select the right BI tool based on end user needs, not software features.

Challenges abound for any data warehouse project. Clinical data warehouses are no exception. The value of these projects is broad and tantamount to any organization's reasons for tackling the challenge. If you always start with a strategy, a diverse team, a plan, a series of impactful projects, a strong leader and a focus on execution, you will be ahead of the game.

Laura Madsen is the health care practice leader for Lancet Software. She is also a veteran of United Health Group, where she managed a business intelligence tool suite.Let us know what you think about the story; email [email protected].

Dig Deeper on Clinical data analytics software and systems