ONC wants help with health IT certification testing program

With a 5-year plan in mind, ONC is asking healthcare stakeholders to offer up their health IT certification testing tools in order to improve the certification process.

The Office of the National Coordinator for Health IT (ONC) wants to work with healthcare industry stakeholders to improve the health IT certification program.

ONC's health IT certification program, established in 2010, included a way for industry stakeholders to submit their own developed testing procedures, tools and methodology, Steve Posnack, director of the Office of Standards and Technology at ONC, said, "because we expected that there would be stakeholders that could identify a better way to do things and we wanted to be open and receptive to improvements or alternatives to the way in which we laid out the government produced testing requirements."

Posnack said that ONC is working to create a five-year program to identify effective health IT certification tools and "by the fifth year we hope to have a more diverse mixture of testing tools and test methods that are some government funded where we can prioritize our sources and some that are public-private developed that allow, as I mentioned, that kind of reciprocity to be done once and used multiple times."

At this point, however, Posnack said that ONC is in the awareness and outreach stage of this process.

Stage two is active engagement with organizations that have testing methods and tools that they want to be approved as part of ONC's testing processes, he said.

"That's where I would expect us to probably be looking at that five-year time line," Posnack said. "Spend a good deal of time with different organizations working out the details and figuring out if they would have something that would work."

ONC approves NCQA health IT certifications tool

In June 2017, ONC approved NCQA's testing methods for electronic clinical quality measures (eCQM) as one of ONC's health IT certifications tools. NCQA is a healthcare accreditation organization

"NCQA built a synthetic test deck generator based on a well-known concept [called] black box testing," Rick Moore, CIO at NCQA, explained. Black box testing is a software testing method that examines the functionality of an application.

"The target system doesn't know what it's going to be getting in terms of synthetic data and where we sent all the value sets and all the different parameters that are inclusive in the measure, like blood pressure [and] gender," Moore said.

He explained that their deck generation system will put out 800 to 1,500 cases in a test deck which is formatted into a continuity of care document (CCD), the standard format for interoperability. That test deck is ingested by the target system -- the system that is being tested by NCQA's testing software. Then the testing software sifts through the data to find the cases that meet the specifications such as codes, gender, enrollment, etc.

"So all those different parameters are being tested in our algorithm validation. The algorithm is invalidated if it didn't correctly select the right records," Moore said. "The final part of that test is the target system has to format the report that would be sent either to another system or to a recipient whether it's CMS for payment or measurement review or whether it's HQA or a recognition program."

Moore added that each system has to correctly identify every known case in order to be certified. If for some reason the software misses something, the company can work to tweak it and then NCQA provides a new test deck.

"We go through that motion anywhere between two to as many times as five times for some to actually get the measure correct," Moore said.

This method of testing ensures that a company's software does what it is supposed to do.

For example, "if you look at what eClinicalWorks was just recently cited for having done where they didn't necessarily include the entire value set, they just included the code, they hard coded it," Moore said.

With the value-based payment model the country's reform initiatives to get towards pay for value and the rates being produced are going to become more scrutinized; probably for good reason if your payments are based on them.
Rick Moor, CIO at NCQA

He explained that that is a good example of a certification test where a vendor knows what is needed in order to pass that test and making sure those requirements -- in this case, codes -- are present versus a test where the vendor doesn't know what is needed to pass and therefore has to be fully equipped and prepared.

"With the value-based payment model the country's reform initiatives to get towards pay for value and the rates being produced are going to become more scrutinized; probably for good reason if your payments are based on them," Moore said. "Vendors would like to know they're doing it right, the payers would like to know they're paying for the right rates and I'm sure the providers would like to know they're being fairly rated."

The future of the health IT certification program

"In the next year it would be our goal to have another, at least one more, industry related kind of testing method approved under the program. I would be ecstatic to have more than one and you know we are certainly looking for more organizations to come and step up into the testing role as well as an authorized testing lab," Posnack said. So that will allow for greater options for health IT developers to get tested as well as [create a] potential competitive market place."

And other health IT vendors are on board with ONC's initiative to improve the health IT certification program as well.

"We support ONC’s activities to enhance its programs to support industry-developed testing envisioned by the Cures Act, among other improvements, and we look forward to continued collaboration," Marlene Bentley, a spokesperson for Cerner, said.

Next Steps

APIs are a certified health IT requirement

OIG weighs in on eCW's improper EHR certification

New EHR certification standards point towards coordination

Dig Deeper on Electronic health record system (EHR) certification