In most industries, keeping offsite data backups promotes business continuity and can speed disaster recovery....
Companies ignore this best practice at their peril.
In health care -- where patients' health and well-being are at stake -- offsite backups are mandatory, hardwired into HIPAA regulations that also mandate patient data availability (see sidebar).
A virtualized server environment, however, makes keeping offsite backups a thorny proposition. Recreating virtual servers from an offsite backup probably wasn't a scenario regulators imagined when writing HIPAA, but it's possible to achieve it, said Lance Madsen, radiology information systems administrator for InnerVision, a Lafayette, Ind.-based radiology practice employing four radiologists rotating through four sites.
InnerVision virtualizes more than 30 servers on four physical servers, using VMware Inc. ESX. The Centers for Medicare and Medicaid Services (CMS) also requires his company to keep Medicare patients' radiology data for five years. (Rules for Medicaid patients vary by state.)
"The standard policy of backups is to take your data that you need to back up, attach your tape drive to the server, stick in the tape, let that tape run its backup process [and] stick that tape in a safety deposit box at an offsite location," Madsen said. "Now that it's all virtualized, you can't stick a tape drive in a server that doesn't really exist. The traditional model doesn't exist anymore."
PACS update begets implementing virtualized server environment
InnerVision tossed out the traditional health IT model about two years ago, when Madsen led an implementation of server virtualization technology. Doing so cuts down on capital outlay for physical servers as well as space and maintenance needs.
InnerVision decided to make the leap to a virtualized server environment when it needed to update its picture archiving and communications system.
His team creates a new virtual server for each new database or application required by the various services InnerVision provides to its patients -- magnetic resonance imaging (MRI), computed tomography (CT), ultrasound, bone densitometry (DEXA), X-ray, nuclear medicine and positron emission tomography (PET) services -- and of course the records, documentation and billing support for all of the above.
InnerVision decided to make the leap to a virtualized server environment when it needed to update its picture archiving and communications system (PACS).
Vendor virtualization support, Madsen said, carried a lot of weight in that purchasing decision. "I came up with about 25 criteria I ranked [PACS vendors] on. Virtualization compatibility and [the vendors'] willingness to support it in that environment was a major one. There were only two or three vendors who were willing to at least work with us. All the others were staunchly, 'There's no way.'"
InnerVision settled upon Amicas Inc. for its new PACS. While the radiology practice was an early customer virtualizing that particular system, Madsen was confident it would work because the vendor had virtualized the PACS using VMware in its own office and was willing to support it at InnerVision. (The practice hasn't yet gone live with a virtual desktop implementation but is currently testing the deployment of medical imaging apps to iPads and other devices via VMware desktop.)
Backup addresses HIPAA compliance requirements
After the PACS went live and the virtualization setup proved functional, Madsen's team turned its attention to implementing HIPAA compliant backups. InnerVision chose Veeam Software Corp.'s Veeam Backup & Replication, which backs up whole virtualized servers to two InnerVision data centers -- including one offsite, in another city, connected by fiber optic cable.
Madsen said the Veeam backup keeps updated copies of the servers nightly, ready to run. The setup eliminates the waiting period posed by old-school backup recovery off tape, not to mention the "praying that you have a solid backup, who knows whether it's going to work or not," Madsen said. It's possible to do real-time backups with their system, but InnerVision has not yet chosen to do that. Madsen can recover data on a per-file basis in larger blocks, up to restoring whole virtual machines.
So far, the backup has worked. While InnerVision has, fortunately, avoided suffering natural disasters or IT-crippling power outages, it did have to use its system to summon a previous backup of the PACS system after its database had been corrupted beyond repair -- the vendor couldn't even get it back online. Without the virtualized backup system, "We would've been pretty screwed, actually," Madsen said.
Backup advice to peers using virtualized server environments
Virtualization, Madsen pointed out, does help security in one sense, as it's impossible to steal a virtual server. However, it also gives potential thieves one more entry point to the network if they can somehow steal credentials for logging into the virtual machine images, either live or backup. Build a policy around managing those credentials, he advised.
Madsen also said VMware customers choosing a backup vendor should find one that can access VMware vStorage APIs. Those APIs access data directly and keep backup times as short as possible. "Being able to tie directly into the storage through VMware is a huge, huge plus," he said.
Let us know what you think about the story; email Don Fluckinger, Features Writer.