When the Oklahoma Arthritis Center in Edmond, Okla., proposed a virtual server environment based on VMware Inc.’s technology, Allscripts-Misys Healthcare Solutions Inc., its electronic medical record (EMR) vendor, raised objections.
“They told us their database and printer/fax servers did not ‘officially’ support a virtual environment,” and there would also be performance issues, said Chris Nelson, the center’s director of IT.
Nelson went ahead anyway, becoming the first U.S. organization to virtualize all of Allscripts’ Enterprise EMR applications, he said. The paybacks were major, he said. His team consolidated 15 physical servers into five, while achieving a 30% performance gain for all virtualized applications, as well as higher availability and lower administrative overhead. The database and printer/fax Allscripts applications worked just fine in the new environment without any special tweaking, he added.
Nelson attributes his success to doing “a lot of careful planning to ensure the availability of sufficient CPU, hard drive, and RAM resources.” For example, the Allscripts database application servers were given high priority within the VMware vSphere environment. Nelson also built redundancy and extra capacity into the underlying virtual server infrastructure. “A Dell EqualLogic SAN[storage area network] and three Dell PowerEdge Servers was kind of overkill, but it left us a lot of room to grow,” he said.
It also helped that Nelson, who had started working at the center only recently, had gained VMware experience in a previous job. “Virtualization isn’t trivial,” said Barry Runyon, a research vice president at Gartner Inc. “And there isn’t a lot of talent out there,” particularly in the health care sector, which is a comparative newcomer to the technology.
Indeed, health care IT managers should be aware that moving from physical to virtual servers means “just trading one set of configuration and provisioning challenges for another,” Runyon said. And the new challenges can be more complex, given the complicated interactions and dynamic nature of a virtual server environment.
The first of a two-part series, this tip provides best practices and advice from the trenches for health care organizations embarking on server virtualization for the first time. The second part provides best practices for desktop virtualization projects.
The basics of a virtual server environment
A virtual server environment enables multiple guest operating systems or virtual machines (VM) to be deployed, operated and managed concurrently on a single physical server or blade. A software program called a hypervisor manages multiple VMs, each with its own guest operating system. The software automatically reallocates resources as applications need them.
At the moment, the virtualization market is highly competitive. Virtualization vendors are aggressively adding new tools and features to their products, so customers should avoid getting locked into vendor-specific products and technologies that limit choice and flexibility, Runyon said. Instead, choose a platform that supports the hypervisor standard, and get the latest release of whichever product you choose, he added.
High availability is one reason Oklahoma Arthritis Center chose VMware vSphere. “In the medical field, you can’t afford downtime. With vSphere, if one of hosts on a server cluster fails, it automatically moves live VMs to another host, so there’s virtually zero downtime,” Nelson said. The software also takes snapshots of software images, “so you can revert back [to the previous state] in minutes.”
Meanwhile, New York City’s St. Vincent’s Hospital went with VMware partly because it was the only major platform at the time that provided memory sharing, said Kane Edupuganti, the hospital’s director of IT operations and communications. “This allowed IT to utilize host memory resources more efficiently.” he said.
Which apps should run in a virtual server environment?
St. Vincent’s Hospital virtualized about 80% of its applications, Edupuganti said. The hospital is shopping for an EMR system, telling vendors that not supporting VMware is a deal breaker, he noted.
On the other hand, the hospital scotched applications that had a lot of SQL queries, with the CPU constantly at 70% utilization, because they would require a large portion of a physical server. Moreover, the U.S. Food and Drug Administration requires some applications to be on a specific, physical hardware platform.
St. Vincent’s Hospital took the right approach, said Jack Wagner, an executive consultant at Vitalize Consulting Services Inc., in Kennett Square, Pa. Indiscriminately virtualizing all application servers is definitely a bad idea: “Most hospitals have a minimum of 100 applications they consider critical,” he said. Many are homegrown and uncertified for virtualization, he added.
Uncertified applications may still work in a virtual server environment, as the Oklahoma Arthritis Center case illustrates. Still, vendor support can be critical, particularly if a company has limited in-house virtualization expertise. For example, a certified vendor typically provides a baseline for minimal hardware requirements, Wagner noted.
Even so, the internal IT staff has to either do its own performance assessment or hire an outside consultant with expertise in workflows across health care processes and servers, Wagner said. “I’ve had hospital clients whose vendors told them they could take 100 [physical] servers down to one, but when they investigated, they discovered factors that made it impossible.”
Addressing virtual server infrastructure concerns
Done right, a virtual server environment enables health care providers to conserve hardware and energy costs -- not to mention data center space -- through consolidation. However, ensuring optimal utilization can be a delicate balancing act between under- and overprovisioning, experts agree.
“Overbuilding a server can cause degradation in your infrastructure, because the system keeps turning processors off when they aren’t needed, then on again,” the Oklahoma Arthritis Center’s Nelson said. On the other hand, insufficient capacity can hurt service levels or even cause failures. “So you overdo a little at first, then monitor performance for a week or two, then maybe back off a little,” he advised.
A virtual server infrastructure needs ongoing performance monitoring and fine-tuning, just as physical servers do. IT administrators need “to get a sense of what’s going on with a given VM, both for up-front planning and to ensure room to grow,” and avoid response time problems, Nelson said.
Management tools have become more sophisticated as server virtualization technology has matured, said Gartner’s Runyon. Software tools such as VMware’s vCenter Server and Microsoft’s HyperAdmin enable administrators to centrally monitor the status and configuration of clusters, hosts, VMs, storage, the guest OS and other critical components of a virtual server infrastructure.
One benefit of a virtual server environment is that capacity can be added on the fly, without having to take servers down, Nelson noted.
Strong virtual server security a must
IT managers need to be aware that migrating to a virtual server environment introduces new configurations and practices that can open the way to security breaches and compliance issues. The hypervisor, which acts as liaison between software and hardware, represents a new potential access point for hackers. The dynamic flow of processes and applications across VMs and hardware devices also makes it hard to monitor and control information access and compliance with security policies.
Overbuilding a server can cause degradation in your infrastructure, because the system keeps turning processors off when they aren’t needed, then on again.
Chris Nelson, director of IT, Oklahoma Arthritis Center
In addition to the usual directory-based user authentication and perimeter controls, companies need tools that are specifically designed to monitor and manage virtual server security. Security software vendors, such as Symantec Corp., NetIQ Corp. and BMC Software Inc., have extended their security and compliance products to virtualized environments. NetIQ, for example, provides VM support for both Secure Configuration Manager, which automatically assesses and baselines server configurations for security, and Security Manager, which monitors user activity, detects changes, and provides log consolidation and reporting.
In addition, a growing body of third-party vendors specialize in virtual server security -- Altor Networks Inc.’s Network Security Analyzer, for example, monitors VM infrastructures for such potential problems as VMs configured to use unwanted protocols, trust breaches when VMs move from low- to high-trust machines, and new VMs with outdated or incorrect configurations.
The best performance and security tools are worthless, however, without the expertise to use them effectively. IT staff training is critical, of course, and hiring an outside expert often advisable.
Aside from that, “Google is your best friend when it comes to getting advice and tips on performance enhancements and best practices on the Web,” the Oklahoma Arthritis Center’s Nelson said. “There are so many granular options, things you can do to enhance performance on the host machine, the cluster, the VM.”
And the more tactics an IT department can deploy to improve performance, the better users will feel about their new virtual server environment. “Time is of the essence for end users, especially for health care providers,” Nelson said. “You always have to be looking for that extra kick.”
Elisabeth Horwitt is a contributing writer based in Waban, Mass. Let us know what you think about the story; email firstname.lastname@example.org.
This was first published in March 2010