As health care providers have expanded and combined to become regional powerhouses, IT has been tasked with connecting the mushrooming sites with reliable high-speed links. For many health care companies, a redundant fiber network is the essential building block.
"We've been fortunate with good reliability and security. The key was going with the SONET ring," said Tom Smith, CIO of NorthShore University Health System in Evanston, Ill. (SONET refers to Synchronous Optical Network.)
In approaching network design, regional health care organizations such as NorthShore face the need to transmit large numbers of big image files and -- because peoples' lives hang in the balance -- to guarantee nonstop availability. A dual-fiber backbone satisfies these needs with bandwidth that reaches gigabit-per-second (Gbps) speeds and high reliability thanks to redundant links.
"The amount of space needed to send images across the network with speed and quality is a huge issue in health care today. A fiber backbone allows for speeds that are relatively instantaneous," said Joy Grosser, vice president and CIO of the Iowa Health Systems in Des Moines.
Although not relying on a SONET ring, Iowa Health Care does have a fiber optic network with redundant connections to major nodes. "We're a very rural state. As you are connecting in a rural state, you want high reliability, so you want alternate paths," Grosser said. Altogether, the Iowa Health Network has 3,200 miles of optical fiber linking facilities that extend across the state, reaching out to Internet access points in Chicago in the east and Denver in the west, according to Grosser.
Making the upgrade to a fiber network
When NorthShore put in its first picture archiving and communication system (PACS) in 1998, the Chicago-area hospital system knew that the 56 Kbps network linking its buildings would have to go. While some institutions may want to carefully model traffic before building out a network, NorthShore leapfrogged that step. "We didn't use any modeling -- just our own sense and analysis. We knew we would be shipping larger images and files," Smith said.
In the past 12 years, the institution expanded and upgraded the fiber network several times. Today, it runs at OC-48 (optical carrier) speeds, with a data rate of 2.488 Gbps. NorthShore's facilities consist of four hospitals in a roughly 15-mile-diameter circle. There are also two major office buildings and two data centers on the dual fiber ring.
"It goes through our buildings and through the central offices of AT&T [Inc]. If the ring is cut by construction or something, it’s smart enough and fast enough to go in the other direction. We have had literally 100% performance since 1998,” Smith said.
Like NorthShore, The Ohio State University Medical Center (OSUMC) relies on local redundant fiber rings to connect five main hospitals and the AT&T Optical Ethernet Metropolitan Area Network (OPT-E-MAN) network to connect to remote locations throughout central Ohio. In 2006, OSUMC embarked on a three-year, $15 million project to upgrade its fiber network. The entire core, backbone and access layer were upgraded, as well as the cabling. Like NorthShore, the IT pros who handled the project saw no need to use modeling tools to closely predict bandwidth needs.
"The design is scalable and we built the network to support well beyond the current bandwidth requirements. We don’t expect to have to perform another upgrade for at least five years. We can bond the channels on our core and support 40Gb without significant investment," said Chad Neal, director of technology at OSUMC, adding that his team had installed four Cisco Systems Inc. 6500 core switches.
Turning to dark fiber, microwave links for redundancy
A fiber backbone allows for speeds that are relatively instantaneous.
Joy Grosser, vice president and CIO, Iowa Health System
To obtain a redundant fiber backbone, you can build your own or rely on a service provider. Building your own fiber network will give you a high level of control. However, if you are seeking to lay cable in the ground, you may need to plan years in advance to allow for necessary approvals. "It's difficult to put in fiber -- you have to have access to poles and right of ways," said Tony Langenstein, director of technology at Iowa Health System.
Smart network pros look for dark fiber, or optical fiber that's been laid in the ground by a network provider but has not been activated because of lack of demand. Iowa Health System has taken advantage of dark fiber in several instances as it has built out its network.
"There are some great opportunities," Grosser said. "Telecom providers have been very proactive, putting in dark fiber next to freeways. We have been able to get very close to a location with dark fiber and then just have to do the last mile."
In locations where it's not practical or too expensive to use optical fiber, a point-to-point microwave link can be a good solution. For example, Iowa Health has deployed several Trango Systems Inc. microwave links at speeds up to 400 Mbps. At NorthShore, a microwave link was deployed to bridge a 30-foot gap between two buildings where a fiber connection was not practical, according to Smith.
It's also a good idea to scan the horizon for funding sources, particularly in the wake of the American Recovery and Reinvestment Act (ARRA). Iowa Health has benefitted from the ARRA-sponsored Broadband Technology Opportunity Program to the tune of $19 million, with $9 million in cash and network value provided by Iowa Health. In addition, Iowa Health is benefiting from the Rural Health Care Pilot Program, which offers $7.8 million in funds against $1.3 million provided by Iowa Health.
It may be impractical to connect all nodes with a redundant fiber network. In those instances, the lower cost of a network service, such as AT&T’s OPT-E-MAN, might prove a good choice. NorthShore is using OPT-E-MAN to connect some 40 office buildings at speeds up to 10 Mbps and relying on Comcast Business Internet for backup.
Farther afield, NorthShore is linking 25 independent doctors’ offices through the public Internet but giving them encrypted links via Citrix Systems Inc. virtual workstations and a virtual private network (VPN) connection.
Managing fiber network outages, throughput
Optical fiber may meet the key requirements for speed but, for reliability, a ring topology is critical. The Iowa Health Network suffers, on average, 15 cable cuts per year, according to Iowa Health’s Langenstein. The breakages are not just from backhoes -- more often, rodents and errant shotgun blasts are the culprits, he said.
However, service providers are not immune from outages, either. AT&T was responsible for a lost OS-48 equipment malfunction that resulted in loss of connectivity between all points on the ring, according to Neal at OSUMC.
To assure high performance, meanwhile, it’s necessary to watch for bottlenecks by monitoring traffic. Langenstein uses a variety of network monitoring tools, including Ipswitch Inc.'s WhatsUpGold as well as tools from SolarWinds Inc. At OSUMC, Neal uses CA Technologies’ NetQoS tools to monitor all legs of the network.
What's next for wide area network (WAN) optimization
With a reliable high-speed network in place, what’s next? Neal is working on a scheme to segment his network according to usage, providing different levels of security as needed.
"We're starting to look at the use of VLANs [Virtual Local Area Networks]. We have requests to carve out pieces of the network and partition them off to make them more secure -- like where credit cards are taken for transactions," Neal said.
He plans to implement Multiprotocol Label Switching to create "little tunnels" that are protected from the rest of the network, with enough bandwidth and security to handle credit card payments. Neal is also planning to implement Cisco Nexus switches -- multiprotocol switches that can handle a variety of traffic, including traffic for data center storage area networks.
Langenstein is upgrading Iowa Health System’s Coarse Wave Division Multiplexing to the higher-performance dense wave division multiplexing, or DWDM. With 10 Gbps speeds currently in the backbone, Iowa Health will start at 30 Gbps with DWDM and go higher as bandwidth needs demand, Langenstein said. And he will continue to implement redundant links. "It's almost a mesh-type environment. There are lots of alternate paths," he said.
Also on the horizon at Iowa Health is a grant-funded foray into Wi-Max technology to connect not only hospitals but community colleges, police stations and public libraries.
Stan Gibson is a Boston-based contributing writer. Let us know what you think about the story; email email@example.com.
This was first published in May 2011