Given that computers are so common not only in health care but everywhere these days, it is often easy to overlook that properly setting up a new computer-based system may not be a simple endeavor. As a result, the necessary time is not always allocated to ensure that systems are set up correctly prior to their use. The following real-world example illustrates the fact that setting up networked medical devices is not as simple as it seems.

An outpatient medical center was installing a new imaging device. It had already scheduled dates for inspections and training, and the installation schedule was set up by moving backward from those dates. In the original schedule, several days were budgeted for device configuration and testing. Then there were some delays in getting the device shipped to the site. However, the vendor guaranteed that even with the delay in shipping it would have someone on-site 24/7 to get everything ready to go to meet the established dates.

The center arranged to have IT resources available to work with the vendor on setting up the device’s network configuration, but the vendor realized it would not be ready to start setting up the configuration until late on a Friday evening or over the weekend, with the inspections and training scheduled to take place on Monday. As this was a remote site and setup would be taking place after hours, the vendor asked IT for the device’s network settings and indicated that it would set everything up without an IT resource on-site. Not wanting to wait around on a weekend, the IT resource agreed to provide the information and left.

Late on Friday evening, the vendor finally input all the network information for the device. It did a quick ping on the network to look for connectivity, and when it got a response, the vendor turned off the device and left for the weekend assuming that everything was set up OK.

With computers so common in health care, it is easy to overlook that the proper setup of networked medical devices is not simple.

The center continued normal operation over the weekend until the staff came in on Monday and turned the device on in preparation for their training. At that point, the entire network at the site stopped functioning with a full schedule of patients coming in. The staff, not realizing that turning on the device caused the problem, alerted their IT support that the network was down, and IT support in turn began looking for failures of network hardware or other items that would cause a major outage. They could not turn up anything that was immediately wrong. They could reach the site’s gateway (main access point on the network for entry into that site), but they could not seem to get beyond it. When they tried to investigate the gateway, it exhibited some strange behavior. After some time, they determined that the gateway was not what they had set up, and after some additional investigation they found that the gateway was actually the new imaging device that had just been installed. When they looked at the device they found that the vendor had input the gateway IP address for the site as the IP address for the individual device, which caused the device to appear on the network as the site gateway. Unfortunately, the device was not set up to act as the gateway, so even though it appeared as a gateway to other devices on the network, it could not route the network traffic, thereby bringing the network to a halt. Inputting the correct IP address into the device resumed normal network operation.

When the IT staff followed up with the vendor who was on-site, the vendor indicated that it did not have much experience in setting up network settings on devices and just looked at a setup script. As it was late and the vendor had worked on the device all day, it made an error and entered the gateway in place of the IP address and did not recognize that the format did not look correct.

Overall, the network was down at the site for a few hours, which led to delays in patient exams and other issues. The good news was that it was an outpatient site and none of the delays caused issues in terms of patient treatment. Had this happened at a hospital, it is easy to see how the impact could have been much more serious. The facility learned a great deal from this incident and has put measures in place to ensure that this type of issue does not occur again. Could this have been prevented? Yes. And the lessons learned apply to all sites.

Lessons Learned

Could this have been prevented? Definitely, and it could have been prevented at many steps along the way.

First, fixed deadlines were allowed to drive the project timeline. Had the site looked at its project schedule it would have determined that once the vendor was going to be delayed in delivering the device that the necessary work to complete the project would not be given adequate time. It could have then rescheduled training and inspections and pushed back its live date. Instead, it pushed the vendor to meet the scheduled deadlines by cutting out the time needed at the end of the project for adequate device setup and testing.

Second, the IT resource allowed the vendor to set up the device without being on-site to check that the setup was correct. The IT resource made the assumption that everything would be done correctly. Had the IT resource been on-site and checking the settings, this would never have happened. Or, had the IT resource decided not to hand over the network information to the vendor and waited until Monday morning, the worst case would have been that the one device would not have been immediately available. Instead, not only was the device unavailable on Monday morning but the entire network. Ensuring that appropriate IT, biomed personnel, or other appropriately trained staff are on-site working with the vendor when network settings are set up or changed is essential to making sure that the information is set up correctly.

Third, the IT resource assumed that the vendor resources had expertise in setting up the device’s network configuration. After the fact, the vendor acknowledged that it had almost no experience in setting up network configurations or knowing what the different values meant. It merely relied on a script provided in the device setup manual. As vendor field resources set up more and more devices, they are becoming much more knowledgeable about network configuration. However, the traditional expertise of many of these field engineers is in mechanically setting up the device and making sure that it performs its intended function, and their networking knowledge is limited. Had the IT resource checked to make sure that the vendor resources understood the various settings and what they meant, the IT resource would have realized that the vendor did not have a lot of networking expertise on-site and would not have relied on the vendor to set up the device on its own.

Taking note of these lessons can ensure the success of your next system setup.

—KO

Ken Olbrish, MSBE, is an enterprise imaging system administrator in the Information Services Department for the Main Line Health System, Suburban Philadelphia, and is a member of 24×7‘s editorial advisory board. For more information, contact .