Technology Testing in the Medical Environment: Intuitive Software and Workstation Achieve Good to Very Good Results in Comprehensive OR Usability Study

Tuesday, 2020 January 14
In the course of digitalization, imaging systems in operating rooms are becoming ever more numerous and complex. Nevertheless, to ensure the optimum use of these systems, hospitals must take into account the difficulties their employees face when using various auxiliary devices. If Operating Room (OR) personnel are occupied with operating numerous complex systems, they can be sidetracked from their primary task in supporting the actual surgical procedure itself. To address this issue, the respective software and hardware components must be very user friendly. For this reason, EIZO GmbH put its Caliop software and Surgical Panels from the CuratOR product portfolio to the test. The usability study started in early 2018 in cooperation with the M3i GmbH Industry-in-Clinic platform, and encompasses multiple test runs through mid-2020. The two test runs to date have achieved good to very good findings and already resulted in changes; the next session is planned for the end of 2019. When the entire study is complete, the results will be summarized in a white paper and made available to the public.
Zitat

"By better understanding these issues, we can optimize our products and minimize related risks during the actual operation."

James Berge, Product Manager at EIZO GmbH

A fast and clear overview of all relevant information during a surgical procedure can be critical to the success of the operation. The Surgical Panel offers OR personnel a central interface to internal hospital systems such as HIS and PACS and when used in conjunction with the Caliop software, it simplifies management of video. The two elements serve as a core for interacting with hospital systems from within the operating room. "We are keenly interested in finding out what difficulties might occur when using our hardware and software in the OR environment," said James Berge, Product Manager at EIZO GmbH. "By better understanding these issues, we can optimize our products and minimize related risks during the actual operation." For this reason, EIZO began preparing a usability study in early 2018 with the industry-in-clinic platform M3i, which specializes in customer-oriented product development. While EIZO provided its Caliop software and Surgical Panels for the test, M3i took care of the agile integration of nursing staff and surgeons as testers.
Abschnitt 2

Initial test groups deliver very good results.

Two series of full-fledged usability tests using different test groups have been performed since late 2018. M3i organized access to a one-way mirror lab with two fully equipped operating rooms at the Institute of Emergency Medicine and Medical Management (INM) at the Medical Center of the University of Munich. The study participants were asked to perform a series of common tasks in simulated OR situations without any prior instruction on using the hardware and software. "In the study, we examined the standard workflow of a surgical procedure in all phases: from system start to patient assignment and video management, through to archiving the recorded images, videos and reports from the simulated OR," said Berge. “In addition to the one-way mirror, we observed and recorded participants with cameras positioned at two angles, and we captured the operation of the Caliop software and use of the touchscreen integrated into the Surgical Panel. This allowed us to later follow and analyze the testers’ actions and responses." The examinations focused primarily on participant interaction with the video management elements, and whether medical intervention was impacted by operation of the Caliop software.
Abschnitt 2 1
Afterwards, the testers were interviewed about their use of the system and were asked to evaluate various aspects of the system on a scale of 1 (easy to use) to 5 (difficult to use) in order to determine its usability based on the System Usability Scale (SUS). The first test run achieved ratings of very good to excellent user friendliness. "The technical affinity within the first test group seemed particularly high, enabling these users to grasp operation very quickly," said Berge. "For this reason we set the bar much higher for the second test group by specifically selecting OR personnel with little prior knowledge of technical systems. Nevertheless, our products still achieved good to very good results according to SUS." Based on the initial data, some useful insight was gained regarding the intuitive operation of the Caliop software and Surgical Panels. For example, the initial layout icon was changed to avoid confusion with other functions. In addition, ambiguous terms such as "delete" and "remove" were formulated more clearly and in some cases replaced with checkboxes. Using feedback from the participants, the way in which patient names are displayed was adapted to facilitate quick and certain identification of the patient, and the selection of recordings for archiving was redesigned to be easier to use. "In addition to the insight we gain from clinical experts, finding lead users through this type of cooperation is particularly important for the continued development of these products using agile methods." said James Berge. "Not only do they provide feedback on using the current software, they offer useful suggestions and ideas for new functions and a better understanding of the challenges facing medical technologists."

Continuous improvement of software and hardware

Additional testing is planned for the coming months, and evaluation of the data should be finalized in the second quarter of 2020. "When this study is completed we hope to use these findings in the further development of all of our CuratOR products," said Berge in summary. "We also plan to present the results of this study in a white paper sometime before the DMEA (Connecting Digital Health) in Berlin in April of next year." While OR personnel were the focus of the previous usability study, EIZO intends to expand its questions to hospital IT administrators in a separate survey. The experts for medical imaging hope to thereby ensure that their video management systems are of even greater benefit in the increasingly interconnected OR environment.
Caliop Keyvisual

Industry-in-clinic platform M3i GmbH in Munich, Germany, has made it its mission to simplify and accelerate access to clinical resources for the development of medical technology products. Frame agreements with hospitals enable the integration of hospital personnel, clinical experts, spaces such as simulation ORs, clinical trial centers, and clinical data that would otherwise be difficult to access due to barriers such as compliance, data security, conflicts of interest, and additional regulations.

We use cookies to optimize our website and improve it continuously. By continuing to use the website, you agree to the use of cookies.Privacy Policy