How Much Simulation Detail is Needed?

Right at the start of a usability study there a number of details that should not be overlooked. This is especially true when the study involves specialised user testing under very specific environments. There are project planning questions that can be tricky to answer, in particular – “how much detail should the simulation have?”, and “is it worth spending more money replicating real-life fidelity?”There is never one single answer for such questions and in the end, it will be a balance between costs and need. However, there is one aspect to usability testing which usually requires attention – simulating the intended use environment. This is because usability tests are normally conducted in simulated-use conditions that could affect the users’ interactions with the medical device. For example, a device with audio alarms that needs to be used in a noisy hospital ward and where the device will be “competing” with so many other sources of sounds, including other medical devices. If not tested with a degree of environment fidelity, how else would anyone know if the device is safe to use? Such as, not knowing if the device has stopped working properly because the alarm is not being heard.Recently THAY Medical performed a summative usability study. The objective was to determine the product effectiveness, intuitiveness, user-friendliness and use-safety.  The unusual aspect of this study was that the complete use of the product had to be done by three users from three different user groups. To complete one whole test session, each user had to perform specific tasks and pass the product to the next participant. Therefore, these participants had to be evaluated individually and also as a team.Not only did the multi-user test present a challenge, but there were key considerations regarding the test environment. In principle, the basic requirements for the study were simple - Product samples, models, some basic surgical instruments, bins to dispose waste materials and of course the intended users (all specialist healthcare professionals). If only considering this basic setup, the location planning would have been also straight forward and testing in an ordinary lab room would have been enough.  In real-life, the product is used in operating theatres and handled in both, the sterile and non-sterile areas. These areas have strict protocols that are in place to prevent cross-contamination, therefore this has an impact on how the product is used, leading to use errors and ultimately impacting on safety. During the test, participants were placed in separate environments and were only allowed to use parts of the product specific to the non-sterile or sterile areas.There were also consideration with regards to the surrounding equipment, such as placement areas of surgical trolleys, surgical instruments and waste bins. Lighting was another consideration, since lack of adequate lighting would almost certainly have a negative effect in the test results. As well, a potential hazard causing impact on use safety.These and other considerations dictated that the product had to be tested in an environment that replicated, with a high degree of fidelity the real use. Therefore, a state-of-the-art operating theatre from a major hospital in London was hired for the project.By hiring these facilities, it allowed THAY Medical to formulate a real life situation with healthcare professionals as users. This provided the confidence that the environment of use did not create any bias in study results.For more information about multi-user usability studies, please get in contact with THAY Medical. We would love to hear from you.The staff at THAY Medical