To put it simply, multi-modal technology makes technology more interactive by utilizing four of the five senses.  This kind of technology lends itself to numerous industries, especially the medical field. Gesture and voice recognition software are becoming players in the medical field like never before, especially in surgery. It really just makes sense, for sanitary and time efficient reasons if nothing else. Without this tech, if a surgeon needs to look at a file or image that is part of the patient’s history, they would have to take of their surgical wear, use a work station, and then re-scrub up. This is not feasible when things are happening quickly in a surgery. With multi-modal technologies, the surgeon can use voice controls to pull up that information either on a monitor or Google Glass, and they don’t even have to stop the work they are doing.

ParallaxAnother aspect of multi-modal technology is that it can inject processes and check points into the surgery to reduce medical mistakes in the operating room. With systems like CHaRMTM, gesture-sensitive checklists recognize the surgeon’s hands using Microsoft Kinect. This interface allows surgeons to easily go through a checklist, where before the checklist was usually just in their head. The predefined checklist improves the safety and accuracy of the operations. The CHaRMTM system also provides alternate checklists should things not go to plan. Which could, quite literally, be a life saver.

Incorporating these multi-modal systems into the operating room will shorten procedure time, ensure team collaboration and communication, enhance surgeon accountability, increase patient engagement, and re-focus the procedure on patient safety. This will allow healthcare experts and hospital professionals to provide a higher quality product to more people with fewer resources.

GlassSurgeonThese technologies also allow you to document patient information like never before. Surgeons can use Google Glass to record surgeries for patient records, training purposes, or even live-stream it to a colleague in order to get recommendations.

Integrating these multi-modal systems into the medical industry would increase safety, security, traceability, and transparency like never before.  Developers who utilize these systems in medical devices allow the end-user to interact with the same data in many different ways. For example, a patient’s data from a routine check-up versus an emergency room visit can be viewed differently, depending on the needs of the doctor, medical staff, and patient. Developing multi-modal integrative technologies opens up the doors of possibility for how data can be used to improve patient outcomes.