How do you train healthcare professionals timely, efficiently and precisely when healthcare resources worldwide are stressed like never before? By building and implementing the mixed reality HoloLens application Virtual Lung Experience, Microsoft partner InSpark and pharmaceutical company Chiesi have cocreated a high-impact educational respiratory healthcare solution that improves treatment outcomes for patients with a respiratory disease.
Challenge
Chiesi is a global R&D-focused pharmaceutical company located in Parma, Italy and one of the top 50 pharmaceutical companies in the world, operating internationally with thirty locations worldwide. Chiesi develops medications and educational materials for multiple therapeutic areas, including chronic lung disease like asthma, COPD, and cystic fibrosis. They observed a great need for ways of training respiratory healthcare professionals practically and quickly in new kinds of treatment and medication. They also experienced that effectively treating chronic pulmonary diseases can be a challenge: it demands a shared view on ailment and treatment from both healthcare professionals and the patient that is hard to establish traditionally.
Solution
Chiesi, who have a solid track record in offering other innovative forms of medical education, and mixed reality expert InSpark, a KPN subsidiary and recipient of several accreditations such as Microsoft Azure Expert MSP and Microsoft Solutions Partner designations, decided to join hands. They created the Virtual Lung Experience: an interactive way of shared education and instruction in which students and professionals experience and interact with a detailed lung system in 3D with mixed reality.
The 3D model of the human respiratory system that was created, experienced through HoloLens 2, helps to better understand the breathing process in all its facets. Awareness of this contributes to optimal treatment of patients. It not only helps distinguish between types of abnormal breathing as seen in patients with chronic lung disease, but it also supports professionals in giving patients better instructions in medication use. A specific use case is instructing patients on the correct use of an inhaler. Inhaling respiratory medication properly helps raise the efficacy of the medication and can diminish side effects that impact a patient’s well-being negatively. It also enables Chiesi to raise the profile of their educational market offerings.
Implementation
The Virtual Lung Experience is a HoloLens 2 mixed reality application that presents a 3D model of a human respiratory system in great detail, including the alveoli and the functioning of receptors. The model includes not only animation, for instance to mimic breathing movements and air flow, but it also encompasses audio: one can hear the model breathe. It is an interactive model: the user can influence the model spatially with simple hand gestures. But what gives the Virtual Lung Experience its large educational impact is that it presents a shared 3D experience. Several students or professionals wearing HoloLens 2.0 can see the model simultaneously, each from their own perspective. This makes the Virtual Lung Experience a highly effective shared educational tool, providing an opportunity to raise healthcare standards worldwide.
The experience has seven modules that each caters to a distinct use case. As an example, the Lung Training module is used to train professionals in recognizing different types of breathing sounds that can be used to identify pulmonary diseases. Other modules are for instance used for inhalation techniques. All of these modules are based on extensive research and the hands-on experience of healthcare professionals. The collaborative process between InSpark and Chiesi entailed high-level collaboration. Together we investigated the possibilities for using holograms to transfer knowledge and insight effectively.
Technical aspects
The HoloLens application that InSpark developed was built in Unity using Azure DevOps and various Azure tools. The interaction that was created with the help of Unity is intuitive and delightful. State of the art modelling resulted in extremely detailed and realistic 3D models that can be manipulated by the hands of the participants in a logical, natural way. When the model is manipulated, this is visible to all participants in real time. The shared experience in combination of animations and sound simulation creates an immersive experience that helps understanding the workings of the human respiratory system, and the proper use of medication, in an attractive and instructive way.
A shared educational session can take place in any location, such as universities, hospitals and doctor’s offices. The Chiesi session facilitator brings a set of HoloLens 2 devices to the session. To ensure the Virtual Lung Experience does not rely on access to the location’s network, a dedicated WIFI network is established for the HoloLenses to connect to. Access to this network and the shared experience is controlled by the session facilitator. This means the Virtual Lung Experience works stand-alone, without any technical requirements for the location.
The HoloLens devices are managed through Intune (EMS). Through Intune the HoloLens is configured for use as a Virtual Lung Experience device. The dedicated app is installed, the HoloLens is set to Kiosk mode and the WIFI profile required for the dedicated local network is loaded. Any time a HoloLens is connected to Intune after a session, HoloLenses are automatically updated. This means session facilitators and end users always have a perfectly working, up-to-date HoloLens with zero effort from their part.
The Virtual Lung Experience is designed for a shared experience on location, but also works remotely. An instruction session can be streamed with Dynamics 365 Remote Assist. This enables users to enter the session from anywhere in the world.
Business impact
At this moment, 70 dedicated HoloLens 2 devices are in use by Chiesi for Virtual Lung Experience sessions in the Netherlands, Belgium, Germany, and Austria. In the Netherlands alone about one hundred sessions have already taken place. The impact of these sessions in the perception of Chiesi in these countries has already been considerable: the NPS score for Chiesi’s respiratory business area has significantly increased in this same period. This result can be directly traced back to the introduction of the Virtual Lung Experience.
Based on this success, the rollout of about 50 more HoloLenses is being planned. Furthermore, Chiesi will start providing sessions in 12 more countries. To accelerate the impact of the Virtual Lung Experience on patients and healthcare professionals, the HoloLens application will be offered to medical institutions, to use in other settings that may be beneficial for education and research.
Shared 3D experiences for other healthcare sectors are also already in development. At Chiesi’s request, nephrologists have been testing a beta version of a 3D kidney experience in an academic setting. The application projects a three-dimensional hologram of a transplant patient into space, which can then be viewed in detail using HoloLens 2.
Many other medical uses are on the horizon because the power of a shared experience and its many possible applications are limitless. By combining knowledge from books with the reality of an experience, education gets an extra dimension. The Virtual Lung Experience gives patients, nurses and doctors a better understanding of the functioning of lungs and medicine, leading to a new frontier in education and collaboration, providing opportunities to improve healthcare across the globe.
