SmartCare Unit – Face Expression Monitoring

Gesichtserkennung im Thermalbild



Additionally to being an unpleasant experience, pain is also a source of stress to the human body. Post-operative pain is common after surgery and has a negative impact on the regeneration process, affecting both the duration as well as the effectivity of the regeneration phase. Therefore one of the clinical staff’s tasks is to ensure that patients experience as little pain as possible. While awake patients are able to inform the staff about their pain is not possible if the patients are sleeping or sedated. Nevertheless, sedated and sleeping patients can still experience stressful pain. For this reason such irresponsive patients are monitored by the clinical staff in ICUs and recovery rooms, allowing to administer pain relief drugs if necessary. However, even with up-to-date medical technology there exists no reliable method of measuring pain. Therefore the main pain indicator for the personnel are the facial expressions of the patients which still can indicate if the persons are experiencing pain even if they are sleeping or under the influence of anesthetics.


The goal of the Face Expression Monitoring research project is to enable a computer vision system to assess pain by analyzing thermal infrared images of a patient’s face. To this end, several research problems have to be addressed by the Institute of Imaging & Computer Vision. In a first step, a fast, robust and reliable method for detection and tracking of facial features in thermal infrared videos has to be developed. The challenging part about using thermal infrared imaging for this task is that facial features in this modality can differ severly from features that can be observed in videos taken with regular visible-light cameras. Infrared thermography (IRT) data also has a much lower contrast, further complicating the task of feature detection. Methods for the extraction of relevant facial areas such as the periorbitar or perinasal area from IRT data need to be developed as these regions yield a high amount of information relevant for pain assessment. The developed methods have to meet high precision requirements while still allowing for real-time tracking. The correct classification of the pain expressions observed in the thermal videos has to be performed in a subsequent step. To this end, machile learning algorithms will be trained using clinically acquired reference data from real ICU scenarios. The aforementioned problems and limitations of IRT imaging apply to the classification procedure as well. In a final step, all components will be put together to form a complete computer vision system that can be used by the clinical perosnnel in real intensive care scenarios.



We are currently accepting candidates for bachelor and master theses for this project. Check the thesis topics page or contact the responsible research scientist at Marcin Kopaczka.




  • Bundesministerium für Wirtschaft und Technologie (BMWi), 2015-2018