Emotion detection from medical conversations: employing deep learning models with text and audio features

Abstract

This study explores sentiment analysis techniques applied to both audio and textual data in the context of medical consultations. The sentiment analysis pipeline incorporates shallow level sentiment analysis using NLTK's VADER, followed by deep learning models for both audio and text data. For audio sentiment analysis, Long Short-Term Memory (LSTM) and Bidirectional LSTM (BiLSTM) models are employed, leveraging audio features extracted from the recordings. In contrast, textual sentiment analysis utilizes LSTM, BiLSTM, and BERT models, which are trained on preprocessed transcriptions of the audio content. The effectiveness of these models is assessed through comprehensive experimentation and comparative analysis. The findings shed light on the performance and suitability of various sentiment analysis approaches for medical consultation data, contributing to the advancement of sentiment analysis techniques in healthcare settings.