Development of a comprehensive fatigue detection model for beekeeping activities based on deep learning and EEG signals

초록

Worker fatigue remains a pressing concern in agricultural production due to its negative effect on workplace safety. Current fatigue detection efforts have primarily targeted the mental fatigue of agricultural machinery operators and the physical fatigue associated with the manual handling of items and harvesting. In beekeeping, beekeepers not only inspect the honeycombs inside beehives for honey collection but also frequently handle the beehives themselves to observe the lower layers or to transport them to different locations, resulting in both mental and physical fatigue, which can lead to property loss and threaten their personal safety. To address this, a comprehensive fatigue detection model for beekeepers is developed in the present study, using electroencephalogram (EEG) signals and deep learning to assess the combined effect of physical and mental fatigue. EEG signals were recorded from participants under two transportation conditions (manual handling and with an auxiliary device). The participants also provided a self-assessment of their fatigue levels according to a five- category scale (no, mild, moderate, severe, and extreme fatigue). The EEG signals were preprocessed, including artifact removal and data cropping, and then a Short-Time Fourier Transform (STFT) method was used to generate time-frequency images from the signals. Data augmentation techniques such as random masking and Gaussian noise were also applied. A deep learning model employing a three-dimensional (3D) convolutional neural network (CNN) with three convolutional layers was then constructed and its performance compared to existing 3D-ResNet-18 and 3D-DenseNet-121 models. A convolutional block attention module (CBAM) was incorporated into these models to capture important spatial and channel weights from the feature images during training (3D-CNN(CBAM), 3D-ResNet-18(CBAM), 3D-DenseNet-121(CBAM)). A mask layer was also applied to the feature images after adding the CBAM to manually adjust the spatial weights and focus the models on mid- to high-frequency signals for fatigue classification (MASK-3DCNN(CBAM), MASK-3D-ResNet-18(CBAM), MASK-3DDenseNet-121(CBAM)). Following training and validation, the model with the highest fatigue prediction accuracy was found to be MASK-3DCNN(CBAM), with an average accuracy of 94.9% and a detection accuracy higher than 93.3 % for each of the five fatigue levels. Given that the inference time for the individual validation data across all models did not exceed 0.01 s, MASK-3DCNN(CBAM) was selected as the optimal model for its simple structure and balanced detection capability across different fatigue levels. This fatigue detection method can rapidly and accurately assess the fatigue levels of beekeepers at work, with significant implications for safety and fatigue assessment in large-scale beekeeping operations.

키워드