Download PDF
Research Article  |  Open Access  |  29 Jul 2024

A novel fatigue driving detection method based on whale optimization and Attention-enhanced GRU

Views: 1042 |  Downloads: 30 |  Cited:  0
Intell Robot 2024;4(3):230-43.
10.20517/ir.2024.15 |  © The Author(s) 2024.
Author Information
Article Notes
Cite This Article

Abstract

Fatigue driving has emerged as the predominant causative factor for road traffic safety accidents. The fatigue driving detection method, derived from laboratory simulation data, faces challenges related to imbalanced data distribution and limited recognition accuracy in practical scenarios. In this study, we introduce a novel approach utilizing a gated recurrent neural network method, employing whale optimization algorithm for fatigue driving identification. Additionally, we incorporate an attention mechanism to enhance identification accuracy. Initially, this study focuses on the driver's operational behavior under authentic vehicular conditions. Subsequently, it employs wavelet energy entropy, scale entropy, and singular entropy analysis to extract the fatigue-related features from the driver's operational behavior. Subsequently, this study adopts the cross-validation recursive feature elimination method to derive the optimal fatigue feature index about operational behavior. To effectively capture long-range dependence relationships, this study employs the gated recurrent unit neural network method. Lastly, an attention mechanism is incorporated in this study to concentrate on pivotal features within the data sequence of driving behavior. It assigns greater weight to crucial information, mitigating information loss caused by the extended temporal sequence. Experimental results obtained from real vehicle data demonstrate that the proposed method achieves an accuracy of 89.84% in third-level fatigue driving detection, with an omission rate of 10.99%. These findings affirm the feasibility of the approach presented in this study.

Keywords

Traffic safety, fatigue driving, operational behavior, whale optimization, neural network

1. INTRODUCTION

Based on data from the traffic management department, the occurrence of road traffic accidents in China has surpassed 20, 000 over the last five years[1]. It is worth noting that nearly 20% of these accidents can be attributed to drowsy driving, so drowsy driving is one of the main causes of road traffic accidents. Hence, offering early warnings to fatigued drivers holds significant practical merit in safeguarding lives and properties. Presently, fatigue driving detection methods are categorized into three groups, focusing on driver physiological characteristics, facial features, and driving operation behavior characteristics as detection targets[2]. Notably, methods reliant on drivers' physiological features involve intrusive detection, potentially impacting driving operations due to the use of wearable data collection devices[3]. Facial feature-based detection methods face susceptibility to external factors such as weather and light, leading to reduced stability[4]. In contrast, behavior-centric detection methods remain unaffected by environmental changes[5], ensuring low-cost feasibility for practical applications. Consequently, these methods have emerged as a prominent research area in contemporary fatigue driving detection. In recent years, researchers have compared the parameters of steering wheel angle (SWA) and lateral vehicle displacement in both fatigued and alert states of drivers, discovering that these parameters can serve as indicators of driver fatigue. Wu et al. found that the fatigue characteristics extracted from SWA are more reflective of the driver's fatigue state compared to lane departure[6]. Forsman et al. conducted a simulated driving experiment and concluded that changes in SWA can be used to develop an economical and efficient fatigue driving detection device[7]. Li et al. proposed a dual time window method to extract approximate entropy features of the SWA and used a binary decision classifier to identify the driver's fatigue state, achieving an average recognition accuracy of 78.01%[8]. Li et al. extracted approximate entropy features of the SWA combined with support vector machine (SVM), showing an average recognition accuracy of 84.6% for three levels of fatigue[9]. Li et al. extracted various features from SWA signals and used a decision tree-like classifier, achieving an average recognition accuracy of 82.07%[10]. Li et al. employed robust feature learning and a fuzzy recurrent neural network, achieving a recognition accuracy of 87.30%[11]. Cai employed a random forest algorithm with an average recognition accuracy of 78.5%[12]. These studies demonstrate the significance of SWA and lateral vehicle displacement features in fatigue detection. The combination of different algorithms and feature extraction methods can significantly improve recognition accuracy. Fatigue induces psychological and physiological changes in drivers, leading to decreased control accuracy and abnormal driving behaviors. For instance, in the awake state, SWA exhibits frequent, small fluctuations. As fatigue sets in, the amplitude of these fluctuations increases, and in extreme fatigue, SWA may show significant fluctuations with periods of stationary motion. Similarly, throttle opening remains stable with minor fluctuations when the driver is awake. Under fatigue, control diminishes, resulting in pronounced throttle fluctuations, and in very fatigued states, there may be prolonged stationary throttle with reduced fluctuation amplitude. The prevailing literature on fatigue driving predominantly focuses on steering wheel dynamics, with limited exploration of other driving behaviors. Notably, most extracted features are of statistical nature, and achieving a recognition accuracy exceeding 80% is primarily based on laboratory simulation data, lacking robust verification in real-world road conditions. Consequently, this study addresses this gap by investigating six driving behaviors under authentic vehicle conditions: SWA, vehicle speed, transverse angular velocity, throttle opening, and vehicle transverse and longitudinal acceleration. Discriminative indexes characterizing fatigue levels are extracted, selected through the cross-validation recursive feature elimination method, and utilized to construct a fatigue driving detection model. This model, integrating whale optimization and the Attention-gated recurrent unit (GRU) neural network, facilitates real-time monitoring and early warning of driver fatigue.

2. METHODS

2.1 Feature extraction and optimization

2.1.1 Driving behavior feature extraction

The extracted driving behavior features are shown in Figure 1. This paper explores time-frequency domain features, including wavelet energy entropy, wavelet scale entropy, and wavelet singular entropy. The chosen wavelet basis function is db6 within the dbN series, and a three-layer wavelet packet decomposition is employed. The computational steps are outlined as follows: (1) Utilize the db6 wavelet basis function to decompose driving behavior data into three layers of wavelet packets, yielding eight subbands. Reconstruct the wavelet subband components to ensure the new driving behavior data's length matches that of the original data; (2) Determine the two-parameter number for each node, square it to yield the node's energy value, and then sum the energy across nodes to compute the total wavelet energy. Subsequently, derive the wavelet energy entropy based on the total wavelet energy; (3) Compute the wavelet scale entropy for each subband; (4) Extract singular values, construct a vector, generate the singular value spectrum, and perform singular value decomposition to obtain the wavelet singular entropy.

A novel fatigue driving detection method based on whale optimization and Attention-enhanced GRU

Figure 1. Block diagram of driving behavior feature extraction.

2.1.2 Driving behavior feature extraction

The recursive feature elimination with cross-validation (RFECV) method involves iterative training of data using a base model, eliminating features with low weights based on weight coefficients in each round until the candidate subset meets termination conditions. Given challenges such as fluctuations in real car driving behavior data, significant noise, and sample imbalance, the paper adopts the random forest as the base model to address these issues. The algorithm consists of the following steps: (a) Train models using all driving behavior features, calculate feature importance, and rank them. Extract the top $$ S_i $$ most important features for each subset $$ S_i $$, where i ranges from 1 to S; (b) Split the training set into a new training set and a validation set. Train the model using the new training set and all features, and then evaluate the model with the validation set; (c) Input the filtered features into the random forest as the initial feature subset and calculate feature importance. Remove features with the lowest importance from the current subset to obtain a new feature subset. Repeat this process, inputting the new subset into the random forest, calculating the importance of each feature, and determining the classification accuracy using cross-validation; (d) Recursively repeat Step 3 until the feature subset is empty. Ultimately, obtain k feature subsets with varying numbers of features, selecting the subset with the highest classification accuracy as the optimal feature combination.

2.2 Attention-GRU identification of fatigue levels

2.2.1 GRU neural network

The driving behavior data is inherently sequential, and the connections among the hidden layers of recurrent neural networks involve integrating the output of the hidden layers from the previous moment into the current network state[13]. This sequential network structure is effective in preserving dependencies within the data. Notably, the GRU is particularly skilled at capturing long-range dependencies and efficiently mitigating the challenges of gradient explosion and gradient disappearance observed in basic recurrent neural networks. The architectural representation of GRU neurons is illustrated in Figure 2.

A novel fatigue driving detection method based on whale optimization and Attention-enhanced GRU

Figure 2. Structure of GRU neurons. GRU: Gated recurrent unit.

where $$ X_t $$ represents the input at moment t, $$ R_t $$ indicates the reset gate; $$ Z_t $$ stands for the update gate; $$ H_t $$ denotes the hidden state; $$ \tilde{H}_t $$ refers to the candidate hidden state. According to the model structure of GRU, it can be calculated by:

$$ \begin{equation} R_t = \sigma \left( X_t W_{xr} + H_{t-1} W_{hr} + b_r \right) \end{equation} $$

$$ \begin{equation} Z_t = \sigma \left( X_t W_{xz} + H_{t-1} W_{hz} + b_z \right) \end{equation} $$

where $$ R_t $$ and $$ Z_t $$ are the relationship functions between the input feature $$ X_t $$ at the current moment and the hidden variable $$ H_{t-1} $$ at the previous moment, using the sigmoid activation function so that the threshold is set within the range of 0 to 1. Where $$ W_{xr} $$, $$ W_{hr} $$, $$ W_{xz} $$, and $$ W_{hz} $$ are the matrices to be trained, and $$ b_{r} $$ and $$ b_{z} $$ are the bias terms to be trained.

$$ \begin{equation} \widetilde{H}_t = \tanh(X_t W_{xh} + (R_t \cdot H_{t-1}) W_{hh} + b_z) \end{equation} $$

$$ \begin{equation} H_t = Z_t \otimes H_{t-1} + (1 - Z_t) \otimes \widetilde{H}_t \end{equation} $$

Where $$ \tilde{H}_t $$ denotes the candidate hidden state, which can also be expressed as the present information, and is determined by the past information $$ H_{t-1} $$ over the reset gate together with the current information. $$ H_{t} $$ incorporates both long-term and short-term memory outputs.

2.2.2 Attention mechanism

Within this research, we integrate the Attention mechanism to focus on crucial features within the sequence of driving behaviors. This entails assigning a higher weight to important information and filtering out low-value information. The calculation process diagram for the Attention mechanism is illustrated in Figure 3.

A novel fatigue driving detection method based on whale optimization and Attention-enhanced GRU

Figure 3. Attention calculation flow.

(a) Calculation stage 1

The inner product value of $$ Query $$ and $$ key $$ is found by the dot product method, and the similarity $$ S_{i} $$ between them is counted.

$$ \begin{equation} Sim_i = Key_i \cdot Query \end{equation} $$

(b) Calculation stage 2

Normalization is performed by $$ Softmax $$, which uses an internal mechanism to further emphasize the weights of key elements.

$$ \begin{equation} a_i = \text{Soft}(\text{Sim}_i) \end{equation} $$

(c) Calculation stage 3

Weighted summation of Value with $$ a_i $$.

$$ \begin{equation} \text{Attention}(\text{Query}, \text{Source}) = \sum\limits_{i=1}^{L_x} a_i \cdot \text{Value}_i \end{equation} $$

2.2.3 Whale optimization algorithm

Whale optimization algorithm (WOA) is a group intelligence optimization algorithm proposed by Mirjalili and Lewis in 2016, which originates from the simulation of the hunting behavior of whale groups in nature. The whole algorithm process includes three stages: encircling prey, bubble netting and searching for prey[14].

(a) Surround the prey

Assuming that in k-dimensional space, there is already a whale that finds the best position to surround its prey[15], other whales will choose this position to approach, and the mathematical model equation is established as follows:

$$ \begin{equation} X_{k}^{j+1} = X_{k}^{\text{best}} - A \cdot D_{k} \end{equation} $$

$$ \begin{equation} D_k = \left| c \cdot x_k^{best} - x_k \right| \end{equation} $$

where $$ x_{best} $$ denotes the current optimal individual whale position; $$ x_j $$ denotes represents the current individual whale position[16]; The position that the whale individual affected by the position of the optimal whale individual will reach in the next moment is set to be $$ x_{j+1} $$. $$ x_{k_{j+1}} $$is the kth component of $$ x_{j+1} $$.

$$ \begin{equation} C = 2r_1, \quad A = a \cdot (2 \cdot r_2 - 1) a = 2 \cdot (1 - \frac{t}{T_{\text{max}}}) \end{equation} $$

where $$ r_1 $$ and $$ r_2 $$ are random variables in the interval [0, 1]; the value of a decreases linearly from 2 to 0 as the number of iterations $$ t $$ increases; and $$ T_{\text{max}} $$ denotes the maximum number of iterations.

(b) Bubble net predation

Whales have two ways to contract the envelope and swim spirally toward their prey when they drive the encircling prey. The spiral wanders toward the prey using the spiral to update the position to represent this roundup behavior. The mathematical model equation is established as follows:

$$ \begin{equation} X_{k}^{j+1} = D_k \cdot e^{b_l} \cdot cos(2 \pi t) + X_{k}^{best} \end{equation} $$

$$ \begin{equation} D_{k} = \left| x_{k}^{best} - x_{k}^{j} \right| \end{equation} $$

where $$ D_k $$ denotes the optimal whale-to-prey spacing; $$ b $$ represents the logarithmic spiral shape constant; and $$ l $$ indicates a random number uniformly distributed in the interval [-1, 1]. The contraction surround mechanism, which is basically the same as the formula of the mathematical model to surround the prey[17], differs in that the value interval of $$ A $$ is adjusted from [-a, a] to [-1, 1]. Then, one of these two methods is chosen with a 50% probability in the whale feeding process[18], with:

$$ \begin{equation} X_k^{j+1} = \begin{cases} D_k \cdot e^{bl} \cdot \cos(2\pi t) + X_k^{best} & \text{if } p \leq 0.5 \\ X_k^{best} - AD_k & \text{if } p \geq 0.5 \end{cases} \end{equation} $$

where the value interval of $$ p $$ is [0, 1].

(c) Search for predation

The value of $$ A $$ determines whether the whale swims toward the optimal individual or toward a random individual, when $$ \lvert A \rvert \leq 1 $$, the whale chooses to swim toward the optimal individual[19], as provided in Equations (13) and (14); when $$ \lvert A \rvert > 1 $$, the whale chooses to swim toward a random individual, which will enhance the search ability of the whale population as a whole[20], and the mathematical model equation is expressed as follows:

$$ \begin{equation} D_k = \left| C \cdot X_k^{rand} - X_k^j \right| \end{equation} $$

$$ \begin{equation} X_k^{j+1} = X_k^{rand} - A \cdot D_k \end{equation} $$

where $$ X_k^{\text{rand}} $$ is a random position vector.

2.2.4 WOA-Attention-GRU fatigue state recognition

The WOA is a heuristic optimization algorithm based on the principles of natural selection and biological behavior. Inspired by the hunting behavior of whales, it efficiently identifies the global optimal solution within few iterations[21]. Furthermore, WOA circumvents the intricacies of parameter tuning, thus reducing the risk of overfitting, which greatly benefits our handling of the problems with multiple parameters. In our work, the role of WOA is to optimize the Attention-GRU model; to be precise, it seeks the optimal model parameters, thereby maximizing the model's accuracy on the fatigue driving behavior dataset. The algorithmic progression of the fatigue driving detection[22], grounded in the synergy of the WOA and Attention-GRU, is visually depicted in Figure 4. It encompasses three main components: whale optimization, processing of driving behavior data, and the establishment of the Attention-GRU neural network model. In the whale optimization phase, the mean squared difference [mean squared error (MSE)] between the predicted fatigue level by the Attention-GRU model and the true fatigue level serves as the fitness function. The aim is to ascertain a set of hyperparameters that minimize this mean squared difference when fed into the Attention-GRU model[23]. The mean squared deviation is expressed as follows:

$$ \begin{equation} MSE = \frac{1}{n} \sum\limits_{i=1}^{n} \left( P_{predict} - P_{true} \right)^2 \end{equation} $$

A novel fatigue driving detection method based on whale optimization and Attention-enhanced GRU

Figure 4. WOA-Attention-GRU algorithm flow (adapted from Li et al., 2023[24]). WOA: Whale optimization algorithm; GRU: gated recurrent unit.

Where $$ P_{predict} $$ denotes the predicted value of fatigue level, $$ P_{true} $$ indicates the true value of fatigue level, and $$ n $$ is the total number of fatigue samples.

In the WOA-Attention-GRU model, the overall framework can be divided into three main components: the WOA part, the data processing part, and the Attention-GRU model part [Figure 4][24]. In the data processing part, we extract features from driving behavior data and conduct relevant analysis and selection. The main steps include data collection and preprocessing: collecting driving behavior data, including vehicle speed, SWA, acceleration, etc., and cleaning the data to remove outliers and noise, ensuring the data's accuracy and reliability. Subsequently, operational behavior features are extracted from the preprocessed data, including but not limited to the rate of change of the SWA, vehicle acceleration, and braking frequency. These features are then selected using correlation analysis methods closely related to driving fatigue. Through correlation and preference analysis, the most representative features are selected as input variables for the model. In the whale optimization part, WOA is used to optimize the hyperparameters of the Attention-GRU model. The main steps include initialization: WOA encodes the initial values, including the number of iterations, batch size, number of neurons working in the GRU layer, and dropout rate. Fitness value calculation: the fitness values of the initialized whale population are calculated, and the global optimum is updated. Iterative optimization: based on the fitness values, the positions of the whale individuals are updated, gradually approaching the global optimum, and finally outputting the optimal hyperparameters of the Attention-GRU model. In the Attention-GRU model part, the model is trained and tested using the hyperparameters optimized by WOA. The main steps include model training: training the model using the training set provided by the data processing part. The attention mechanism focuses on important features in the sequences of driving behavior data, assigning greater weight to important information and reducing information loss. Model testing: testing the trained model using the test set to evaluate the model's performance. The model's predictive performance is then assessed by calculating the MSE. Through these steps, we can effectively detect the fatigue state of drivers, providing more accurate and reliable detection results. “Optimal hyperparameters” refer to the best set of parameters that can minimize the MSE between the predicted fatigue level and the actual fatigue level. These hyperparameters include the number of iterations, batch size, the number of neurons in each GRU layer, and the dropout rate. The network model structure of Attention-GRU is presented in Figure 5.

A novel fatigue driving detection method based on whale optimization and Attention-enhanced GRU

Figure 5. Structure of Attention-GRU network model. GRU: Gated recurrent unit.

2.2.5 Fatigue state recognition based on Transformer

Transformer excels at handling long-range dependencies in sequence data, which is particularly beneficial for time series analysis. We use the standard Transformer architecture, including self-attention mechanism. The number of layers, number of attention heads, and hidden layer dimensions were adjusted to optimize performance on our dataset. The model is trained using the same driving behavior data set and adopts the same preprocessing method as the WOA-Attention-GRU model. The results show that although Transformer performs well in capturing long-range dependencies, in the context of driving behavior analysis, the method combining WOA and Attention mechanisms in the GRU model provides more targeted feature extraction and optimization.

3. RESULTS

3.1 Experimental data

In this paper, the experimental data sampling path is a 270 km long section of Beijing-Harbin Expressway from Beijing to Qinhuangdao [Figure 6]. The number of participants is 8, the acquisition time is 1-3 h, and the acquisition frequency is 100 Hz. The collected data are sliced according to the standard of about 1 min, and the consistency of the sliced driving behavior data with the driver's facial video is determined according to the synchronization pulse signal. Each segment was scored as 0 (awake), 1 (fatigued) and 2 (very fatigued) according to the driver's facial fatigue score. A new fatigue driving sample dataset was obtained. The vehicle speed below 60 km/h in the dataset is considered as indicating slow sections, and the steering wheel turning angle over 20° is considered as indicative of overtaking lane change. There are 243 sober samples, 71 fatigue samples and 30 very fatigue samples after excluding these abnormal data, totaling 237 samples. Considering the unbalanced and too few samples of the three-level fatigue samples, the smote method was used to expand 171 fatigue samples and 212 very fatigue samples, each containing 12 dimensions and 6, 000 lines of operation behavior data.

A novel fatigue driving detection method based on whale optimization and Attention-enhanced GRU

Figure 6. Experimental driving path.

3.2 Data analysis

The fatigue state induces psychological and physiological changes in the driver, leading to a decrease in the driver's control accuracy over the vehicle and subsequent abnormal operating behaviors. Consequently, monitoring indicators related to driving operation behaviors allows for a real-time assessment of the driver's state. The SWA, being the device most directly manipulated by the driver, is also the most frequently operated. The data is illustrated in Figure 7[24]. In the awake state, the SWA exhibits frequent fluctuations with a small amplitude. In the fatigue state, the fluctuation amplitude increases, and in a very fatigued state, the SWA may show stationary motion with significant fluctuations. The driver modulates vehicle speed through the throttle opening and brake pedal. Throttle opening (CAN_throttle) data [Figure 8][24], remains stable for a period with small fluctuations in the awake state. In the fatigue state, diminished control accuracy results in pronounced throttle fluctuations. In a very fatigued state, delayed driver consciousness may lead to a prolonged stationary throttle, accompanied by a decrease in fluctuation amplitude.

A novel fatigue driving detection method based on whale optimization and Attention-enhanced GRU

Figure 7. Waveform of SWA (adapted from Li et al., 2023[24]). SWA: Steering wheel angle.

A novel fatigue driving detection method based on whale optimization and Attention-enhanced GRU

Figure 8. Waveform of throttle opening (adapted from Li et al., 2023[24]).

Cross-swing angular velocity (Yaw Rate) serves as a crucial indicator reflecting the vehicle's stability and driving smoothness [Figure 9][24]. The sustained stability of vehicle speed and the limited acceleration and deceleration contribute to understanding the driver's state. Speed (Speed) data is visualized in Figure 10[24]. Horizontal and longitudinal acceleration (X_Accel and Y_Accel) denote the motion acceleration of the car vertically and horizontally in the driving direction, respectively, with data presented in Figures 11[24] and 12[24].

A novel fatigue driving detection method based on whale optimization and Attention-enhanced GRU

Figure 9. Waveform of transverse angular velocity (adapted from Li et al., 2023[24]).

A novel fatigue driving detection method based on whale optimization and Attention-enhanced GRU

Figure 10. Waveform of vehicle speed (adapted from Li et al., 2023[24]).

A novel fatigue driving detection method based on whale optimization and Attention-enhanced GRU

Figure 11. Waveform of lateral acceleration (adapted from Li et al., 2023 [24]).

A novel fatigue driving detection method based on whale optimization and Attention-enhanced GRU

Figure 12. Waveform of longitudinal acceleration (adapted from Li et al., 2023[24]).

The variations observed in the waveforms presented in the six driving behavior data graphs above indicate the presence of numerous indicators associated with fatigue characteristics within the driver's operational behavior. As a result, this paper employs six types of driving behavior data collected from real vehicles - comprising SWA, vehicle speed, vehicle transverse and longitudinal acceleration, and throttle opening - as the experimental dataset.

3.3 Experimental results

The fatigue driving recognition model, WOA-Attention-GRU, developed in this study, utilizes a cross-validation method with ten sample clusters. To evaluate the performance of the proposed Attention-GRU method, we compared it with the Transformer-based model on the same dataset. The results are presented in Table 1[24]. In the experimental results, the prediction accuracy for the fatigue state is indeed significantly lower than for the awake and very fatigued states. This discrepancy can be attributed to the subtler behavioral indicators associated with the fatigue state, which makes it more challenging to distinguish compared to the more distinct characteristics of alertness and extreme fatigue. The awake state is characterized by highly responsive and consistent driving behaviors, while the very fatigued state exhibits more pronounced deviations and irregularities due to extreme tiredness. In contrast, the fatigue state presents less obvious signs, such as slight deviations or minor lapses in attention, which can be harder to detect accurately. Additionally, individual differences in how drivers exhibit fatigue can contribute to this challenge. While some drivers may show clear signs of fatigue, others may have more subtle or varied manifestations, making it difficult for the model to generalize and predict the fatigue state with high accuracy.

Table 1

Comparison of recognition accuracy of WOA-Attention-GRU algorithm (adapted from Li et al., 2023[24])

ProjectionsWOA-Attention-GRUrecognition accuracyAttention-GRUrecognition accuracyGRUrecognition accuracyTransformer-basedrecognition accuracy
Awake94.44%84.09%84.38%83.42%
Fatigued81.63%79.59%64.06%70.52%
Very fatigued93.44%86.79%84%84.26%
Overall percentages89.84%83.56%75.34%82.65%

3.4 Methodological evaluation

This paper introduces four evaluation metrics - Precision, Recall, Condition positive, and F1-score - alongside Accuracy to comprehensively assess the fatigue recognition model. The results of the WOA-Attention-GRU fatigue recognition model are presented in Table 2[24].

Table 2

WOA-Attention-GRU fatigue recognition model detection results (adapted from Li et al., 2023[24])

Actual testProjectionsActual sample size
AwakeFatiguedVery fatigued
Awake34 (TN)8 (FP)1 (FR)43
Fatigued2 (FN)40 (TP)3 (FR)45
Very fatigued0 (FN)1 (FP)57 (TR)58
Predicted sample size364961146

In the confusion matrix presented in Table 2, $$ TN $$ denotes samples accurately predicted as awake, $$ TP $$ indicates samples correctly predicted as fatigued, and $$ TR $$ represents samples accurately predicted as very fatigued, $$ FN $$ stands for fatigued and very fatigued samples falsely predicted as awake, $$ FP $$ refers to awake or very fatigued samples falsely predicted as fatigued, and $$ FR $$ points to awake or fatigued samples predicted as very fatigued. Using the awake sample as an illustration, the equations for each evaluation metric are established below:

a. Exact rate: indicates the probability that actual positive samples are among those predicted to be positive.

$$ \begin{equation} P = \frac{TN}{TN + FN} \end{equation} $$

b. Recall rate: indicates the probability that samples predicted to be positive are among those actually positive.

$$ \begin{equation} R = \frac{TN}{TN + FP + FR} \end{equation} $$

c. Underreporting rate:

$$ \begin{equation} CP = 1 - \frac{TN}{TN + FP + FR} \end{equation} $$

d. F1-score:

$$ \begin{equation} F1 = \frac{2 \cdot R \cdot P}{R + P} \end{equation} $$

Applying the definitions given in the equation above, the evaluation results for the Attention-GRU fatigue driving detection model, optimized by the whale algorithm in this research, are presented in Table 3[24]. The model achieves an accuracy rate of 89.84%, a recall rate of 88.77%, a miss rate of 10.99%, and an F1-score of 88.99%.

Table 3

Evaluation results of the WOA-Attention-GRU fatigue driving detection model (adapted from Li et al., 2023[24])

Type of samplePrecisionRecallCondition positiveF1-score
Awake94.44%79.07%20.13%86.07%
Fatigued81.63%88.89%11.11%85.11%
Very fatigued93.44%98.28%1.72%95.80%
Overall percentages89.84%88.75%10.99%88.99%

4. DISCUSSION

This study presents a fatigue driving recognition method based on a WOA-enhanced Attention-GRU model. After optimization through the WOA, the overall recognition accuracy of the Attention-GRU model for fatigue driving reaches 89.84%. This represents a 6% improvement over the non-optimized Attention-GRU model, a 14% enhancement over the GRU model, and approximately an 11% increase compared to fatigue driving detection methods that focus solely on the real vehicle steering angle. The missed detection rate is 10.99%. The proposed fatigue driving recognition method utilizes real car driving operation data, which enhances its practical engineering applicability. However, this study does not account for individual driver differences. In future research, it is imperative to expand the fatigue driving sample database and explore the variations in operational behavior among different drivers to improve the robustness and generalizability of the fatigue driving recognition model. Additionally, to enhance the model's performance in long-term monitoring scenarios, more extensive studies are planned to investigate how drivers adapt to the fatigue monitoring system over time, tracking behavioral changes post-implementation.

5. CONCLUSIONS

In this paper, we developed a fatigue driving recognition model, WOA-Attention-GRU, which demonstrated promising results in detecting various states of driver fatigue. The model was validated using real measured data, ensuring its reliability and relevance to practical driving scenarios. However, we acknowledge that the generalizability of our findings can be further enhanced by testing the model on larger datasets. Future work will involve collecting more extensive data for further verification and validation of the proposed method to ensure robustness and wider applicability across different driving scenarios. Furthermore, acknowledging the importance of addressing long-term monitoring challenges, we plan to update and improve the monitoring system based on actual usage feedback, ensuring that it can adapt to evolving driver needs and behaviors. Special algorithm adjustments or model updates may be necessary to address time-related changes effectively.

DECLARATIONS

Acknowledgment

I hereby declare that some of the results presented in this manuscript [Manuscript ID: (IR-2024-15)] are derived from our previously published conference paper titled “A Fatigue Driving Recognition Method Based on WOA-Attention-GRU” (DOI: 10.1109/RAIIC59453.2023.10281143). All co-authors of this manuscript have been informed and agree to the submission and publication of this work. Additionally, we acknowledge the conference paper within this new manuscript by appropriately citing it.

Authors’ contributions

Made substantial contributions to the conception and design of the study and performed data analysis and interpretation: Li Z, Li M

Performed process guidance responsibility for the planning and execution of the study and the evolution of overarching research aims, critical review, and material support; review and revise the original draft: Shi L, Li D

Availability of data and materials

Not applicable.

Financial support and sponsorship

The Natural Science Foundation of Chongqing (CSTB2023NSCQMSX0760, cstc2021ycjh-bfzxm0071, cstc2020jcyj-msxmX0818), and the Science Technology Research Program of Chongqing Municipal Education Commission (KJZD-M202301502).

Conflicts of interest

Li Z is an Editorial Board Member of the journal Intelligence & Robotics; Shi L is affiliated with Qingling Motors Co. Ltd; while the other authors have declared that they have no conflicts of interest.

Ethical approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Copyright

© The Author(s) 2024.

REFERENCES

1. Jiang K, Ling F, Feng Z, Wang K, Shao C. Why do drivers continue driving while fatigued? An application of the theory of planned behavior. Transport Res A Pol 2017;98:141-9.

2. MacLean AW. Chapter 40 - Sleep and driving. In: Handbook of behavioral neuroscience. Elsevier; 2019. pp. 611–22.

3. Ding L. Fatigue driving detection method based on steering wheel monitoring. Agric Mach Use Maint 2020:25.

4. Al-Libawy H, Al-Ataby A, Al-Nuaimy W, Al-Taee MA. Modular design of fatigue detection in naturalistic driving environments. Accid Anal Prev 2018;120:188-94.

5. Luo H, Qiu T, Liu C, Huang P. Research on fatigue driving detection using forehead EEG based on adaptive multi-scale entropy. Biomed Signal Process 2019;51:50-8.

6. Wu C, Zhang H, Mao Z, Chu X, Yan X. A Model for Identifying Fatigue Status of Vehicle Drivers Based on Driving Operation. China Saf Sci J 2007;17:162-5.

7. Forsman PM, Vila BJ, Short RA, Mott CG, Van Dongen HPA. Efficient driver drowsiness detection at moderate levels of drowsiness. Accid Anal Prev 2013;50:341-50.

8. Li Z, Li SE, Li R, Cheng B, Shi J. Online detection of driver fatigue using steering wheel angles for real driving conditions. Sensors 2017;17:495.

9. Li Z, Li R, Li S, Wang W, Cheng B. Driver fatigue recognition based on approximated entropy and complexity of steering wheel angle. J Automot Safety Energy 2016;7:279-84.

10. Li Z, Li S, Li R, Cheng B, Shi J. Driver fatigue detection using approximate entropic features of steering wheel angle from real driving data. Int J Robot Autom 2017;32:291-8.

11. Li Z, Yang Q, Chen S, et al. A fuzzy recurrent neural network for driver fatigue detection based on steering-wheel angle sensor data. Int J Distrib Sens N 2019;15:1-12.

12. Cai S, Du C, Zhou S, Wang Y. Fatigue driving state detection based on vehicle operation data. J Trans Syst Eng Inform Technol 2020;20:77-82.

13. Chen Y, Ge P, Wang G, Weng G, Chen H. An overview of intelligent image segmentation using active contour models. Intell Robot 2023;3:23-55.

14. Mirjalili S, Lewis A. The whale optimization algorithm. Adv Eng Softw 2016;95:51-67.

15. Chazhoor AAP, Ho ESL, Gao B, Woo WL. Deep transfer learning benchmark for plastic waste classification. Intell Robot 2022;2:1-19.

16. Li Z, Chen L, Nie L, Yang SX. A novel learning model of driver fatigue features representation for steering wheel angle. IEEE Tran Veh Technol 2022;71:269-81.

17. Wang Y, Huang R, Guo L. Eye gaze pattern analysis for fatigue detection based on GP-BCNN with ESM. Pattern Recogn Lett 2019;123:61-74.

18. Li Z, Chen L, Peng J, Wu Y. Automatic detection of driver fatigue using driving operation information for transportation safety. Sensors 2017;17:1212.

19. Guo Y, Wang Y, Yang IH, Sycara K. Reinforcement learning methods for network-based transfer parameter selection. Intell Robot 2023;3:402-19.

20. Wang D, Yang SX. Intelligent feature extraction, data fusion and detection of concrete bridge cracks: current development and challenges. Intell Robot 2022;2:391-406.

21. Zhang K, Wang S, Jia N, Zhao L, Han C, Li L. Integrating visual large language model and reasoning chain for driver behavior analysis and risk assessment. Accid Anal Prev 2024;198:107497.

22. Michelaraki E, Katrakazas C, Kaiser S, Brijs T, Yannis G. Real-time monitoring of driver distraction: state-of-the-art and future insights. Accid Anal Prev 2023;192:107241.

23. Ping P, Huang C, Ding W, Liu Y, Chiyomi M, Kazuya T. Distracted driving detection based on the fusion of deep learning and causal reasoning. Inform Fusion 2023;89:121-42.

24. Li M, Li Z, Shi L, Li D. A fatigue driving recognition method based on WOA-Attention-GRU. In: 2023 2nd International Conference on Robotics, Artificial Intelligence and Intelligent Control (RAIIC); 2023 Aug 11-13; Mianyang, China. IEEE; 2023. pp. 62-5.

Cite This Article

Research Article
Open Access
A novel fatigue driving detection method based on whale optimization and Attention-enhanced GRU
Zuojin LiZuojin Li, ... Dongyang Li

How to Cite

Li, Z.; Li M.; Shi L.; Li D. A novel fatigue driving detection method based on whale optimization and Attention-enhanced GRU. Intell. Robot. 2024, 4, 230-43. http://dx.doi.org/10.20517/ir.2024.15

Download Citation

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click on download.

Export Citation File:

Type of Import

Tips on Downloading Citation

This feature enables you to download the bibliographic information (also called citation data, header data, or metadata) for the articles on our site.

Citation Manager File Format

Use the radio buttons to choose how to format the bibliographic data you're harvesting. Several citation manager formats are available, including EndNote and BibTex.

Type of Import

If you have citation management software installed on your computer your Web browser should be able to import metadata directly into your reference database.

Direct Import: When the Direct Import option is selected (the default state), a dialogue box will give you the option to Save or Open the downloaded citation data. Choosing Open will either launch your citation manager or give you a choice of applications with which to use the metadata. The Save option saves the file locally for later use.

Indirect Import: When the Indirect Import option is selected, the metadata is displayed and may be copied and pasted as needed.

About This Article

Special Issue

© The Author(s) 2024. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, sharing, adaptation, distribution and reproduction in any medium or format, for any purpose, even commercially, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Data & Comments

Data

Views
1042
Downloads
30
Citations
0
Comments
0
6

Comments

Comments must be written in English. Spam, offensive content, impersonation, and private information will not be permitted. If any comment is reported and identified as inappropriate content by OAE staff, the comment will be removed without notice. If you have any queries or need any help, please contact us at support@oaepublish.com.

0
Download PDF
Share This Article
Scan the QR code for reading!
See Updates
Contents
Figures
Related
Intelligence & Robotics
ISSN 2770-3541 (Online)
Follow Us

Portico

All published articles are preserved here permanently:

https://www.portico.org/publishers/oae/

Portico

All published articles are preserved here permanently:

https://www.portico.org/publishers/oae/