2.1

BP neural network

Artificial neural network is an algorithm that imitates biological neuron. It can be simply divided into input layer, hidden layer and output layer. And the hidden layer can be divided into multi-layer structure according to the complexity of the model^4,5. BP neural network is an error back propagation neural network, which trains weights along the direction of reducing error. But it is not a feedback neural network.

The principle of BP neural network is to use the output error to calculate the error of the upper layer, and then use the calculated error of the upper layer to calculate the error of the previous layer, so as to calculate the error of each layer, and then modify the connection weight according to the error value, and repeat the process until the output error reaches the convergence condition, then, the model training is over.

The learning rules of BP neural network generally use the steepest descent method, through back propagation, that is, layer by layer forward propagation, constantly dynamically adjust the connection weight, so as to minimize the global error.

The step of BP neural network is simple. Firstly, the error of output layer is obtained by forward propagation, and then the convergence condition is judged. If it is reached, it will be over, otherwise, the error will be back propagated.

According to the above, the global error objective function of BP neural network is as following:

where, L represents the global error, n represents the number of samples, k represents the number of categories. y_ij represents the output value of the category j for the sample i. d_ij represents the expected value of the category j of the sample i. BP neural network model needs to minimize the global error value of L.

2.2

Steepest descent algorithm

In BP neural network, the steepest descent algorithm is usually used to minimize the error. The principle of the steepest descent algorithm is to use the opposite direction of the gradient to iterate until the convergence condition is reached. The steepest descent algorithm is to execute the current descent with the fastest speed, not the global descent with the fastest speed. Therefore, it is necessary to determine the length of descent while determining the direction of each iteration^6,7.

Assume that f(x) is the objective function, d_k is the searched direction of x_k, i.e., d_k = –∇f(x_k), λ_k is the search step size. Then,

Assume that H is the Hessian matrix of f(x), and expand f(x_k + λ_kd_k) at x_k with Taylor expansion of second order as following:

The derivation of equation (4) is performed to λ_k, i.e.,

Optimal search step size can be found as following:

The implementation steps of steepest descent method are as follows:

Step 1: Calculate the value of function f(x₀), gradient ∇f(x₀), search direction and H₀ of the initial point x₀, and set k = 0;

Step 2: Do the line search x_k+1 = x_k + λ_kd_k. λ_k is attained by equation (6), then calculate ∇f(x_k+1).

Step 3: Judge if it is on the convergence condition ||x_k+1 – x_k||_∞ < ɛ or ||∇f(x_k+1)||_∞ < ɛ or k > k_max : If yes, the iteration is over and x_k+1 is output; otherwise, calculate f(x_k+1), d_k+1 and H_k+1, set k = k+1. And turn to step 2.

If the Hessian matrix of the objective function is computationally complex or there is no second derivative, the search step size λ_k can be calculated by Armijo delimitation method or golden section method.

2.3

Levenberg-Marquardt algorithm

Levenberg Marquardt (LM) is a widely used optimization algorithm⁸. The convergence speed of LM algorithm is generally higher than the steepest descent method, which simplifies the difficulty of Newton method in calculating Hessian matrix and solves the problem that the approximate Hessian matrix of Gauss Newton method is not full of rank^9,10.

Assume that M(x, θ) is a model function, and set

Taylor expansion is performed at x and according to the least square method,

where,

Assume that J(x) is the Jacobian matrix, then,

If M(x,t) fits well, i.e. f_i(x)≈0, then

Therefore,

Implementation steps of Levenberg-Marquardt algorithm are as following:

Step 1: Calculate f(x₀), g₀ and H₀ of the initial point x₀, and set μ = 0.001, k = 0;

Step 2: Calculate x_k+1 = x_k – (H_k – μI)^–1 g_k, f(x_k+1), g_k+1 and H_k+1 by equation (14);

Step 3: Judge whether ||x_k+1 – x_k||_∞ < ɛ or k ≥ k_max is on the convergence condition. If yes, the iteration is over and x_k+1 is output; otherwise, go to Step 4.

Step 4: Judge whether condition |f(x_k+1)|| < ||f(x)|| is satisfied, if yes, μ = μ/10, or, μ = μ*10;

Step 5: set k=k+1, and go to Step 2.

2.4

LM-BP model

LM-BP model is based on BP neural network, using LM algorithm to calculate the minimum error. Firstly, the weights of the model need to be modified by a large number of sample data in the training stage, and then it can be applied to the real environment. The model is shown as Figure 1.

Figure 1.

LM-BP model.

According to the mental health level identified by the model, it can accurately and quickly predict the mental status of teenagers, so as to improve the work effectiveness of relevant departments.

3.1

Inversion analysis of optimization algorithm

In this paper, two equations are constructed for inversion experiments to compare the convergence speed and accuracy of steepest descent method and LM algorithm. The equations are defined as follows:

The variable x of f₁(x) means to get 100 equidistant values between 0 and 10. The true coefficient is [3.4, 2.3, 5.3, 6.2], the initial coefficient is [3.9, 0.8, 41.3, 3.2]. The experimental results of inversion are shown in Table 1.

Table 1.

Coefficient inversion comparison of f1(x).

The variable x of f₂(x) is equidistant between [0, 10] and takes 100 values of them. The true coefficient is [3.4, 2.3, 5.3, 6.2, 1.5, 8.4], the initial coefficient is [8.1, 1.3, 4.3, 3.2, 3.5, 4.4]. The experimental result is shown in Table 2.

Table 2.

Coefficient inversion comparison of f2(x).

From above, it can be proved that the convergence speed of steepest descent algorithm is much slower than LM algorithm. When the complexity of the equation increases, the accuracy of inversion coefficient of steepest descent method is not as good as LM algorithm.

3.2

Model analysis

Emotional data¹¹, the hotel reviews is used as experimental data. This data has 1172 negative emotion data and 5358 positive emotion data. Nlpir tool is used to process Chinese word segmentation. Precision, recall and F1 values were selected to evaluate the experimental results. In this paper, the steepest descent method and LM algorithm are used to compare and analyse the sample data. The result is shown in Table 3.

Table 3.

Contrastive analysis.

According to the above contrastive analysis, Precision, Recall and F1 of model LM-BP is higher than model SD-BP.