On the correction of errors in English grammar by deep learning

3.1 An analysis of the need for error correction in English grammar

English grammatical error correction refers to putting forward the grammatical problems existing in the written sentences and correcting these errors to get a correct sentence. From the analysis of daily English grammar problems, we can see that the common grammatical errors mainly include nouns, verbs, prepositions, singular and plural nouns, subject predicate, and so on. Due to the complexity of grammatical errors, it is quite difficult to correct them automatically. In English grammar automatic error correction, the common difficulties are as follows:

1. There is more than one error in a complete single sentence. These multiple errors will make it more difficult for the machine to recognize. It is easy to form a misjudgment based on the whole sentence.

2. As there are many types of errors involved in the use of grammar, and different types of errors may cross each other to form some new complex grammatical errors. Even some errors with low frequency may also form a variety of grammatical errors.

3. The composition of many words is often correct, but it is wrong to put them in the specific context of the text. Therefore, when judging, we must make a comprehensive judgment combined with the context, which undoubtedly increases the difficulty of English grammatical error recognition.

The correction of English grammar is not limited to the correction of closed grammatical errors such as articles, prepositions, and verb forms but should be extended to open grammatical errors, such as word order, collocation, and word selection. According to the common English grammatical errors, they can be divided into five levels: textual structure, semantics, pragmatics, syntax, and vocabulary. In the past, grammatical error correction in English translation would not be limited to the partial grammatical errors in the text but was regarded as a monolingual translation task from the sentence level of the whole text. Traditional English grammar error correction often has the following two obvious shortcomings: first, the exposure deviation between the sequence and the sequence model is large, that is, when it is difficult to output the correct prediction in a certain time step of the model, it will affect the subsequent time step of the whole model, and it becomes difficult to return to the correct track smoothly; and the second is that the performance of word granularity prediction has a decisive impact on the loss of the whole model, so this type of model usually adopts phrase or sentence level when selecting evaluation indicators.

3.2 Basic principles of generative confrontation network learning model

In order to solve the problem of English grammar error correction, this study proposes a confrontation network model based on deep learning. The birth of generative confrontation network model is the inevitable product of the upsurge of modern artificial intelligence. At present, artificial intelligence is divided into perception stage and cognitive stage. Machines have their own understanding of the world in the cognitive stage, but this understanding is the internal performance that cannot be directly measured, and confrontation network can deepen the understanding of artificial intelligence. As a typical network learning model in deep learning, neural network can solve the problem of data training to a certain extent in the face of a large number of data analysis and data required by submission calculation ability. Using this idea of game confrontation, the training process of game confrontation is realized through generator and discriminator so as to continuously optimize the learning to get the best model parameters. The general calculation flow of generative countermeasure network is shown in Figure 2.

Figure 2

General calculation flow of countermeasure network.

In the framework of confrontation network learning, there is a binary classification model as a discriminator. In the process of confrontation learning, the generator and the discriminator are part of cooperative training, and they promote each other through training. At the beginning of the training, the error correction of manually labeled grammatical sentences is taken as positive samples, and the error correction sentences generated by the iterative update generator are taken as negative samples. The discriminator can improve its discrimination ability by discriminating the two samples. The training process of the discriminator will be fed back to the generator, and the generator will continuously improve its own parameters to cheat the discriminator by correcting sentences with higher quality. The overall confrontation learning framework of the confrontation network model is shown in Figure 3.

Figure 3

General learning framework of confrontation network model.

It can be seen from the analysis in Figure 1 that in the confrontation learning framework, the generator adopts the encoder model of the sequence order framework, and the discriminator is a binary classification model. The basic principle of the model is derived from convolutional neural network. The learning and training process of the confrontation learning framework can be understood as that the generator generates the correct sentence timed step based on the random strategy. After generating a complete correct sentence after correction, it inputs the uncorrected wrong sentences into the discriminator, and the discriminator discriminates these sentences. For the output result of the discriminator, the model is fed back to the generator in the form of probability value to encourage it to determine the correct error correction sentence. The learning process of confrontation network is to adjust the parameters of the discriminator constantly in order to obtain greater probability value.

Let a generator G be trained based on parallel corpus ( X , Y ) . the original error sentences not corrected are represented by x , and the error correction target sentences generated by the generator are y .

In time step t , the generator’s state is s , the prefix sequence generated in the current time step is ( y 1 , y 2 , ⋯ , y t − 1 ) , and the next generated word y _t is the deterministic state transition of the generator based on the random strategy model. The action a of y _t word generated by generator can be understood as the probability of y _1:t−1 → y _t:1 from state is 1.

3.3 Optimization of English grammar error correction model based on deep learning

The error sentences at the source end are x, the target sentences are y after manual correction, and the sentences generated by model generator are y′. They constitute the modeling problem of English grammar correction. Convolutional neural network as the basic component of discriminator can give full play to its superior performance in classification tasks. The model of the countermeasure network is optimized and improved based on convolutional neural network. The improved structure of the discriminator of the model of the antijamming network is shown in Figure 4.

Figure 4

Discriminator structure of confrontation network model.

In Figure 4, (x, y) is the input sentence of discriminator network. The first step of discriminator model is to combine x and y word vectors into an input representation similar to two-dimensional image. In this image, the width and height corresponds to the length of sentences y and x, respectively. Let the ith word in x correspond to the jth word in y. According to the previous image formation rules, the position feature mapping of the input matrix (i, j) can be expressed as

The convolution operation based on convolution neural network can capture the corresponding relationship between x and y in the window as

In equation (2), a is σ nonlinear activation function and its relu is

After completing the convolution operation of 3 × 3, the pooling operation of 2 × 2 window size is given by

By repeating the above steps, we can capture the fragment correspondence between x and y at different levels of abstraction. The feature image pixels are evaluated and stitched, and classified by a fully connected network layer so that the discriminator outputs a probability value, which is in the range of [0,1]. Set the initial state of generating and correcting sentences to be corrected is s 0 = ( 〈 BOS 〉 ) , the generator generates the correct sentence as y _1:T , and the generator generates the next y _t word according to policy G _θ (θ is the generator parameter). This operation process can be expressed as the action value function R D G θ ( y 1 : t − 1 , x , y t ) , and thus the generator target function formula can be expressed as

The value of manually labeled probability in the output sentence of the model pair discriminator is R D G θ ( y 1 : T − 1 , x , y T ) , and its calculation formula is

In equation (6), b(x, y _1:T ) is the baseline value of the output probability of the model, which is introduced to reduce the estimation of reward value. Here, its value can be set to 0.5.

In order to make the discriminator have a clear meaning for the corrected prefix sequence, the model Carroll search strategy is used to generate the word sequence of the corrected sentence. At the end of the search, it is necessary to reduce the estimation variance in the reward value through N repeated Monte Carlo searches in order to avoid the search spaces index level becoming too large. Let the sequence length of the ith search be T _i , the current state obtained by Monte Carlo search be (y _1:t ,x), and the single sequence generated by G _θ strategy be y t + 1 : T i i .

In the training phase of the confrontation network learning model, the discriminator and the generator improve each other’s performance. The parameter results generated by the discriminator can be updated to the generator to make the generator generate higher quality corrective sentences. This further trains the discriminator, which can be represented by the minimum loss function of the discriminator

Combined with equation (5), the parameter generation formula of the objective function can be further deduced by

Finally, the gradient optimization algorithm is used to update the generator parameters. Let the learning efficiency of step h be a _h , and the formula of parameter update is

Thus, the whole learning process of confrontation training is completed.

4.1 Index setting

The selected evaluation indexes mainly include precision (P value) evaluation, recall (R value) evaluation, and the comprehensive evaluation index F _0.5 value of the two indexes. Precision evaluation is the proportion of the correct degree of the whole module modification action, and recall evaluation is the proportion of the degree of error in the modification of the module. The prediction results of the model are positive whereas those of the positive ones are TP and FN are the negative ones. The prediction results are negative whereas those of positive ones are FP and those of negative ones are TN. The formula of P value and R value can be obtained as

Let the set of sentences to be corrected be n, the correct sentence i to be modified be g _i , and the correct sentence i to be modified be e _i . The P and R values of equations (12) and (13) can be converted into the following definitions:

In the model, the accuracy rate and recall rate are mutually restricted indicators, so F _0.5 value is selected to comprehensively evaluate the comprehensive performance of the model:

In the judgment on English grammar, if a correct sentence is wrongly judged and corrected, it is better not to judge the play. The introduction of F _0.5 value can weaken the contribution of R value and enhance the contribution of P value.

For the final output of model error correction, it is essentially the result of scoring and ranking, so it is necessary to implement quantitative evaluation on the output sequence of the model, which is expressed by the mean reciprocal ranking (MRR). Suppose there are n sentences that need to be corrected, the result list of the i correct correction is the r _i , and the calculation formula of MRR is:

The optimized algorithm flow is shown in Figure 5.

Figure 5

Optimized algorithm flow chart.

4.2 Comparative analysis of algorithm results

This paper selected the data of CoNLL-2014 test set and Lang-8 test set to compare the P value, R value, F _0.5 value, and MRR index of the proposed algorithm model and the traditional machine translation transformer method. The analysis results are shown in Table 1.

Here, five common grammatical errors are selected for error correction, specifically articles, prepositions, verb forms, singular and plural nouns, and subject predicate consistency. The choice of a, an, or the in the article can be inferred from the context. The preposition may have errors in the target word, such as the target word of while the correct word is to. When adding the target word information, it may interfere with the selection of the model. The subject predicate consistency problem can be inferred from the context. Singular and plural nouns may be words similar to water with the same singular and plural, or words with large differences in singular and plural in person, which can give a model according to the word information to achieve better results. There are also different tenses or large differences in dynamics, so its own information can be used to help predict.

Furthermore, the comparison chart of the evaluation indexes of the two methods is drawn (Figure 6).

Figure 6

Comparison and analysis of accuracy of three methods.

It can be seen from Table 1 and Figure 5 that the proposed algorithm of confrontation network model based on deep learning has better error correction effect and higher error correction performance in five common error types. The difference between the proposed algorithm and transformer method is +08.87.

Furthermore, the data of Lang-8 test set was used for training comparative analysis of the two algorithm models. The overall P-value, R-value, F _0.5-value and MRR-value are compared here, and specific grammatical errors are no longer classified. The test results are shown in Table 2.

It can be further seen from Table 2 that the scores of each index of the proposed model algorithm are better than the traditional machine translation method.

Considering that in the literature, specifically [13] and [14], certain typical representative significance in English grammatical error correction are mentioned, the methods of this study are compared with those in the two works. In all, 1,000 groups of data in the CoNLL-2014 data set were selected for testing to verify the different performance of the three syntax error correction methods. The comparative analysis results are shown in Figure 6.

As can be seen from Figure 6, the accuracy of the English grammar error correction method proposed in this paper and the other two methods in the literature decreases with the increase of the number of data. Among them, the decline range of the method proposed in this paper is lower, and it has higher accuracy in the case of the same data set.