Machine translation of English speech: Comparison of multiple algorithms

3.1 Machine translation algorithm based on deep learning

For the recognized speech text, the traditional machine translation is to translate the speech text word-by-word with the help of the word bank of Chinese–English mutual translation [9]. Although this machine translation method is simple and fast, and the general meaning of the translated text can be guaranteed in conventional translation, the effect is poor when translating long English sentences. The first reason is that word-for-word translation often leads to grammatical confusion or even the complete opposite semantics because of grammatical differences between Chinese and English. The second reason is that some of the auxiliary words that do not have specific meanings in English statements are also translated, affecting the coherence of the translation [10].

The emergence of deep learning-based intelligent algorithms has remedied the above-mentioned defects of machine translation. The basic structure of intelligent algorithm-based machine translation methods is shown in Figure 2. The overall structure is divided into an encoder and a decoder. The specific algorithms used in the encoder and decoder are deep learning algorithms, including BP neural network, convolutional neural network, and so on. When the English text is translated, the English text is first converted into an encoding vector by the encoder, and then the encoding vector is converted to Chinese by the decoder.

Figure 2

Basic structure of deep learning-based machine translation algorithm.

3.2 Machine translation algorithm with a LSTM encoder and a RNN decoder

A conventional BP neural network [11] is sufficient to convert a text with a length of n into an encoded vector string. The purpose of converting the text into a coded vector by the encoder in this study is to realize machine translation of the text in subsequent decoding; however, the meaning of the words in the utterance will be affected by the sequence, and the conventional BP network does not consider the influence of the word sequence when encoding the text. In the training and use of the RNN algorithm, the results at the current moment are influenced by the results at the previous moment, which fits well with the property of language that word sequences influence the meaning of words; therefore, the RNN algorithm is used for encoding and decoding machine translation.

However, the encoder faces the problem of gradient explosion when encoding English using the RNN algorithm, which makes the algorithm inefficient and less accurate in training and use. To remedy this deficiency of encoder in machine translation algorithm, this study adopts LSTM to encode English instead of the RNN algorithm. The LSTM algorithm is a kind of the RNN algorithm after introducing forgetting mechanism [12], and the input gate, forget gate, and output gate are the cores of the LSTM algorithm. The forget gate unit can round off the unimportant parts of the text when coding and converting it, highlighting the key points and reducing the number of operations while enhancing the accuracy. The forward calculation formula for encoding English text with LSTM is:

where ε is the English text to be translated; ε _n is the n + 1th word in the English text; x _t is the word vector after processing ε _t with the vector transformation function e ( ⋅ ) ; t is the time step (every time step corresponds to the input moment of the corresponding sequence of words); h t − 1 , h t are hidden states of word vectors whose word sequences are t − 1 and t [13]; f t is the output of the forget gate; b f , U f , and W f are the bias term, input term weight, and forget gate weight in the forget gate, respectively; s t is the output of the recurrent gate; b , U , and W are the bias term, input term weight, and recurrent gate weight in the recurrent gate, respectively; g t is the external input gate unit; b _g , U g , and W g are the bias term, input term weight, and input gate weight in the input gate, respectively; q t is the output gate unit; and b q , U q , and W q are the output gate bias term, input term weight, and output gate weight, respectively.

After the forward operation of equation (2) in the encoder, every word ε t in the English sentence gets the corresponding coding vector a _t , and the coding vector of every word is influenced by the previous word vector, which ensures that the coding vector of the whole sentence contains the influence of word order on the meaning of words. After that, the coding vector needs to be decoded to obtain the corresponding Chinese translation, and the decoding is carried out by the decoder. In order to guarantee the influence of word order on word meaning, the RNN algorithm is used to decode the coding vector, and the forward calculation formula in the hidden layer is

where ω and U are the hidden state at time t − 1 and the weight matrix of the input ε t word vector at time t ; b is the bias term; Z t is a word whose order is t in the translated sentence; y t − 1 is the vector of a word whose order is t − 1 in the translated sentence; d is the bias term; V is the weight matrix; y ˆ t is the probability distribution of different translated characters [14]; and z t is the hidden state in the decoder.

4.1 Experimental environment

The experiments were conducted on a lab server [15] with the following configuration: Windows 7 system, I7 processor, and 16G memory.

4.2 Experimental data

In this study, the English data set was crawled from the Internet by a crawler software. The pronouncers involved in this data set cover most age groups from 12 to 70 years old. A total of 10,000 sentences with clean and clear pronunciation were selected, among which, 9,000 sentences were randomly selected as the training samples, and the remaining 1,000 sentences were used as the test sample. The experimenter read the sentences aloud and collected the speech feature parameters of the sentences. The sampling rate was set as 16 kHz, and 16-bit encoding was used. Some of the sentences that were read aloud are as follows.

1. How are you?;

2. How can I get to the airport, please;

3. What’s the weather like today.

4.3 Experimental projects

4.4 Evaluation criteria

In this study, the translation was evaluated by the word error rate, and the calculation formula is

where X is the number of incorrect words substituted; Y is the number of incorrect words deleted; Z is the number of incorrect words inserted; and P is the number of all words in the test set. In addition to the word error rate, the evaluation criteria for the practical application of the machine translation algorithm also include the overall translation level of the sentence. Therefore, in this study, ten professional translators were invited to evaluate the translation, and the translation was scored according to the expression and grammatical structure. The total score was 100 points. The average score was taken as the final result.

4.5 Experimental results

As shown in Table 1, the word error rate of English speech recognition by artificial recognition was 7.53%, and the recognition time was 34 min; the word error rate of English speech recognition by BP neural network was 1.54%, and the recognition time was 8 min. The comparison of the speech recognition effect of the two methods showed that BP neural network had a lower word error rate and spent less time than artificial recognition. The reason for the above result was that computers were more efficient in terms of computational efficiency, and when faced with a large number of speech sounds in the test set, people could not maintain their attention for a long time, resulting in a higher word error rate and longer recognition time; however, BP neural network used computers to recognize speech sounds, which was not only computationally efficient, but also does not have the disadvantage of inattentiveness.

It was seen from Table 2 that the translation word error rate of the machine translation algorithm with BP neural network for encoder and RNN for decoder was 25.4%; the translation word error rate of the machine translation algorithm with RNN for both encoder and decoder was 18.7%; the translation word error rate of the machine translation algorithm with LSTM for encoder and RNN for decoder was 3.2%. The BP–RNN algorithm had the highest error rate of translated words, the RNN–RNN algorithm was the second, and the LSTM–RNN algorithm had the lowest error rate of translated words.

The two test results shown above were obtained from testing in the laboratory using a predetermined set of samples after training. All three machine translation algorithms utilized the neural network algorithm. The neural network, when learning with a limited number of training samples, will appear to have increasingly better performance on the samples, but once it steps out of the pre-prepared samples and tests on out-of-sample data, the performance decreases instead, i.e., it falls into an overfitting state, resulting in no generalization performance of the whole algorithm. In To test the generalization performance of the three machine translation algorithms after training, i.e., the actual application performance, this study selected ten volunteers to read aloud English scripts with different word counts and translated the speech read by the volunteers using the three machine translation algorithms that have been trained respectively. The word error rate and translation score of the final translation are shown in Table 3. As the length of the English speech to be translated increased, the scores of the translations obtained by the three translation algorithms decreased, and the BP–RNN algorithm had the largest decrease, the RNN–RNN algorithm had the second largest decrease, and the LSTM–RNN algorithm had the smallest decrease. Under the same speech length, the LSTM–RNN algorithm had the highest translation score, followed by the RNN–RNN algorithm and the BP–RNN algorithm.

It was seen from Table 3 that the word error rate of the translations obtained by all three translation algorithms increased as the length of English speech increased; the BP–RNN algorithm had the largest increase, the RNN–RNN algorithm had the second largest increase, and the LSTM–RNN algorithm nearly had no increase. Under the same speech length, the BP–RNN algorithm had the highest word error rate, followed by the RNN–RNN algorithm and the LSTM–RNN algorithm.