Vowel Phoneme Segmentation for Speaker Identification Using an ANN-Based Framework

3.1 SOM, PNN, and LVQ

A brief treatment of the three soft computational tools is presented here, with descriptions modified to make it as relevant as possible for the present work.

3.1.1 SOM

This is a special form of ANN trained with unsupervised paradigm of learning. It follows a competitive method of learning that enables it to work effectively as a feature map, classifier, and, at times, filter. SOM has a special property of effectively creating spatially organized “internal representations” of various features of input signals and their abstractions [12]. SOM can be considered as a data visualization technique, i.e., it provides an underlying structure of data [9]. This idea is used in our vowel segmentation technique. If we train the same SOM with different epochs or iteration numbers for a particular epoch, SOM provides a weight vector consisting of the winning neuron along with the neighbors. Thus, with different epochs, different internal segments or patterns of speech signal can be obtained. Suppose we have a one-dimensional field of neurons, and say the LPC samples of spoken word uttered by the speaker has a form

When such an input vector is presented to the field of neurons, the algorithm will start to search the best matching weight vector W_i and thereby identify a neighborhood ℵ_i around the winning neuron. While doing so, it will try to minimize the Euclidian distance ‖X_k−W_i(k)‖. The adaptation of the algorithm will take place according to the relation

where learning rate η_k has the form

Thus, with the change of epoch number, different W_i will be obtained.

3.1.2 PNN

This is based on the statistical principles from Bayes decision strategy and non-parametric kernel-based estimators of probability density functions. PNNs handle data that have spikes and points outside the norm better than other ANNs. Therefore, the PNN is suitable for problems such as phoneme classification [23]. The PNN used in this work has the structure shown in Figure 2. Each pattern unit of the PNN forms a dot product of the input clean phoneme vector X with a weight vector W_i, Z_i=XW_i, and then performs a non-linear operation on Z_i before directing its activation level to the summation layer. Thus, the dot product is followed by a non-linear neuron activation function of the form

Figure 2

Structure of PNN.

where σ is the smoothing parameter. The summation units simply add the inputs from the pattern units that correspond to the category from which the training pattern was selected. The output, or decision, units of the PNN are two-input neurons that produces binary outputs between the two phoneme patterns obtained from the summation layer. The smoothing parameter, σ, plays a very important role in the proper classification of the input phoneme patterns. Because it controls the scale factor of the exponential activation function, its value should be the same for every pattern unit. As described by Specht [23], a small value of σ causes the estimated parent density function to have distinct modes corresponding to the locations of the training samples. A larger value of σ produces a greater degree of interpolation between points. A very large value of σ would cause the estimated density to be Gaussian regardless of the true underlying distribution. However, it has been found that in practical problems, it is not difficult to find a good value of σ, and the misclassification rate does not change dramatically with small changes in σ.

3.2 Linear Prediction Coding

The source filter model of the vocal tract system can be represented by a discrete time-linear time-invariant filter, which is characterized by the system function of the form given in Eq. (5):

where the filter coefficients a_k and b_k change at a rate on the order of 50–100 times per second [19].

The LPC model is based on the mathematical approximation of the vocal tract. At a particular time, t, the speech sample s(t) is represented as a linear sum of the p previous samples [19]. Thus, the linear prediction error sequence or the LP residual provides the source information suppressing the vocal tract information obtained in the form of LPC coefficients [16]. The prediction error sequence possess the form of the output of an FIR linear system whose system function is

This is the prediction error filter, A(z), which is an inverse filter for the vocal tract system, H(z), i.e.,

According to Eq. (7), the zeros of A(z) are the poles of H(z) [19]. The source information obtained from the error sequence can be used as a relevant feature for speaker identification. The representation of source information in the LP residual depends upon the order of prediction. According to Pati and Prasanna [16], for a speech signal sampled at 8 kHz, the LP residual extracted using the LP order in the range of 8–20 best represents the speaker-specific source information. This work uses a prediction order of 20 to extract speaker-specific features for both clean vowel and word, which has provided a satisfactory success rate.

4.1 Proposed SOM- and PNN-Based Vowel Segmentation

In most approaches, the speech signals are demarcated using constant-time segmentation, for example, into 25-ms blocks [4]. Static segmentation shows a risk of losing information on phonemes – different sounds may be merged into single blocks and individual phonemes may be lost completely. Discrete wavelet transform (DWT) is frequently used for such kind of phoneme segmentation [24]. However, DWT has the problem that, at each level, the signals are required to be reconstructed. We approached this issue using an ANN framework that does not need the reconstruction of the signal. The SOM- and PNN-based vowel segmentation technique proposed here is considered novel because such a composite ANN framework provides different segments of underlying structures and takes the most probable decision with the help of a prior knowledge of the vowel pattern and characteristics. Very few works using such ANN-based segmentation have been explored previously [18]. However, in the case of Devanagari-based speech recognition, no such report has been observed, and this work on speaker-recognition system is based on the Assamese language, which is robust enough to deal with variations resulting from linguistic, regional, and cultural backgrounds. The work used a SOM- and PNN-based technique to segment the vowel phoneme from any three phoneme words that the speaker utters. The validation of the vowel segmentation is checked by matching the first formant frequency (F1) of the vowel with the predetermined F1 value. The first formant frequency of all the Assamese vowels are estimated as explained by Sarma and Sarma [21].

The weight vector obtained by training a one-dimensional SOM with the LP features of a word is used in the work. This word is used to perform the segmentation. By training the same SOM block for various numbers of iterations, we get different weight vectors, each of which is considered as a segment of different phonemes constituting the word. From these segments, the relevant speech portions are recognized by pattern matching done with some PNNs that are trained to learn the patterns of all Assamese vowel phonemes. The SOM weight vector extraction algorithm can be summarized in the form of a block diagram shown in Figure 3.

Figure 3

SOM Segmentation Block.

The algorithm for a particular iteration can be mathematically stated as in Table 1. The SOM weight vectors thus extracted are stored as SW1, SW2, SW3, SW4, SW5, and SW6. The SOMs’ role is to provide segmentation boundaries for the phonemes. Here, six different segmentation boundaries were obtained with that many number separate sets of weight vectors. Figure 4 shows how SOM weight vectors change with the change of iteration number.

Table 1

SOM Weight Vector Extraction Algorithm.

1.	Input: Spoken word S of size m × n, sampling frequency fs, duration T second
2.	Preprocess the signal using preprocessing algorithms described in Section 5.2
3.	Initialize P = 20, order of linear prediction
3.	Find the coefficients of a Pth-order linear predictor FIR filter, =−a(2)Q(n−1)−a(3)Q(n−2)−…−a(p+1)Q(n−p) that predicts the current value of the real-valued preprocessed time series Q, based on past samples.
4.	Store a = [1, a(2), a(3), …, a(p+1)]
5.	Take an topology map whose neurons are arranged in a 1×(P+1)-dimensional hexagonal pattern.
6.	Initialize weight Wi(k) to a small random number.
7.	Initialize learning parameter, η and neighbors, ℵI(k)
8.	for k = 1 to (p+1) do pick a(k) find winning neuron as, ‖a(k) − Wi(k)‖ = min(1≤i≤(p+1)) {‖a(k) − Wi(k)‖} Update synaptic vectors of the winning cluster, Wi(k) = Wi(k)+ηk(a(k) − Wi(k)) Update ηk, ℵI(k)
9.	Store updated weight Wi(k) as SWj, where j = 1:6

Figure 4

SOM Weight Vectors.

Four PNNs trained with clean vowel phonemes were used for identifying the patterns of Assamese vowel speech. Here, a two-class PNN-based classification is performed, where four PNNs are trained with two clean vowel phonemes and are named PNN1, PNN2, PNN3, and PNN4, i.e., the output classes of PNN1 are /i/ and /u/, the output classes of PNN2 are /e/ and /o/, and so forth. These four PNNs are used sequentially to identify the segmented vowel phonemes. Clean vowels are recorded from five male and five female speakers used subsequently as the inputs in the input layer to the PNN, enabling it to provide each neuron the scope of learning the patterns and to group them per the derived decision. This happens in the pattern layer of the PNN. These speakers are again asked to record five variations of eight different vowel utterances under at least three to five types of background noise content. The details of the data sets used are provided in Section 5.1. These samples are used for training and testing purposes. The PNN learning algorithm can be stated as in Table 2.

Table 2

PNN Learning Algorithm.

1.	Statement: Classify input vowel patterns X into two category of vowel, VOWEL-A and VOWEL-B
2.	Initialize: Smoothing parameter σ=0.111 (Determined from observation of successful learning)
3.	Output of each pattern unit,
	Zi=X·Wi
	(Wi is the weight vector)
4.	Find neuron activation function by performing non-linear operation of the form
	g(Zi) = exp[(Zi − 1)/σ2]
5.	Sum all the g(Zi) for category VOWEL-A and do the same for category VOWEL-B
6.	Take binary decision for the two summation outputs with variable weight given by –
	where,
	hA and hB = Priori probability of occurrence of pattern from
	VOWEL-A and VOWEL-B, respectively
	I = Loss associated with wrong decision
	nA and nB=No of patterns in VOWEL-A and VOWEL-B, respectively,
	which is 10 for both categories

A two-step decision is taken by the recognition algorithm. First, the vowel segment is matched with the PNN patterns. Next, its first formant frequency F1 is checked, whether it lies in the predetermined range or not. The PNN- and F1-based recognition algorithm for a particular vowel /i/ can be stated as in Table 3. This process is repeated for the complete set of samples. Testing is done after training as well as the validation of the extent of learning taking place in the framework within the stipulated time.

4.2 Proposed LVQ Codebook-Based Speaker Identification

The speaker-specific information obtained from the LP method is used to design the LVQ-based code book. Clean vowels recorded from every speaker was first passed to the feature extraction block, and the resulting feature vector was used to train the LVQ network. LVQ is a method for training competitive layers in a supervised manner. In an LVQ network, a competitive layer learns to classify input vectors into target classes chosen by the user, unlike the strictly competitive layer possessed by the SOM [10]. The code book training uses the LVQ1 algorithm described in Section 3.1 and given by Kohonen et al. [13]. The LVQ code book provides a unique code for every speaker based on LP residual features. The speaker-recognition process uses the features of human vocal tract, which are different for different individuals. The LPC parameters can model the characteristics of the human vocal tract through the mathematical approximation of the vocal tract transfer function. Therefore, the LPC features were found to be suitable for the work. Clean vowel phonemes such as /i/, /u/, etc. recorded from five different male and female speakers were used to train the LVQ code book. Testing samples were created by performing the speech recording of the selected speakers under at least three to five background noise variations.

5.1 Speaker Database and Speech Samples Collection

The database used for this work has 10 speakers (five male and five female) with the associated variations as reported earlier. In the first phase, the clean vowel phonemes such as /i/, /u/, etc. uttered by the 10 different speakers were recorded, on which LPC analysis was performed; the prediction error sequence was codified in the form of an LVQ codebook. The second phase of recording covered three phoneme words containing the vowel phonemes /xit/, /xir/, /xik/, /xis/, /dukh/, /dur/, /dut/, /dub/, /bex/, /bed/, /bes/, /bel/, /mon/, /mok/, /mor/, /mot/, /khεl/, /khεp/, /khεd/, /khεr/, /raon/, /raokh/, /raoth/, /raox/, /labh/, /laz/, /lath/, /lakh/, /nool/, /nood/, /nookh/, /noom/, etc. The words were recorded from the same male and female speakers because these were used as testing words for the speaker-identification process. We have covered five variations of words for each eight vowels of the Assamese language. This forms a data set with ∼400 samples with three to five different background noise variations. The set is further subdivided for training (30%), validation (30%), and the rest for testing purposes. Thus, a total of at least 40 sample words were used for the identification of one speaker with multiple recording background noise variations. For recording the speech signal, a PC headset and a sound-recording software, Gold Wave, were used. The recorded speech sample has the following specifications:

• Duration = 2 s

• Sampling rate = 8000 samples per second

• Bit resolution = 16 bits per sample.

5.2 Preprocessing

The preprocessing of the speech signals consists of two operations – smoothing of the signal by median filtering and removal of the silent part using the threshold method. Although the speech signals were recorded in a noise-free environment, the presence of some unwanted spikes were observed. Therefore, a median filtering operation is performed on the raw speech signals, so that the vowel segmentation does not suffer from any type of an unwanted frequency component [19, 22, 24].

The smoothed signal S_smooth contains both speech and non-speech parts. The non-speech, or silent, part occurs in the speech signal before and after uttering the speech, and this time, information is considered to be redundant for the vowel segmentation purpose. The silent part ideally has a zero intensity. However, in practical cases, it is observed that even after smoothing, the silent part of the speech signal has an intensity ∼0.04. Our preprocessing algorithm considers this intensity value as the threshold, as in the algorithm shown in Table 4. Thus, a pure signal containing only the necessary speech part is obtained.

5.3 Vowel Segmentation Results

The vowel phoneme from any word spoken by the speaker to be identified are segmented and recognized by the SOM-based segmentation and the PNN- and F1-based vowel-recognition algorithm as described in Section 4. The experiments were repeated for several times, and the success rates were calculated. The vowel-recognition success rate for various vowels is summarized in Table 5.

As can be seen from Table 5, the success rate obtained from SOM-based segmentation is satisfactory. To check the superiority of the present segmentation technique, we also performed the segmentation using the conventional DWT-based technique. The input spoken word is decomposed at six levels, which cover the frequency band of a human voice. It was observed that if the decomposed part of the speech signals are reconstructed at various levels, a different part of the signal is obtained. We used Daubechies’ wavelet as the mother wavelet function and four decomposition and reconstruction orthogonal wavelet filters. For a smaller-order Daubechies wavelet, we obtained a smaller wavelet and better time resolution. However, the frequency response of low-order wavelets has many sidelobes. By increasing the order, we get a smoother version of the mother wavelet, which is better for analyzing the voiced signal. Therefore, according to Tang et al. [24], we choose a 10th-order wavelet. It was observed that the segmentation success rate for various vowels abruptly increased (from 89.7% to 97%) with the SOM-based segmentation technique. Table 6 summarizes this performance difference.

5.4 Speaker Identification Results

The segmented vowels were next applied for pattern matching with the trained LVQ code book, which discriminates between speakers. Although there were 10 subjects (five male and five female) in this study, eight Assamese vowels were each segmented from the five different words recorded under three to five different background noises as described in Section 5.2. The speaker identification success rate for two different predictor sizes was summarized in Table 7. The experimental results show that with a predictor size of 20, the success rate shows an ∼3% improvement in comparison to previous reported result [5]. Tables 8 and 9 show the correct speaker identification of segmented vowels from five different words containing the vowel in the case of Boy Speaker I and Girl Speaker I, respectively. The correct identification of the speakers directly depends on the segmentation process. If an error occurs in the segmentation part, then obviously, speaker identification will go wrong.

The disadvantage of the proposed speaker-identification method is that the computational time is large due to the repetition of training of the SOM six times. Because the success rate reached a satisfactory level, the increase in speed is tolerable. It is to be noted that if the decision is wrong, then a longer time is needed because the algorithm passes through each and every decision level. Table 10 shows the computational time for wrong and correct decisions. The success rate attained and the efficiency of recognition performed with varied recording backgrounds established the framework to be an effective mechanism for speaker identification in Assamese.