Vietnam Journal of Mathematics 33:4 (2005) 443–461  

      Central Limit Theorem for Functional of Jump Markov Processes Nguyen Van Huu, Vuong Quan Hoang, and Tran Minh Ngoc Department of Mathematics Hanoi National University, 334 Nguyen Trai Str., Hanoi, Vietnam Received February 8, 2005 Revised May 19, 2005 Abstract. In this paper some conditions are given to ensure that for a jump homogeneous Markov process {X(t), t ≥ 0} the law of the integral functional of the process: T−1/2 ∫ T 0 φ(X(t))dt, converges to the normal law N(0, σ 2) as T → ∞, where φ is a mapping from the state space E into R. 1. Introduction The central limit theorem is a subject investigated intensively by many wellknown probabilists such as Linderberg, Chung,.... The results concerning central limit theorems, the iterated logarithm law, the lower and upper bounds of the moderate deviations are well understood for independent random variable sequences and for martingales but less is known for dependent random variables such as Markov chains and Markov processes. The first result on central limit for functionals of stationary Markov chain with a finite state space can be found in the book of Chung [5]. A technical method for establishing the central limit is the regeneration method. The main idea of this method is to analyse the Markov process with arbitrary state space by dividing it into independent and identically distributed random blocks between visits to fixed state (or atom). This technique has been developed by Athreya Ney [2], Nummelin [10], Meyn Tweedie [9] and recently by Chen [4]. The technical method used in this paper is based on central limit for martingales and ergodic theorem. The paper is ogranized as follows: In Sec. 2, we shall prove that for a positive recurrent Markov sequence 444 Nguyen Van Huu, Vuong Quan Hoang, and Tran Minh Ngoc {Xn, n ≥ 0} with Borel state space (E,B) and for φ : E → R such that φ(x) = f(x) − Pf(x) = f(x) − ∫ E f(y)P (x, dy) with f : E → R such that ∫ E f2(x)Π(dx) < ∞, where P (x, .) is the transition probability and Π(.) is the stationary distribution of the process, the distribution of n−1/2 ∑n i=1 φ(Xi) converges to the normal law N(0, σ 2) with σ2 = ∫ E (φ2(x)+ 2φ(x)Pf(x))Π(dx). The central limit theorem for the integral functional T−1/2 ∫ T 0 φ(X(t))dt of jump Markov process {X(t), t ≥ 0} will be established and proved in Sec. 3. Some examples will be given in Sec. 4. It is necessary to emphasize that the conditions for normal asymptoticity of n−1/2 ∑n i=1 φ(Xi) is the same as in [8] but they are not equivalent to the ones established in [10, 11]. The results on the central limit for jump Markov processes obtained in this paper are quite new. 2. Central Limit for the Functional of Markov Sequence Let us consider a Markov sequence {Xn, n ≥ 0} defined on a basic probability space (Ω,F , P ) with the Borel state space (E,B), where B is the σ-algebra generated by the countable family of subsets of E. Suppose that {Xn, n ≥ 0} is homogeneous with transition probability P (x,A) = P (Xn+1 ∈ A|Xn = x), A ∈ B. We have the following definitions Definition 2.1. Markov process {Xn, n ≥ 0} is said to be irreducible if there exists a σfinite measure μ on (E,B) such that for all A ∈ B μ(A) > 0 implies ∞∑ n=1 Pn(x,A) > 0, ∀x ∈ E where Pn(x,A) = P (Xm+n ∈ A|Xm = x). The measure μ is called irreducible measure. By Proposition 2.4 of Nummelin [10], there exists a maximum irreducible measure μ∗ possessing the property that if μ is any irreducible measure then μ μ∗. Definition 2.2. Markov process {Xn, n ≥ 0} is said to be recurrent if ∞∑ n=1 Pn(x,A) = ∞, ∀x ∈ E, ∀A ∈ B : μ∗(A) > 0. The process is said to be Harris recurrent if Px(Xn ∈ A i.o.) = 1. Central Limit Theorem for Functional of Jump Markov Processes 445 Let us notice that a process which is Harris recurrent is also recurrent. Theorem 2.1. If {Xn, n ≥ 0} is recurrent then there exists a uniquely invariant measure Π(.) on (E,B) (up to constant multiples) in the sense Π(A) = ∫ E Π(dx)P (x,A), ∀A ∈ B, (1) or equivalently Π(.) = ΠP (.). (2) (see Theorem 10.4.4 of Meyn-Tweedie, [9]). Definition 2.3. A Markov sequence {Xn, n ≥ 0} is said to be positive recurrent (null recurrent) if the invariant measure Π is finite (infinite). For a positive recurrent Markov sequence {Xn, n ≥ 0}, its unique invariant probability measure is called stationary distribution and is denoted by Π. Hereafter we always denote the stationary distribution of Markov sequence {Xn, n ≥ 0} by Π and if ν is the initial distribution of Markov sequence then Pν(.), Eν(.) are denoted for probability and expectation operator responding to ν. In particular, Pν(.), Eν(.) are replaced by Px(.), Ex(.) if ν is the Dirac measure at x. We have the following ergodic theorem: Theorem 2.2. If Markov sequence {Xn, n ≥ 0} possesses the unique invariant distribution Π such that P (x, .)  Π(.), ∀x ∈ E, (3) then {Xn, n ≥ 0} is metrically transitive when initial distribution is the stationary distribution. Further, for any measurable mapping φ : E×E :→ R such that EΠ|φ(X0, X1)| <∞, with probability one lim n→∞n −1 n−1∑ k=0 φ(Xk, Xk+1) = EΠφ(X0, X1) (4) and the limit does not depend on the initial distribution. (See Theorem 1.1 from Patrick Billingsley [3]). The following notations will be used in this paper: For a measurable mapping φ : E → R we denote Πφ = ∫ E φ(x)Π(dx), Pφ(x) = ∫ E φ(y)P (x, dy) = E(φ(Xn+1)|Xn = x), Pnφ(x) = ∫ E φ(y)Pn(x, dy) = E(φ(Xn+m)|Xm = x). 446 Nguyen Van Huu, Vuong Quan Hoang, and Tran Minh Ngoc For the countable state space E = {1, 2, ...} we denote Pij = P (i, {j}) = P (Xn+1 = j|Xn = i), P (n)ij = Pn(i, {j}) = P (Xm+n = j|Xn = i) πj = Π({j}), P = [Pij , i, j ∈ E], P (n) = [P (n)ij , i, j ∈ E] = Pn. Then Πφ = ∑ j∈E φ(j)πj , Pφ(j) = ∑ k∈E φ(k)Pjk , Pnφ(j) = ∑ k∈E φ(k)P (n)jk . If the distribution of random variable Yn converges to the normal distribution N(μ, σ2) then we denote L−→ N(μ, σ2). The indicator function of a set A is denoted by 1A, where 1A(ω) = { 1, if ω ∈ A 0, else. Finally, the mapping φ : E = {1, 2, ...} −→ R is denoted by column vector φ = (φ(1), φ(2), ...)T . The main result of this section is to establish the conditions for n−1/2 n∑ k=1 φ(Xk) L−→ N(μ, σ2). We need a central limit theorem for martingale differences as follows Theorem 2.3. (Central limit theorem for martingale differences) Suppose that {uk, k ≥ 0} is a sequence of martingale differences defined on a probability space (Ω,F , P ) corresponding to a filter {Fk, k ≥ 0}, i.e., E(uk+1|Fk) = 0, k = 0, 1, 2, * * * Further, assume that the following conditions are satisfied (A1) n−1 n∑ k=1 E(u2k|Fk−1) P−→ σ2, (A2) n−1 n∑ k=1 E(u2k1[|uk|≥ε √ n]|Fk−1) P−→ 0, for each ε > 0 (the conditional Linderberg's condition). Then n−1/2 n∑ k=1 uk L−→ N(0, σ2). (5) (see Corollary of Theorem 3.2, [7]). Remark 1. Theorem 2.3 remains valid for {uk, k ≥ 0} being a m-dimensional martingale differences where the condition (A1) is replaced by n−1 n∑ k=1 Var (uk|Fk−1) P−→ σ2 = [σij , i, j = 1, 2, * * * ,m] Central Limit Theorem for Functional of Jump Markov Processes 447 with Var (uk|Fk−1) = [E(uikujk|Fk−1), i, j = 1, 2, * * * ,m]. We shall prove the following theorem. Theorem 2.4. (Central limit theorem for functional of Markov sequence) Suppose that the following conditions hold: (H1) The Markov sequence {Xn, n ≥ 0} is positive recurrent with the transition probability P (x, .) and the unique stationary distribution Π(.) satisfying the condition (3). (H2) The mapping φ : E → R can be represented in the form φ(x) = f(x) − Pf(x), x ∈ E, (6) where f : E → R is measurable and Πf2 <∞. Then n−1/2 n∑ k=1 φ(Xk) L−→ N(0, σ2) (7) for any initial distribution, where σ2 = Π(f2 − (Pf)2) = Π(φ2 + 2φPf). (8) Proof. We have n−1/2 n∑ k=1 φ(Xk) = n−1/2 n∑ k=1 [f(Xk) − Pf(Xk)] = n−1/2 n∑ k=1 [f(Xk) − Pf(Xk−1)] + n−1/2 n∑ k=1 Pf(Xk−1) − n−1/2 n∑ k=1 Pf(Xk) = n−1/2 n∑ k=1 uk + n−1/2[Pf(X0) − Pf(Xn)], where uk = f(Xk) − Pf(Xk−1) = f(Xk) − E(f(Xk)|Xk−1) are martingale differences with respect to Fk = σ(X0, X1, * * * , Xk), whereas n−1/2[Pf(X0) − Pf(Xn)] P−→ 0 by Chebyshev's inequality. Thus, it is sufficient to prove that Yn := n−1/2 n∑ k=1 uk L−→ N(0, σ2) and the convergence does not depend on the initial distribution. For this purpose, we shall show that the martingale differences {uk, k ≥ 1} satisfy the conditions (A1), (A2). According to assumption (H2) we have 448 Nguyen Van Huu, Vuong Quan Hoang, and Tran Minh Ngoc EΠ[E(u21|F0)] = EΠ(u21) = EΠ[f(X1) − Pf(X0)]2 = EΠf2(X1) − EΠ[Pf(X0)]2, thus EΠ(u21) = Πf 2 − Π(Pf)2 <∞. (9) Therefore, by the ergodic Theorem 2.2, for any initial distribution with probability one n−1 n∑ k=1 E(u2k|Fk−1) −→ EΠu21 = σ2. Thus the condition (A1) of Theorem 2.3 is satisfied. On the other hand, by (9) we have EΠ(u211[|u1|≥t]) −→ 0, (10) as t ↑ ∞. Again by the ergodic Theorem 2.2, for any initial distribution, with probability one n−1 n∑ k=1 E(u2k1[|uk|≥t]|Fk−1) −→ EΠ(u211[|u1|≥t]) (11) for each t > 0. By (11) and then (10) we have with probability one 0 ≤ lim n→∞n −1 n∑ k=1 EΠ(u2k1[|uk|≥ε √ n]) ≤ lim n→∞n −1 n∑ k=1 EΠ(u2k1[|uk|≥t]) = EΠ(u211[|u1|≥t]) −→ 0 as t ↑ ∞. Thus condition (A2) is satisfied, hence by the central limit theorem for martingale differences {uk, k ≥ 1} (7) holds.  Remark 2. If the series ∞∑ n=0 Pnφ(x) = ∞∑ n=0 ∫ E φ(y)Pn(x, dy) converges, then we always have φ(x) = f(x) − Pf(x) with f(x) = ∞∑ n=0 Pnφ(x). In fact, it is obvious that Central Limit Theorem for Functional of Jump Markov Processes 449 f(x) = φ(x) + ∞∑ n=1 Pnφ(x) = φ(x) + P ∞∑ n=0 Pnφ(x) = φ(x) + Pf(x). Furthermore, in this case σ2 = Π [ φ2 + 2 ∞∑ n=0 φPnφ ] . Remark 3. If φ = f − Pf holds, then Πφ = Πf − ΠPf = 0. (12) So the condition (12) is necessary for φ = f − Pf . Furthermore, in addition if we have lim n→∞P nf(x) = Πf, ∀x ∈ E then f(x) is also given by f(x) = ∞∑ n=0 Pnφ(x) + Πf. In fact, we have φ(x) = f(x) − Pf(x) Pφ(x) = Pf(x) − P 2f(x) * * * Pnφ(x) = Pnf(x) − Pn+1f(x). Summing the above equalities we obtain n∑ k=0 P kφ(x) = f(x) − Pn+1f(x) −→ f(x) − Πf. Remark 4. Function f given by (6) is defined uniquely up to an additional constant if limn→∞ Png(x) = Πg for all g Πintegrable. In fact, suppose that f1, f2 are the functions satisfying (6). Then g = f1−f2 is a solution of the equations: g(x) = Pg(x), g(x) = P (Pg(x)) = P 2g(x) = * * * = Png(x), ∀x ∈ E for all n = 1, 2, * * * . Thus there exists the limit g(x) = lim n→∞P ng(x) = Πg (a constant). It also follows from Remark 4 and from (8) that if f satisfies the equation (6) then σ2 is defined uniquely, i.e., σ2 does not change if f is replaced by f + C with C being any constant, since 450 Nguyen Van Huu, Vuong Quan Hoang, and Tran Minh Ngoc Π[φ2 + 2φP (f + C)] = Π[φ2 + 2φPf ] + 2CΠφ = Π[φ2 + 2φPf ]. Remark 5. If Πφ = 0 we can replace φ by φ∗ = φ− Πφ. Corollary 2.1. Assume that a Markov chain {Xn, n ≥ 0} is irreducible, ergodic with the countable state space E = {1, 2, * * * } and with the ergodic distribution Π = (π1, π2, * * * ) and the following condition is satisfied (H3) The mapping φ : E → R takes the form φ(x) = f(x) − Pf(x), ∀x ∈ E with f : E → R being measurable such that Πf2 <∞. Put σ2 = Π[f2 − (Pf)2] = Π[φ2 + 2φPf ]. Then n−1/2 n∑ k=1 φ(Xk) L−→ N(0, σ2) as n→ ∞. 3. Central Limit for Integral Functional of Jump Markov Process 3.1. Jump Markov Process Let {X(t), t ≥ 0} be a random process defined on some probability space (Ω,F , P ) with measurable state space (E,B). Definition 3.1. The process {X(t), t ≥ 0} is called jump homogeneous Markov process with the state space (E,B) if it is a Markov process with transition probability P (t, x, A) = P (X(t+ s) ∈ A|X(s) = x), s, t ≥ 0 satisfying the following condition lim t→0 P (t, x, {x}) = 1, ∀x ∈ E. (13) We suppose also that {X(t), t ≥ 0} is right continuous and the limit (13) is uniform in x ∈ E. By Theorem 2.4 in [6] the sample functions of {X(t), t ≥ 0} are step functions with probability one, and there exist two q− functions q(.) and q(., .) being Baire functions where q(x, .) is finite measure on Borel subsets of E \ {x}, q(x) = q(x,E \ {x}) is bounded. Further lim t→0 (1 − P (t, x, {x}) t = q(x), lim t→0 P (t, x, A) t = q(x,A) Central Limit Theorem for Functional of Jump Markov Processes 451 uniformly in A ⊂ E \ {x}. If q(x) > 0 ∀x ∈ E then the process has no absorbing state. We assume also that q(x) is bounded from 0. Since {X(t), t ≥ 0} is right continuous and step process, the system starts out in some state Z1, stays there a length of time ρ1, then jumps immediately to a new state Z2, stays a length of time ρ2, etc. Therefore there exist random variables Z1, Z2, * * * and ρ1, ρ2, * * * such that X(t) = Z1, if 0 ≤ t < ρ1, X(t) = Zn, if ρ1 + * * * + ρn−1 ≤ t < ρ1 + * * * + ρn, n ≥ 2. ρn's are all finite because we have assumed that q(x) > 0 ∀x ∈ E. Let ν(t) be the random variable defined by ν(t) = max{k : ρ1 + * * * + ρk < t} then ν(t) is the number of jumps which occur up to time t. It follows from the general theory of discontinuous Markov process (see [6], p.266) that {Zn, n ≥ 1} is a Markov chain with transition probability P (x,A) = q(x,A) q(x) , (14) furthermore P (ρn+1 > s|ρ1, * * * , ρn, Z1, * * * , Zn+1) = e−q(Zn+1)s, s > 0 (15) P (Zn+1 ∈ A|ρ1, * * * , ρn, Z1, * * * , Zn) = P (Zn, A). (16) The function q(., .) is called the transition intensity. It follows from (15), (16) that {(Zn, ρn), n ≥ 1} is a Markov chain on the cartesian product E×R+, where R+ = (0,∞). This chain is called the imbedded chain with the transition probability Q(x, s, A×B) = P (Zn+1 ∈ A, ρn+1 ∈ B|Zn = x, ρn = s) = ∫ A P (x, dy) ∫ B q(y)e−q(y)udu, A × B ∈ B × B(R+), where B(R+) denotes the Borel σalgebra on R+. This transition probability does not depend on s and we rewrite it by Q(x,A×B) or formally by Q(x, dy × du) = P (x, dy)q(y) exp(−q(y)u)du. Definition 3.2. The probability measure Π∗ on (E × R+,B × B(R+)) is called the stationary distribution of the imbedded chain {(Zn, ρn), n ≥ 1} if Π∗(A×B) = ∫ E×R+ Π∗(dx × ds)Q(x,A×B), A×B ∈ B × B(R+). (17) 452 Nguyen Van Huu, Vuong Quan Hoang, and Tran Minh Ngoc Letting B = R+, then Π∗ is the stationary distribution of the imbedded chain if and only if Π(.) = Π∗(.× R+) (18) is the one of {Zn, n ≥ 1} with the transition probability P (x,A) = Q(x,A×R+) and Π∗(A×B) = ∫ E Π(dx)Q(x,A ×B). Since ΠP (.) = Π(.), we have Π∗(A×B) = ∫ E Π(dx) ∫ A P (x, dy) ∫ B q(y) exp(−q(y)u)du = ∫ A ( ∫ E Π(dx)P (x, dy)) ∫ B q(y) exp(−q(y)u)du or Π∗(A×B) = ∫ A Π(dy) ∫ B q(y) exp(−q(y)u)du (19) or in differential form Π∗(dy × du) = Π(dy)q(y) exp(−q(y)u)du. (20) Thus we have the following proposition: Proposition 3.1. If the Markov chain {Zn, n ≥ 1} with the transition probability P (x,A) has the stationary distribution Π then the imbedded chain possesses also the stationary distribution Π∗ defined by (19) or (20). Proposition 3.2. If P (x, .)  Π(.) ∀x ∈ E, where Π is the stationary distribution of {Zn, n ≥ 1} then the transition probability Q(x, .) of the imbedded chain is also absolutely continuous with respect to the stationary distribution Π∗, i.e. Q(x, .)  Π∗(.), ∀x ∈ E. (see [3], p.66). Here and after we shall denote by Π,Π∗ the stationary distributions of Markov chain {Zn, n ≥ 1} and the imbedded chain {(Zn, ρn), n ≥ 1}, respectively. 3.2. Functional Central Limit Theorem We have the following ergodic theorem for the imbedded chain Theorem 3.1. (Ergodic theorem for the imbedded process) If Markov chain {Zn, n ≥ 1} with the transition probability P (x, .) having the stationary distribution Π such that Central Limit Theorem for Functional of Jump Markov Processes 453 P (x, .)  Π(.) ∀x ∈ E, and if φ(Z1, ρ1;Z2, ρ2) is the random variable possessing the finite expectation μ w.r.t. the probability measure PΠ∗ , then for any initial distribution lim n→∞n −1 n∑ k=1 φ(Zk, ρk;Zk+1, ρk+1) = μ ; a.s. (21) In particular, if Πq−1 <∞ then lim n→∞n −1 n∑ k=1 ρk = ∫ E Π(dy)(q(y))−1 = Πq−1 a.s. (22) Furthermore lim t→∞ ν(t) t = (Πq−1)−1 =: α > 0 a.s. (23) and (21), (22) remain valid if in the limits n is replaced by ν(t), then limits are taken as t→ ∞. Proof. (21) follows from the ergodic theorem for Markov chain {(Zn, ρn), n ≥ 1}, and (23) follows from (22) by the same argument as in the renewal theory.  Applying Theorem 2.4 for the imbedded chain {(Zn, ρn), n ≥ 1} we obtain the following theorem. Theorem 3.2. (Central limit theorem for the imbedded chain) Assume that the following conditions (C1), (C2) are satisfied: (C1) The jump Markov process {X(t), t ≥ 0} has the imbedded chain {(Zn, ρn), n ≥ 1} such that the Markov chain {Zn, n ≥ 1} has the transition probability P (x, .) with the stationary distribution Π satisfying the following condition P (x, .)  Π(.) ∀x ∈ E. (C2) The function ψ : E × R+ → R takes the form ψ(x, s) = f(x, s) −Qf(x, s), where f : E × R+ → R is B × B(R+)measurable and Qf(x) = Qf(x, s) = ∫ E P (x, dy) ∫ R+ f(y, u)q(y) exp(−q(y)u)du. Furthermore, the function f has the following property Π∗f2 = ∫ E Π(dy) ∫ R+ |f(y, u)|2q(y) exp(−q(y)u)du <∞. (24) Then we have n−1/2 n∑ k=1 ψ(Zk, ρk) L−→ N(0, σ2) (25) 454 Nguyen Van Huu, Vuong Quan Hoang, and Tran Minh Ngoc for any initial distribution, where σ2 = Π∗(f2 − (Qf)2) = Π∗(ψ2 + 2ψQf). (26) The goal of this section is to investigate the limit law of the integral functional T−1/2 ∫ T 0 φ(X(t))dt as T → ∞. Let us at first notice that ∫ T 0 φ(X(t))dt = ν(T )∑ k=1 φ(Zk)ρk + φ(Zν(T )+1)(T − τν(T )), (27) where τ1 = ρ1, τ2 = ρ1 + ρ2, * * * , τn = ρ1 + ρ2 + ...+ ρn, * * * are the jump times of the process {X(t), t ≥ 0}. In what follows we suppose always that the condition (C1) is satisfied. We need the following lemmas. Lemma 3.1. If Πφ2q−2 <∞ then 1√ T φ(Zν(T )+1)(T − τν(T )) P−→ 0 (28) for any initial distribution. Proof. Noticing that for ψ(x, s) = φ(x)s we have Π∗ψ2 = ∫ E Π(dy)φ2(y) ∫ R+ u2q(y) exp(−q(y)u)du = 2 ∫ E φ2(y)q−2(y)Π(dy) = 2Πφ2q−2 <∞ and ν(T ) → ∞ a.s. as T → ∞ by (23). By those and by the ergodic Theorem 3.1 (ν(T ) + 1)−1 ν(T )+1∑ k=1 |φ(Zk)ρk|2 −→ Π∗ψ2 a.s. as T → ∞. Hence with probability one (ν(T ) + 1)−1|φ(Zν(T )+1)ρν(T )+1|2 −→ 0 and (28) follows from 1√ T |φ(Zν(T )+1)(T−τν(T ))| ≤ (ν(T ) + 1 T )1/2(ν(T )+1)−1/2|φ(Zν(T )+1)ρν(T )+1| → 0 a.s.  Lemma 3.2. Suppose that {uk,Fk, k ≥ 1} defined on (Ω,F , P ) are the square integrable martingale differences such that Central Limit Theorem for Functional of Jump Markov Processes 455 sup n,m≥1 (n−1 m+n∑ k=m Eu2k) = C <∞ (29) and that {ν(t), t ≥ 0} is a random process valued in {1, 2, * * * } such that {ν(t) = k} ∈ Fk ∀t ≥ 0 and lim t→∞ ν(t) t = α > 0 a.s. (30) Then T−1/2 ∣∣∣ ν(T )∑ k=1 uk − [αT ]∑ k=1 uk ∣∣∣ P−→ 0 as T → ∞. (31) Proof. It follows from condition (30) that: for all ε > 0, and T sufficiently large we have P (|ν(T ) T − α| > ε3) < ε or P ((α− ε3)T < ν(T ) < (α+ ε3)T ) ≥ 1 − ε. (32) Putting Aε = {α− ε3)T < ν(T ) < (α+ ε3)T }, we have P ( T− 1 2 ∣∣∣ ν(T )∑ k=1 uk − [αT ]∑ k=1 uk ∣∣∣ > ε) ≤ P (Acε) + P ({ T− 1 2 ∣∣∣ ν(T )∑ k=1 uk − [αT ]∑ k=1 uk ∣∣∣ > ε} ∩Aε ) ≤ ε+ P ( T− 1 2 max |l−[αT ]|<ε3T ∣∣∣ l∑ k=1 uk − [αT ]∑ k=1 uk ∣∣∣ > ε) ≤ ε+ P ( max a≤l≤b ∣∣∣ l∑ k=a uk ∣∣∣ > εT 1 2 2 ) (33) where a = [αT ] − [ε3T ], b = [αT ] + [ε3T ] with [r] denoting the integer part of the number r. By Kolmogorov's inequality for a martingale P ( max 1≤n≤N ∣∣∣ n∑ k=1 uk ∣∣∣ > λ) ≤ 1 λ2 E [ N∑ k=1 uk ]2 = 1 λ2 N∑ k=1 Eu2k, we have P ( max a≤l≤b ∣∣∣ l∑ k=a uk ∣∣∣ > εT 1 2 2 ) ≤ 8ε 1 2ε3T E [ 2[Tε3]+a∑ k=a u2k ] ≤ 8εC. (34) It follows from (33), (34) that (31) holds.  456 Nguyen Van Huu, Vuong Quan Hoang, and Tran Minh Ngoc Corollary 3.1. Assume that the martingale differences {uk, k ≥ 1} take the form uk = f(Xk) − E(f(Xk)|Xk−1), k = 1, 2, . . . where {Xk, k ≥ 0} is a Markov chain with the stationary distribution Π such that Πf2 <∞. Then (31) holds for any initial distribution. Proof. It is obvious that EΠu 2 k ≤ EΠf2(Xk−1) = Πf2 <∞, therefore EΠ(n−1 m+n∑ k=m u2k) ≤ Πf2 = C, ∀m,n. Denoting the quantity in the left-hand side of (31) by ηT , by Lemma 3.2 we obtain lim T→∞ PΠ(|ηT | ≥ ε) = 0 ∀ε > 0 or lim T→∞ ∫ E Px(|ηT | ≥ ε)Π(dx) = 0 ∀ε > 0. It follows that there exists a subset Λ ⊂ E such that Π(Λ) = 0 and lim T→∞ Px(|ηT | ≥ ε) = 0 ∀x ∈ E \ Λ. Since P (x, .)  Π(.) ∀x, P (x,E \ Λ) = 1 ∀x ∈ E. On the other hand, letting AT = {|ηT | ≥ ε}, we observe that AT ∈ ∪n≥n0Fn with n0 > 1, where Fn = σ(Xk, k ≥ n). Then by Markov property: Px(AT ) = E(1AT |X0 = x) = E[E(1AT |X1)|X0 = x] = ∫ E E(1AT |X1 = y)P (x, dy) = ∫ E\Λ Ey(1AT )P (x, dy). Therefore 0 ≤ lim sup T→∞ Px(AT ) = lim sup T→∞ ∫ E\Λ Py(AT )P (x, dy) = ∫ E\Λ lim T→∞ Py(AT )P (x, dy) = 0. So lim T→∞ Px(AT ) = 0 ∀x and hence lim T→∞ Pν(|ηT | ≥ ε) = lim T→∞ ∫ E Px(|ηT | ≥ ε)ν(dx) = 0. Central Limit Theorem for Functional of Jump Markov Processes 457 This implies (31).  Lemma 3.3. Assume that the following equation has a solution g(x) (I − P )g(x) = Pφq−1(x). (35) Then, putting f(x, s) = φ(x)s + g(x), (36) we have the representation φ(x)s = f(x, s) −Qf(x), (37) where Qf(x) = g(x). Proof. At first let us notice that for ψ : E × R+ → R given by ψ(x, s) = φ(x)s we have Qg(x) = ∫ E g(y)P (x, dy) ∫ R+ q(y) exp(−q(y)u)du = ∫ E g(y)P (x, dy) = Pg(x), Qψ(x) = ∫ E φ(y)P (x, dy) ∫ R+ uq(y) exp(−q(y)u)du = ∫ E φ(y)q−1(y)P (x, dy) = Pφq−1(x). In order to prove (37) we shall prove that if g(x) is a solution of (35) then g(x) = Qf(x). In fact, by (36) Qf(x) = Qψ(x) +Qg(x) = Pφq−1(x) + Pg(x) = g(x).  Remark 6. A necessary condition for the existence of a solution of (35) is Πφq−1 = 0. (38) In fact, applying operator Π on both sides of (35) we have Πg − ΠPg = 0 = Πφq−1. Let us notice that the condition (38) is satisfied if the function φ is represented in the form φ(x) = φ∗(x) − αΠφ∗q−1 where φ∗ : E → R, α is given by (23). Lemma 3.4. Assume that the following equation has a solution g (I − P )g(x) = Pφq−1(x) and that Πφ2q−2 <∞,Πg2 <∞. Furthermore, if the condition (C1) of Theorem 3.2 are satisfied then 458 Nguyen Van Huu, Vuong Quan Hoang, and Tran Minh Ngoc T−1/2 ν(T )∑ k=1 φ(Zk)ρk L−→ N(0, αδ2) (39) for any initial distribution, where α is given by (23) and δ2 = 2Π(φ2q−2 + φq−1g). Proof. By Lemma 3.3, we have the representation ψ(Zk, ρk) = φ(Zk)ρk = f(Zk, ρk) −Qf(Zk) = f(Zk, ρk) −Qf(Zk−1) +Qf(Zk−1) −Qf(Zk) = uk + g(Zk−1) − g(Zk) where {uk = f(Zk, ρk)−Qf(Zk−1), k ≥ 1} are martingale differences. Therefore T−1/2 ν(T )∑ k=1 φ(Zk)ρk = T−1/2 ν(T )∑ k=1 uk + T−1/2 ν(T )∑ k=1 (g(Zk−1) − g(Zk)) = T−1/2 ν(T )∑ k=1 uk + T−1/2(g(Z0) − g(Zν(T ))). (40) Since Πg2 < ∞, by the same argument as in the proof of Lemma 3.1, we can show that T−1/2(g(Z0) − g(Zν(T ))) P−→ 0 (41) for any initial distribution. Furthermore, we have by (36) Π∗f2 ≤ 2Π∗(ψ2 + g2) = 2(Πφ2q−2 + Πg2) <∞, hence by Corollary 3.1, (31) holds for any initial distibution. Applying Theorem 3.2 for the imbedded chain {(Zk, ρk), k ≥ 1} we obtain T−1/2 [αT ]∑ k=1 uk L−→ N(0, αδ2) (42) with δ2 = Π∗(f2 − (Qf)2) = Π∗(f2 − g2) = Π∗(ψ2 + 2ψg) = 2Π(φ2q−2 + φgq−1). Finally, it follows from (40), (31), (41). (34), (42) that (39) holds for any initial distribution.  Now we state and prove the main theorem as follows Central Limit Theorem for Functional of Jump Markov Processes 459 Theorem 3.3. Assume that the condition (C1) of Theorem 3.2 and the following condition (C3) are satisfied (C3) (i) Πφ2q−2 <∞ and, (ii) The following equation has a solution g (I − P )g(x) = Pφq−1(x) with Πg2 <∞. Then T−1/2 ∫ T 0 φ(X(t))dt L−→ N(0, αδ2) for any initial distribution, where δ2 = 2Π(φ2q−2 + φgq−1). Proof. The conclusion of Theorem 3.3 follows from Theorem 3.2 and Lemmas 3.1, 3.4.  4. Examples Example 1. Assume that the jump Markov process {X(t), t ≥ 0} with the state space E = {1, 2, 3} has the transition intensity matrix Q = ⎡ ⎣−1 0.5 0.50.4 −1 0.6 0.8 0.2 −1 ⎤ ⎦ . Then the Markov chain {Zk, k ≥ 1} has the transition probability matrix P = ⎡ ⎣ 0 0.5 0.50.4 0 0.6 0.8 0.2 0 ⎤ ⎦ . It is easy to see that {Zk, k ≥ 1} possesses the ergodic distribution as follows Π = [ 0.38596 0.26316 0.35088 ] , whereas the sequence {ρk, k ≥ 1} is the sequence of independent, exponentially indentically distributed random variables with the parameter q = 1 (i.e., q(x) = 1 for all x ∈ E) and hence α = 1. Let us consider φ∗ = [1, 2, 4]T , i.e. φ∗(1) = 1, φ∗(2) = 2, φ∗(3) = 4. Then Πφ∗ = 2.3158, φ = φ∗ − Πφ∗ = [−1.3158 −0.3158 1.6842 ]T . We shall prove that as T → ∞ 1√ T ∫ T 0 (φ∗(X(t)) − 2.3158)dt L−→ N(0, σ2). (43) For this purpose, we try to find a function g = [g1, g2, g3]T satisfying the following equation 460 Nguyen Van Huu, Vuong Quan Hoang, and Tran Minh Ngoc (I − P )g = Pφq−1 = Pφ or in detail ⎡ ⎣ 1 −0.5 −0.5−0.4 1 −0.6 −0.8 −0.2 1 ⎤ ⎦ ⎡ ⎣ g1g2 g3 ⎤ ⎦ = ⎡ ⎣ 0.68420.4842 −1.1158 ⎤ ⎦ . The above algebraic equation has the following solution g = [ 1.15788 0.94735 0 ] . Since E has a finite number of elements, Πg2 and Πφ2 are finite. By Theorem 3.3 we have (43) with σ2 = δ2 = 2Π(φ2 + φg) = 2.046. Example 2. Let us consider the integral functional of the jump Markov process with the state space E = {1, 2, * * * } defined by: Ti = ∫ T 0 1{X(t)=i}, i ∈ E. This integral is the total time length during which the process visits the state i. Assume that this process satisfies the condition (C1). For each state i, put φ∗(x) = 1{x=i} then αΠφ∗q−1 = απiq −1 i . Let us consider φ(x) = φ∗(x) − απiq−1i . Suppose that the equation (I − P )g(x) = Pφq−1(x) (44) has a solution g such that Πg2 <∞. Then by Theorem 3.3 1√ T ∫ T 0 φ(X(t))dt = 1√ T ∫ T 0 (1{X(t)=i} − απiq−1i )dt L−→ N(0, αδ2) where δ2 = 2Π(φ2q−2 + φq−1g). In particular, for the case where E = {1, 2}, Q = [−q1 q1 q2 −q2 ] , P = [ 0 1 1 0 ] , q1, q2 > 0, we have the stationary distribution Π = (1/2, 1/2) and α = (Πq−1)−1 = (1 2 ( 1 q1 + 1 q2 ))−1 = 2q1q2 q1 + q2 . Put φ∗(x) = 1{x=1}, then αΠφ∗q−1 = απ1q−11 = q2 q1 + q2 , φ(x) = 1{x=1} − q2 q1 + q2 , and 1√ T ∫ T 0 ( 1{X(t)=1} − q2 q1 + q2 ) dt L−→ N(0, αδ2) (45) Central Limit Theorem for Functional of Jump Markov Processes 461 for any initial distribution. In order to find δ2 we have to solve the equation (44) for i = 1, i.e. [ 1 −1 −1 1 ] ; [ g1 g2 ] = [ Pφq−1(1) Pφq−1(2) ] (46) with notice that [ Pφq−1(1) Pφq−1(2) ] = [ 0 1 1 0 ] [ (1 − q2/(q1 + q2))q−11 −(q2/(q1 + q2))q−12 ] = [−1/(q1 + q2) 1/(q1 + q2) ] . (46) has a solution g1 = −1/(q1+q2), g2 = 0. Hence, by Theorem 3.3, we obtain (45) with δ2 = 2Π(φ2q−2 + φq−1g) = 1 (q1 + q2)2 . We obtain from (45) √ T (T1 T − q2 (q1 + q2) ) L−→ N(0, 2q1q2 (q1 + q2)3 ) . Acknowledgement. The authors would like to thank Prof. Dr. Nguyen Huu Du and Prof. Dr. Tran Hung Thao for useful discussions. References 1. A. de Acosta, Moderate deviations for vector valued functional of a Markov chain: Lower bounds, Ann. Probab. 25 (1997) 259–294. 2. Athreya and P. Ney, A new approach to the limit theory of recurrent Markov chain, Trans. Amer. Math. Society 245 (1978) 493–501. 3. P. Billingsley, Statistical Inference for Markov Processes, The University of Chicago Press, 1958. 4. X. Chen, Limit theorem for functionals of Ergodic Markov Chains with general state space, Memoir of the American Mathematical Society 139 (1999) 1–200. 5. K. L. Chung, Markov Chains with Stationary Transition Probabilities, Springer– Verlag, Berlin, 2nd Edition, 1967. 6. J. L. Doob, Stochastic Process, John Wiley & Sons, New York, 1953. 7. P. Hall and C.C. Heyde, Martingale Limit Theory and Its Application, Academic Press, 1980. 8. S. Niem and E. Nummelin, Central Limit Theorems for Markov Random Walks, Commentations Physico-Mathematica, 54, Societas Scientiarum Fennica, Helsinki, 1982. 9. S. P. Meyn and R.L. Tweedie, Markov Chains and Stochastic Stability, Springer– Verlag, London, 1993. 10. E. Nummelin, General Irreducible Markov Chains and Non-negative Operators, Cambridge University Press, 1984. 11. M. S. Ross, Introduction to Probability Models, 7th Edition, Harcourt Academic Press, 2000. View publication stats