IMPORTANT DEFINITIONS, THEOREMS AND EXAMPLES,

AND DIFFICULT PROOFS, IN WEAK CONVERGENCE

  After `Billingsley (1999):  Convergence of Probability Measures'

Graduate Course in Weak Convergence, Spring 2002

• \underline{(S, d), S′, S′′} metric spaces with Borel sets \underline{𝔖, 𝔖′, 𝔖′′}; \underline{S_{r}(x)} ≡ \{y ∈S : d(x, y)<r\}, \underline{S_{r}[x]} ≡ \{y ∈S: d(x, y)≤r\} and \underline{A_{r}} ≡ \{y ∈S: d(y, A)≤r\} for r >0, x∈S, A⊆S;

• \underline{ℂ_{B}(S)} ≡ \{bounded and continuous f:S →ℝ\}; \underline{∥f∥_{ℂ_{B}(S)}} ≡ sup_{x∈S} |f(x)|;

• \underline{BL(S)} ≡ \{bounded and Lipschitz uniformly continuous f: S → ℝ\}; \underline{∥f∥_{BL(S)}} ≡ ∥f∥_{ℂ_{B}(S)} + sup{|f(x)−f(y)|/d(x, y) : S ∋x≠y ∈S} (cf. MT 20);

• \underline{P f} ≡ ∫_{S} f dP for P ∈ \underline{P(S)} ≡ \{P probability measure on S\} and f ∈ℂ_{B}(S);

• \underline{(T , ϱ)} compact metric; \underline{C(T )} ≡ \{continuous f:T →ℝ\}; \underline{∥f∥_{C(T )}}≡ sup_{x∈T} |f(x)|;

• C(T )∋x→\underline{π_{t_{1}...t_{k}}}x≡(x(t_{1}), …, x(t_{k}))∈ℝ^{k}; ℝ^{∞}∋x→\underline{π_{i_{1}...i_{k}}}x≡(x_{i_{1}}, …, x_{i_{k}})∈ℝ^{k};

• \underline{D} ≡ \{càdlàg f:[0, 1]→E\} (continu á droite limites á gauche);

• \underline{B} separable real Banach space, \underline{dual B^{⋆}} and \underline{bidual B^{⋆⋆}} (cf. MT 19, 34).

1. Measures on Metric Spaces. Theorem 1.1. P ∈ P(S) is \underline{regular}, i.e., P (A) = inf_{open G⊇A} P (G) = sup_{closed} F ⊆A P (F) for A∈𝔖. (Cf. MT 14.) Theorem 1.2. P, Q∈P(S) agree if P f =Qf for f ∈BL(S). Theorem 1.3. For S separable and complete P ∈P(S) is \underline{tight}, i.e., sup_{compact} K⊆S P (K) = 1. Definition. \underline{π-system} Π⊆𝔖 closed under finite intersection. D⊆𝔖 \underline{Dynkin system} if S ∈D, B\A∈D if D ∋A⊆B ∈D, and ⋃_{i=1}^{∞} A_{i} ∈D if {A_{i}}_{i=1}^{∞} ⊆D increasing. Lemma. (Dynkin) If π-system Π⊆D Dynkin system, then σ(Π)⊆D. Definition. A_{P} ⊆𝔖 \underline{separating for P} ∈P(S) if Q∈P(S) agrees with P if Q(·) =P (·) on A_{P}. A⊆𝔖 \underline{separating} if P, Q∈P(S) agree if P (·)=Q(·) on A. Proposition. π-system Π with σ(Π)=𝔖 is separating. Example 1.1. P ∈ P(ℝ^{k}) is tight, with {(−∞, y] : y ∈ ℝ^{k}}, {(−∞, y) : y ∈ ℝ^{k}}, {(y, ∞) : y ∈ℝ^{k}} and {[y, ∞) : y ∈ℝ^{k}} separating. Example 1.2. (\underline{ℝ^{∞}}, ∑_{i=1}^{∞} \frac{1∧|x_{i}−y_{i}|}{2^{i}}) is separable and complete, P ∈P(ℝ^{∞}) tight, and \underline{finite dimensional sets ℝ_{f}^{∞}} ≡ \{⋂_{i=1}^{k}π_{n_{i}}^{−1}(H_{i}), H_{i} open, n_{i},k ∈ℕ\} separating.

Example 1.3. \underline{C} ≡ C[0, 1] is separable and complete, P ∈ P(C) tight, and \underline{finite} \underline{dimensional sets C_{f}} ≡ \{π_{t_{i}}^{−1}(H_{i}) : H_{i} open, t_{i}∈[0, 1], k ∈ℕ\} separating, cf. MT 1.

2. Properties of Weak Convergence. Definition. P(S) ∋ \underline{P_{n} ⇒P} ∈ P(S) if P_{n}f →P f for f ∈ℂ_{B}(S). Definition. Give P(S) weak \underline{topology} generated by P(S)∋P →P f ∈ℝ, f ∈ℂ_{B}(S). Proposition. P_{n} →P in P(S) iff. P_{n} ⇒P in P(S)! (Cf. MT 8.) Example 2.1. δ_{x_{n}}⇒δ_{x} in P(S) iff. x_{n} →_{S} x. Definition. A∈𝔖 \underline{P -continuity set} of P ∈P(S) if P (∂A)=0. Theorem 2.1. (Portmanteau) (Cf. MT 10.) Equivalent are

(i) P_{n} ⇒P in P(S); (ii) P_{n}f →P f for f ∈BL(S); (iii) lim sup_{n} P_{n}F ≤P F for closed F ⊆S; (iv) lim inf_{n} P_{n}G≥P G for open G⊆S;

(v) lim_{n} P_{n}A=P A for P -continuity A∈𝔖. Theorem 2.2. For P ∈P(S), if A_{P} π-system with opens in S countable unions of sets in A_{P}, then P_{n} ⇒P in P(S) if P_{n}(A)→P (A) for A∈A_{P}. Theorem 2.3. For S separable, P_{n} ⇒ P in P(S) if P_{n}(A) → P (A) for A in π-system A_{P} such that {A∈A_{P} : x∈Int(A), A⊆S_{ε}(x)} ≠ ∅ for x∈S and ε>0. Definition. A⊆𝔖 \underline{convergence determining} if, for any P ∈P(S), P_{n}(A)→P (A) for P -continuity A∈A implies P_{n} ⇒P in P(S). (Cf. MT 13.) Theorem 2.4. If S separable, π-system A is convergence determining if {P -conti-nuity A∈A : x∈Int(A), A⊆S_{ε}(x)} ≠ ∅ for P ∈P(S), x∈S, ε>0. (Cf. MT 15.) Example. Finite intersections of open balls convergence determining if S separable. Example 2.4. ℝ_{f}^{∞} is convergence determining for P(ℝ^{∞}). Example 2.5. C_{f} is \underline{not} convergence determining for P(C). Theorem 2.6. P_{n} ⇒P in P(S) iff. subsequences have subsequences ⇒P in P(S). Proposition. \underline{D_{h}} ≡ \{x∈S : h not continuous at x\} ∈ 𝔖 for h:S →S′. Theorem 2.7. (Mapping Theorem) For h:S →S′ measurable with P (D_{h})=0, P_{n}h^{−1} ⇒P h^{−1} in P(S′) if P_{n} ⇒P in P(S). Example 2.6. P_{n} ⇒P in P(ℝ^{∞}) iff. P_{n}π_{k}^{−1} ⇒P π_{k}^{−1} in P(ℝ^{k}) for k ∈ℕ.

Example 2.7. P_{n} ⇒P in P(C) \overrightarrow{⇍} P_{n}π_{t_{1},...,t_{k}}^{−1} ⇒P π_{t_{1},...,t_{k}}^{−1} in P(ℝ^{k}) for k ∈ℕ. Theorem 2.8 (MT 12). For S′, S′′ separable, P_{n} ⇒P in P(S′×S′′) iff. P_{n}(A′× A′′)→P (A′×A′′) for P (·×S′′)-continuity A′ ∈𝔖′ and P (S′×·)-continuity A′′ ∈𝔖′′. Corollary. For S′, S′′ separable, P′_{n}×P′′_{n} ⇒ P′×P′′ in P(S′×S′′) iff. P′_{n} ⇒ P′ in P(S′) and P′′_{n} ⇒P′′ in P(S′′).

3. Convergence in Distribution. Definition. \underline{X_{n} ⇒X} for S-valued r.v.’s if PX_{n}^{−1} ⇒PX^{−1} in P(S). Definition. A∈𝔖 \underline{X-continuity set} of S-valued r.v. X if P\{X ∈∂A\} = 0. Proposition. (Portmanteau) For S-valued r.v. X_{n}, X, equivalent are

(i) X_{n} ⇒X in S; (ii) E{f(X_{n})} → E{f(X)} for f ∈BL(S); (iii) lim sup_{n} P{X_{n} ∈F } ≤ P{X ∈F } for closed F ⊆S; (iv) lim inf_{n} P{X_{n} ∈G} ≥ P{X ∈G} for open G⊆S;

(v) lim_{n} P{X_{n} ∈A} = P{X ∈A} for X-continuity sets A. Definition. \underline{X_{n} →_{P} X} in S if P\{d(X, X_{n})>ε\} → 0 for ε>0. Proposition. X_{n} →_{P} X in S iff. d(X, X_{n}) ⇒ 0 in P(ℝ). X_{n} →_{P} x in S iff. X_{n} ⇒x in P(S) for (non-random) x∈S. (Cf. MT 9.) Theorem 3.1. Y_{n}⇒X in P(S) if X_{n}⇒X in P(S) and d(X_{n},Y_{n})⇒0 in P(ℝ). Definition. P ∈ P(S) has \underline{density} f : S → ℝ wrt. measure μ on S if P (A) = ∫_{A} f dμ for A∈𝔖. Theorem. (Scheffé) If f, f_{n}: S → ℝ are densities wrt. measure μ on S with f_{n} →f a.e. (μ), then ∫_{S} |f_{n}−f| dμ → 0. (Cf. MT 3-5.) Corollary. If P_{n}, P ∈P(S) have densities f, f_{n} wrt. measure μ on (S, 𝔖), and f_{n} →f a.e. (μ), then P_{n} ⇒P in P(S). Theorem 3.4. If X_{n} ⇒X in P(ℝ), then E{|X|} ≤ lim inf_{n} E{|X_{n}|}. Definition. R.v.’s X_{n} \underline{uniformly integrable} if lim_{x↑∞} sup_{n} ∫_{|X_{n}|>x}|X_{n}|dP = 0. Theorems 3.5-3.6 (MT 2). For {X_{n}}_{n=1}^{∞} ⊆𝕃^{1}(Ω) with X_{n} ⇒X in P(ℝ), E{| X_{n}|}→E{|X|}<∞ iff. {X_{n}}_{n=1}^{∞} are uniformly integrable. Then E{X_{n}}→E{X}.

5. Prohorov’s Theorem. Definition. Π ⊆ P(S) \underline{relatively compact} if each sequence in Π has subsequence that converges in P(S). (Cf. MT 27.) Proposition (MT 28). P_{n} ⇒ P in P(S) if {P_{n}}_{n=1}^{∞} is relatively compact and P_{n}(A) → P (A) for A ∈ A, where A ⊆ 𝔖 contains a separating class of continuity sets for each Q∈P(S). Example 5.1. Let {P_{n}}_{n=1}^{∞} ⊆P(C) be relatively compact. Then P_{n} ⇒P in P(C) if P_{n}π_{t_{1},...,t_{k}}^{−1}(B)→P π_{t_{1},...,t_{k}}^{−1} (B) for B a finite intersection of P π_{t_{1},...,t_{k}}^{−1}-continuity balls in ℝ^{k}. Hence P_{n} ⇒P if P_{n}π_{t_{1},...,t_{k}}^{−1} ⇒P π_{t_{1},...,t_{k}}^{−1} in P(ℝ^{k}) for k ∈ℕ. Definition. Π⊆P(S) \underline{tight} if sup\{inf\{P (K) : P ∈Π\} : S ⊇K compact\} = 1. Theorem 5.1-5.2. (Prohorov) Π⊆P(S) tight ⇒ Π relatively compact. For S separable and complete, Π⊆P(S) relatively compact ⇒ Π tight. Definition. \underline{M(T )} ≡ \{signed finite Borel measures on T \} with \underline{w^{⋆}-topology} [i.e., \{ν ∈M(T ) : |νf−μf|<ε\}, f ∈C(T ), ε>0, subbasic open], \underline{M^{+}(T )} ≡ \{μ∈M(T ) : μ non-negative\} and \underline{M_{1}^{+}(T )} = \{μ∈M^{+}(T ) : μ(T )≤1\}. Lemma. M_{1}^{+}(T ) is sequentially compact. Math Proof. By Riesz Representation Theorem, C(T )^{⋆} = M(T ) (MT 18) with ∥μ∥_{M(T )} = (μ^{+}+μ^{−})(T ). By Alaoglus’s Theorem, unit ball (0)_{1}^{⋆} in M(T ) is w^{⋆}- compact, and so is w^{⋆}-closed subset M_{1}^{+}(T ). w^{⋆}-topology on (0)_{1}^{⋆} is metrizable iff. B =C(T ) is separable (MT 22), which it is (MT 24)! Since M_{1}^{+}(T ) metrizable and compact, it is sequentially compact. □ Direct Proof. Take {μ_{i}} ⊆ M_{1}^{+}(T ) and dense F = {f_{i}}_{i=1}^{∞} in C(T ) (MT 24). Since {μ_{n}f_{1}}_{n=1}^{∞} is bounded, there is convergent subsequence {μ_{n}^{(1)}f_{1}}_{n=1}^{∞}. Since {μ_{n}^{(1)}f_{2}}_{n=1}^{∞} is bounded, there is convergent subsequence {μ_{n}^{(2)}f_{2}}_{n=1}^{∞}. Continue like that ... . Diagonalize so Lf_{i} ≡ lim_{n} μ_{n}^{(n)}f_{i} = lim_{n} μ_{n}^{(i)}f_{i} exists for i∈ℕ.

Extend L from F to C(T ): For g ∈ C(T ), take F ∋ g_{i} →_{C(T )} g. Then {Lg_{i}}_{i=1}^{∞} is Cauchy, since |Lg_{i}−Lg_{j}| ≤ lim_{n} μ_{n}^{(i∨j)}|g_{i}−g_{j}| ≤ ∥g_{i}−g_{j}∥_{C(T )}, so Lg ≡lim_{i} Lg_{i} exists, and is independent of particular sequence F ∋g_{i} →_{C(T )} g, since |Lĝ_{i}−Lg| = lim_{j} |L(ĝ_{i}−g_{j})| ≤ ∥ĝ_{i}−g∥_{C(T )} + lim sup_{j} ∥g−ĝ_{i}∥_{C(T )} = ∥ĝ_{i}−g∥_{C(T )} for any sequence F ∋ĝ_{i} →_{C(T )} g. In same fashion, L is linear, continuous, and (trivially) positive [i.e., Lg ≥ 0 for non-negative g ∈ C(T )], so Riesz Representation Theorem gives Lg =μg, some μ∈M^{+}(T ), which is in M_{1}^{+}(T ) (take g =1).

To show μ_{n}^{(n)} →_{w^{⋆}} μ, pick g ∈C(T ) and F ∋g_{i} →_{C(T )} g. Since (μ_{n}^{(n)}−μ)g_{i} →0, lim_{n}|μ_{n}^{(n)}g−μg| ≤ lim sup_{n}(|(μ_{n}^{(n)}−μ)(g−g_{i})| + |(μ_{n}^{(n)}−μ)g_{i}|) ≤ 2∥g−g_{i}∥_{C(T )}. □

Proof of ⇒ in Theorem. Take {P_{i}}_{i=1}^{∞} ⊆ Π, and compacts K_{1} ⊆ K_{2} ⊆ … ⊆ S with sup\{P_{i}(K_{n}^{c}) : i ∈ ℕ\} ≤ \frac{1}{n}. By Lemma, restriction \{P_{i}|K_{1}\}_{i=1}^{∞} has w^{⋆}- convegent subsequence {P_{i}^{(1)}|K_{1}}_{i=1}^{∞}. By Lemma, {P_{i}^{(1)}|K_{2}}_{i=1}^{∞} has w^{⋆}-convegent subsequence {P_{i}^{(2)}|K_{2}}_{i=1}^{∞}. Continue like that ... .

Since P^{(k)}(·∩K_{k}) ≡ lim_{i} P_{i}^{(k)}(·∩K_{k}) increases with k, P ≡ lim_{k} P^{(k)}(·∩K_{k}) exists. Here P is finitely additive, by additivity of P^{(k)}(·∩K_{k}). Further, for ε>0,

sup_{compact K⊆A} P (K) ≥ sup_{compact K⊆A} P_{i}^{(k)}(K∩K_{k}) − ε

= P_{i}^{(k)}(A∩K_{k}) − ε → P (A) − ε as i→∞ and k →∞.

So P (A) = sup_{compact K⊆A} P (K), i.e., \underline{tightness}. Tight finitely additive probability setfunctions are additive (MT 17). And P_{i}^{(i)} ⇒P in P(S), since for closed F ⊆S

P (F) ≥ lim_{i} P_{i}^{(n)}(F ∩K_{n}) = lim_{i} P_{i}^{(i)}(F ∩K_{n}) ≥ lim sup_{i} P_{i}^{(i)}(F) − \frac{1}{n}. □

Corollary. If {P_{n}}_{n=1}^{∞} ⊆P(S) is tight, and each convergent subsequence converges to P in P(S), then P_{n} ⇒P in P(S).

B-valued Random Variables. Definition. \underline{Mean E} ∈B^{⋆⋆} of B-valued r.v. X is B^{⋆} ∋ η → E(η) ≡ E\{η(X)\} ∈ ℝ. If mean E ∈B^{⋆⋆} lies in B (cf. MT 22), it is \underline{Pettis-mean} of X. Definition. B-valued r.v. X has \underline{Bochner mean} E\{X\} ≡ lim_{n} ∫_{Ω} X_{n} dP ∈ B if {∑_{i=1}^{k} x_{i}1_{A_{i}} : x_{i} ∈B, A_{i} events in Ω, k ∈ℕ} ∋ X_{n} →_{a.e.} X with E{∥X−X_{n}∥}→0. Proposition (MT 34). B-valued r.v. X has Bochner mean iff. E{∥X∥} < ∞. If so X has same Pettis mean, and ∥E{X}∥≤E{∥X∥}. Definition. \underline{Covariance Q}: B^{⋆} → B^{⋆⋆} of B-valued r.v. X is B^{⋆} ∋ η → Q(η) ≡ Cov{η(X), ·(X)} ∈ ℝ. Theorem. B-valued r.v. X has mean E ∈ B^{⋆⋆} if E{|η(X)|} < ∞ for η ∈ B^{⋆}, and covariance Q:B^{⋆} →B^{⋆⋆} if E{(η(X))^{2}}<∞ for η ∈B^{⋆}. Proof. B^{⋆} ∋η →L(η)≡η(X)∈𝕃^{1}(Ω) is linear with {(η, L(η)) : η ∈B^{⋆}} closed, since η_{α} →_{B^{⋆}} η and L(η_{α}) = η_{α}(X) →_{𝕃^{1}(Ω)} Y give η_{α}(X) → η(X), ω ∈ Ω, so η_{α}(X) →_{P} η(X) and 𝕃^{1}(Ω)-limit Y must be η(X). By Closed Graph Theorem L is bounded, so linear map E(η) = E{η(X)} has |E(η)| ≤ E{|η(X)|} = ∥L(η)∥_{𝕃^{1}(Ω)} ≤ ∥L∥∥η∥_{B^{⋆}}, i.e., is in B^{⋆⋆}. Q is similar (MT 35). □ Definition. B-valued r.v. X [P ∈P(B)] \underline{Gaussian} if η(X) [P η^{−1}] Gaussian for η∈B^{⋆}. Corollary. Gaussian B-valued r.v. has well-define mean and covariance.

Direct Proof. (Qη)(ν) = Cov{η(X),ν(X)} is linear in ν. If ν_{α} →_{B^{⋆}} 0, then ν_{α}(X) → 0, ω ∈ Ω, so that Var{ν_{α}(X)} → 0 by nature of Gaussian r.v.’s. Hence |(Qη)(ν_{α})| ≤ \sqrt{Var\{η(X)\}}\sqrt{Var\{ν_{α}(X)\}} → 0. E is similar. □ Definition. Process \{X(t)\}_{t∈T} \underline{Gaussian} if ∑_{i=1}^{k} a_{i}X(t_{i}) N-distributed, a_{i} ∈ ℝ, t_{i} ∈T, k ∈ℕ, and \underline{standard Wiener} if zero-mean Gaussian with E\{X(s)X(t)\}=s∧t. Example. Pick continuous Gaussian process {X(t)}_{t∈[0,1]} with (continuous) mean m(t)=E{X(t)} and covariance r(s, t) = Cov{X(s), X(t)}. Since X_{n}(t) ≡ X(⌊n t⌋/n)+(nt−⌊nt⌋)X(⌈nt⌉/n) →_{C} X surely, for F ∈ BV[0, 1] = C^{⋆}, F X_{n} → F X surely. Since F X_{n} is Gaussian, F X is, so X ∼N_{C}(E, Q). (Conversly, a C-valued Gaussian r.v. trivially corresponds to continuous Gaussian process.)

For F, G∈BV[0, 1], by Fubini, E(F) = E{F X} = E{∫_{0}^{1}X dF } = ∫_{0}^{1}m dF, and (QF)(G) = Cov{F X, QX} = Cov{∫_{0}^{1}X dF, ∫_{0}^{1}X dG} = ∫_{0}^{1}∫_{0}^{1}r(s, t) dF (s)dG(t). Since E(F)=F (m)=m(F), mean is Pettis. For \underline{standard Wiener process W}

∫_{0}^{1}∫_{0}^{1}r_{W} dF dG = ∫_{0}^{1}t(∫_{t}^{1} dF (s))dG(t) + ∫_{0}^{1}s(∫_{s}^{1} dG(t))dF (s)
= ∫_{0}^{1}t(F (1)−F (t))dG(t) + ∫_{0}^{1}s(G(1)−G(s))dF (s)

= [−t(F(1)−F(t))(G(1)−G(t))]_{0}^{1} + ∫_{0}^{1}(F(1)−F(t))(G(1)−G(t)) dt.

Characteristic Functions (chf.). Definition. \underline{Chf.} of B-valued r.v. X is B^{⋆} ∋ \underline{ϕ_{X}(η)} ≡ E\{e^{iη(X)}\} ∈ ℂ. Proposition. Chf. of B-valued r.v. X is w^{⋆}-continuous and non-negative definite. Proof. For {η_{α}}_{α∈A} ⊆ B^{⋆} with η_{α} →_{w^{⋆}} η, η_{α}(X) → η(X) surely, so η_{α}(X) ⇒ η(X) in P(ℝ) and ϕ_{X}(η_{α}) = ϕ_{η_{α}(X)}(1) → ϕ_{η(X)}(1) = ϕ_{X}(η). Further, ∑_{k,ℓ=1}^{n}a_{k}\overline{a_{ℓ}} ϕ_{X}(η_{k}−η_{ℓ}) = E\{∑_{k,ℓ=1}^{n}a_{k}\overline{a_{ℓ}} e^{i[η_{k}(X)−η_{ℓ}(X)]}\} = E\{|∑_{k}^{n}a_{k}e^{iη_{k}(X)}|^{2}\}. □

Proposition. Distributions of B-valued r.v.’s agree iff. their chf. do. Proof. ⇒ PX^{−1} =PY^{−1} on Dynkin System containing π-system Π ≡ {⋂_{i=1}^{k}{x ∈ B : |η_{i}(x−y_{i})| < ε_{i}}, η_{i} ∈ B^{⋆}, y_{i} ∈ B, ε_{i} > 0, k ∈ ℕ}, since E{e^{i ∑_{ℓ=1}^{k}a_{ℓ}η_{ℓ}(X)}} = E{e^{i ∑_{ℓ=1}^{k}a_{ℓ}η_{ℓ}(Y)}}. For z ∈B, there is η_{z} ∈(0)_{1}^{⋆} with η_{z}(z)=∥z∥ (MT 33), so for countable w^{⋆}-dense D^{⋆} ⊆(0)_{1}^{⋆} (that is w^{⋆}-compact metric, by MT 22)

(⋆)  {y ∈B : ∥x−y∥<r} = _{⋂η∈D_{⋆}}{y ∈B : |η(x−y)|<r},

so σ(Π) hosts B-balls. Here definition of ∥·∥^{⋆} gives ⊆, while ⊇ is from D^{⋆} includes items arbitrarily w^{⋆}-close to η_{x−y}. [Notice (⋆) means (0)_{1}^{⋆} separates points in B, since η(x)=η(x̂) for η ∈(0)_{1}^{⋆} ⇒ {y : ∥x−y∥<r} = {y : ∥x̂−y∥<r}, r >0]. □

Corollary. X_{n} ⇒ X in P(B), some X, iff. {PX_{n}^{−1}} is relatively compact and ϕ_{X_{n}} →ϕ on B^{⋆}, some ϕ. Then ϕ_{X} =ϕ. Example. Distribution of B-valued Gaussian X given by E, Q, since ϕ_{X}(η) = e^{iEη− \frac{1}{2} (Qη)(η)}, and given by marginal distributions of Gaussian process \{η(X)\}_{η∈B^{⋆}}. To E ∈ B^{⋆⋆} and symmetric non-negative Q: B^{⋆} → B^{⋆⋆}, there is Gaussian process {x(η)}_{η∈B^{⋆}} with E{x(η)} = E(η) and Cov{x(η), x(ν)} = (Qη)(ν). If Q is covariance of B-valued Gaussian r.v. X, η(X) =_{d} x(η) for η ∈B^{⋆} (MT 36). Example. For ξ_{1}, ξ_{2}, … iid. standardized r.v.’s, the C-valued r.v.’s X_{n}(t) ≡ n^{−1/2}(∑_{j=1}^{⌊nt⌋} ξ_{j} +(nt−⌊nt⌋)ξ_{⌈nt⌉}) satisfy ϕ_{X_{n}}(F) → ϕ_{W} (η) for continuous F ∈ BV[0, 1], since elementary chf. ϕ_{ξ_{1}}(t) is 1 − \frac{1}{2}t^{2}(1+o(1)), so that

ϕ_{X_{n}}(F) = E{e^{i∫_{0}^{1}X_{n}dF}} = E{exp[[iX_{n}(t)(F(t)−F(1))]_{0}^{1} + i∫_{0}^{1}X′_{n}(t)(F(1)−F(t))dt]}
= E\{exp[0 + i∑_{k=1}^{n}∫_{(k−1)/n}^{k/n}\sqrt{n}ξ_{k} (F (1)−F (t)) dt]\}
∼ ∏_{k=1}^{n}[1 − \frac{1}{2}(\sqrt{n}∫_{(k−1)/n}^{k/n}(F (1)−F (t)) dt)^{2}]
= exp\{∑_{k=1}^{n} ln[1 − \frac{1}{2}n(∫_{(k−1)/n}^{k/n}(F (1)−F (t)) dt)^{2}]\}
∼ exp\{−\frac{1}{2}∑_{k=1}^{n}∫_{(k−1)/n}^{k/n}(F (1)−F (t))^{2} dt\} = ϕ_{W} (F).

Chf. for infinite dimensional r.v.’s are chiefly used to show unicity, since tightness issue is difficult. We give flavour of theory by listing few results from the literature.

\underline{Hilbert-Schmidt topology} on real separable Hilbert space H is weak topology generated by the Hilbert-Schmidt operators [i.e., all linear functions L: H → H with ∑_{k=1}^{∞} ∥Le_{k}∥^{2} < ∞ for some ortonormal basis {e_{k}}_{k=1}^{∞} ⊆H]. Theorem. (Sazonov) Let H be a real separable Hilbert space. A function ϕ:H → ℂ is chf. for some H-valued r.v. iff. it is continuous in the Hilbert-Schmidt topology, with ϕ(0)=1, and non-negative definite.

For converse, notice linear L: H → H is continuous iff. bilinear form H ×H ∋ (x, y) → L(x, y) ≡ ⟨Lx, Ly⟩ ∈ ℝ is. Theorem. (Mouchtari) If there exists weak topology on B^{⋆} generated by class of bilinear forms B^{⋆}×B^{⋆} ∋ (x, y) → L(x, y) ∈ ℝ, such that ϕ:B^{⋆} →ℂ is chf. iff. it is non-negative definite, with ϕ(0)=1, and continuous, then B is Hilbert space.

Complete linear metric space Φ over ℝ \underline{countably Hilbert} if metric is ∑_{k=1}^{∞} 2^{−k}(1∧∥·∥_{k}) for Hilbertian norms {∥·∥_{k}}_{k=1}^{∞} with ∥·∥_{k}-Cauchy ∥·∥_{ℓ}-convergent sequences ∥·∥_{k}-convergent for k, ℓ ∈ ℕ. Countably Hilbert space Φ \underline{nuclear} if ∑_{n=1}^{∞} ∥η_{n}∥<∞ iff. ∑_{n=1}^{∞} |η_{n}(x)|<∞ for x∈Φ, whenever {η_{n}}_{n=1}^{∞} ⊆Φ^{⋆}.

Countably Hilbert spaces are much more general than Hilbertian ones. Nuclear spaces are very special, but include e.g., the Schwarz spaces D and S.

A r.v. X with values in a topological vector space (TVS) E, with \underline{dual E^{⋆}} ≡ {continuous linear η:E →ℝ}, has chf. E^{⋆} ∋ η → ϕ_{X}(η) ≡ E{e^{iη(X)}}. Theorem. (Boulicaut) Let E be a separable metrizable HLCTVS (Hausdorff locally convex TVS). We have, for any individually tight P_{n}, P ∈ P(E) with chf. ϕ_{n}, ϕ, that ϕ_{n}(η)→ϕ(η) for η ∈E^{⋆} implies P_{n} ⇒P in P(E), iff. E is nuclear.

6. Miscellany [Metrics for Weak Convergence (cf. MT 30)]. Theorem. For S separable, P(S) is metrized by \underline{dual bounded Lipschitz norm} \underline{∥P ∥_{BL}^{⋆}} ≡ sup\{P f : f ∈BL(S), ∥f∥_{BL} ≤1\}. (Cf. MT 21.) Proof. We show ∥P_{α}−P ∥_{BL}^{⋆}→0 iff. P_{α}f →P f for f ∈BL(S). ⇒ is trivial, sin-

ce |P_{α}f−P f| ≤ ∥f∥_{BL} ∥P_{α}−P ∥_{BL}^{⋆}. For ⇐ , pick ε>0, dense {x_{i}}_{i=1}^{∞} ⊆S, and P -

continuity S_{r_{i}}(x_{i}), r_{i} ∈[\frac{ε}{4}, \frac{ε}{2}], so B_{i} ≡ S_{r_{i}}(x_{i})\(∪_{j=1}^{i−1}S_{r_{j}}(x_{j})) partition S. Since |f( x) −f(x_{i})|≤ \frac{ε}{2}∥f∥_{BL}, x∈B_{i}, and |f|≤∥f∥_{BL}, for f ∈BL(S), Scheffé gives

|P_{α}f −P f| = |_{∑i}(_{∫B_{i}}(f −f(x_{i})) dP_{α} + f(x_{i})(P_{α}(B_{i})−P (B_{i})) +_{∫B_{i}}(f(x_{i})−f) dP )|
≤ (ε + ∑_{i=1}^{∞}|P_{α}(B_{i})−P (B_{i})|)∥f∥_{BL} → ε∥f∥_{BL}. □

Theorem 6.8. For S separable, P(S) is separable and metrized by \underline{Prohorov met}- \underline{ric ρ(P, Q)} ≡ inf\{ε≥0 : P (F)≤Q(F_{ε})+ε for closed F ⊆S\}. Proof. ρ(P, P)=0. If ρ(P, Q)=0, P (F)≤Q(F), closed F, since Q(F_{ε})↓Q(F), so P (G) = 1−P (G^{c}) ≥ 1−Q(G^{c}) = Q(G), open G, i.e., P = Q by regularity. If P (F) ≤ Q(F_{ε})+ε, closed F, P (F_{ε}) = 1−lim_{ε̂↓ε} P ((Fε̂)^{c}) ≥ 1−lim_{ε̂↓ε} Q(((Fε̂)^{c})_{ε})−ε ≥ Q(F)−ε since F ⊆(((Fε̂)^{c})_{ε})^{c} (see MT 11) so symmetry. P (F) ≤ R(Fε̂)+ε̂≤ Q ((Fε̂)ε̃)+ε̂+ε̃≤ Q(F_{ε̂+ε̃})+ε̂+ε̃ (MT 11) if ρ(P, R)<ε̂, ρ(R, Q)<ε̃, so Δ-inequailty.

If P_{n} →_{ρ} P, then, given ε>0, P_{n}(F)≤P (F_{ε})+ε for closed F ∈S, for n large enough, so that lim sup_{n} P_{n}(F)≤P (F), i.e., P_{n} ⇒P in P(S).

Let P_{n} ⇒ P in P(S), so that P_{n} →_{∥·∥_{BL}^{⋆}} P. Given ε > 0, for closed F ⊆ S, f_{F} ≡ (1−d(·, F)/ε))^{+} ∈ BL(S) with ∥f_{F} ∥_{BL} ≤1+1/ε and 1_{F} ≤ f_{F} ≤ 1_{F_{ε}}. Hence P (F) ≤ P f_{F} ≤ P_{n}f_{F} + ∥P −P_{n}∥_{BL}^{⋆}∥f_{F} ∥_{BL} ≤ P_{n}(F_{ε}) + ε for ∥P −P_{n}∥_{BL}^{⋆} ≤ \frac{ε}{1+ \frac{1}{ε}}.

Let \{A_{k}\}_{k=1}^{∞} ⊆𝔖 partition S with d(A_{k})≤ \frac{1}{n}, and a_{k} ∈A_{k}. Put N_{n} ≡ \{∑_{k=1}^{∞} r_{k}δ_{a_{k}} ∈P(S) : r_{k} ∈ℚ}. For P ∈P(S), pick Q = ∑_{k≤ℓ} r_{k}δ_{a_{k}} ∈ N_{n} with ∑_{k=1}^{ℓ} |P ( A_{k})−r_{k}|≤ \frac{1}{2n}, where P\{∪_{k>ℓ}A_{k}\}< \frac{1}{2n}. Then ρ(P, Q)≤ \frac{1}{n}, since for F ⊆S closed,

P (F) ≤ _{∑k≤ℓ:F ∩A_{k}≠∅}P (A_{k}) + \frac{1}{2n} ≤ _{∑k≤ℓ:F ∩A_{k}≠∅}r_{k} + \frac{1}{n} ≤ Q(F_{1/n}) + \frac{1}{n}. □

Theorem 6.8. (P(S), ρ) and (P(S), ∥ · ∥_{BL}^{⋆}) are separable complete for S such. Proof. By inspection of above proof, ∥·∥_{BL}^{⋆}-Cauchy is ρ-Cauchy, so is enough do latter. For such, is enough show tight, by Prohorov (cf. MT 42). By familiar arguing, this follows if, given ε, δ >0, P_{n}(∪_{i=1}^{j}S_{δ}[x_{i}]) ≥ 1−ε, n∈ℕ, some {x_{i}}_{i=1}^{j} ⊆S. But ρ(P_{k}, P_{n}) ≤ (ε∧δ)/2, k, n ≥ n_{0}, and P_{n}(∪_{i=1}^{j}S_{δ/2}[x_{i}]) ≥ 1−ε/2, n ≤ n_{0}, give P_{n}( ∪_{i=1}^{j}S_{δ}[x_{i}]) ≥ P_{n}((∪_{i=1}^{j}S_{δ/2}[x_{i}])_{δ/2}) ≥ P_{n_{0}}(∪_{i=1}^{j}S_{δ/2}[x_{i}])−ε/2 ≥ 1−ε, n≥n_{0}. □

7. Weak Convergence in P(C(T )). Theorem 7.3. Π⊆P(C(T )) is relatively compact iff. the following two holds

(i) {P π_{t}^{−1} : P ∈Π} is tight for t∈T ; and (ii) To each ε>0 there exist δ >0 and finite Γ ⊆Π such that

P ({x∈C(T ) : sup{|x(s)−x(t)|: s, t∈T , ϱ(s, t)≤δ}>ε}) ≤ ε for P ∈Π\Γ.

Proof. ⇒ Since S =C(T ) separable (MT 24) and complete, Prohorov shows Π tight, so for ε>0, inf{P (K) : P ∈Π} ≥ 1−ε, some compact K⊆S. By Ascoli, K ⊆ {x∈S: sup_{s,t∈T , ϱ(s,t)≤δ} |x(t)−x(s)|≤ε}, some δ >0, which gives (ii), with Γ =∅. By Ascoli, for t∈T , K ⊆ {x∈S : |x(t)|≤M}, some M >0, so (i) follows from

inf{P π_{t}^{−1}([−M, M]) : P ∈Π} = inf{P ({x∈S : |x(t)|≤M}) : P ∈Π} ≥ 1−ε. ⇐ By Prohorov’s Theorem, it suffices show Π tight. Since Γ tight, proof of ⇒ gives (ii) for Γ, so may put Γ =∅ in (ii). Take δ_{k} >0 with P ({x∈S: sup{|x(s)− x(t)|: s, t∈T , ϱ(s, t)≤δ_{k}\}> \frac{1}{k} \}) ≤ 2^{−k}ε for P ∈Π. Let T ⊆ (\{t_{1}, …, t_{m}\})_{δ_{1}} (T totally bounded), where by (i), P (\{x ∈ S : |x(t_{i})| > M\}) ≤ \frac{ε}{2m}, some M > 0. By Ascoli, the following set K =clos(K) is compact, with P (K^{c})≤ε for P ∈Π, K ≡ ⋂_{i=1}^{m}\{x∈S: |x(t_{i})|≤M\} ∩ ⋂_{k=1}^{∞}\{x∈S: sup_{s,t∈T , ϱ(s,t)≤δ_{k}} |x(s)−x(t)|≤ \frac{1}{k} \}. □

Corollary (MT 33). For T connected, (i) in Theorem 7.3 may be replaced with (i′) {P π_{t}^{−1} : P ∈Π} is tight for some t∈T . Corollary (MT 33). (ii) in Theorem 7.3 may be replaced with (ii′) To each ε>0 there exist δ >0 and finite Γ ⊆Π such that P ({x∈C(T ) : sup{|x(s)−x(t)|: s∈T , ϱ(s, t)≤δ}>ε}) ≤ ε for P ∈Π\Γ, t∈T .

Corollary (MT 33). P_{n} ⇒P in P(C(T )), some P, iff. the following two hold (i) P_{n}π_{t_{1},...,t_{k}}^{−1} ⇒ in P(ℝ^{k}) for t_{i} ∈T , k ∈ℕ (where the limit must be P π_{t_{1},...,t_{k}}^{−1}); (ii) To each ε>0 there exist δ >0 and n_{0} ∈ℕ such that P_{n}({x∈C(T ) : sup{|x(s)−x(t)|: s∈T , ϱ(s, t)≤δ}>ε}) ≤ ε for n≥n_{0}, t∈T .

Criterion. (Cf. MT 38) C-valued r.v.’s {X_{n}}_{n=1}^{∞} relatively compact if lim_{M→∞} sup_{n} P{|X_{n}(t̂)|>M}=0, some t̂∈[0, 1], and for some constants C, β, γ, x_{0} >0

sup_{n} P{|X_{n}(s)−X_{n}(t)|>x} ≤ C |t−s|^{1+γ}/x^{β} for s,t∈[0,1], x∈(0, x_{0}].

Proof. (i′) from first condition, and (ii) from second, since picking a∈(0, γ/β),

P{sup_{s∈[t−δ,t+δ]} |X_{n}(s)−X_{n}(t)|>ε} ≤ P\{⋃_{k=1}^{∞} ⋃_{\frac{ℓ}{2^{k}} ∈[−δ,δ]}\{|X_{n}(t+ \frac{ℓ}{2^{k}})−X_{n}(t)|>(1− \frac{1}{2^{ak}})ε\}\} ≤ P\{⋃_{\frac{ℓ}{2} ∈[−δ,δ]}\{|X_{n}(t+ \frac{ℓ}{2})−X_{n}(t)|>(1− \frac{1}{2^{a}})ε\}\}

+ ∑_{k=1}^{∞}P\{⋃_{\frac{i}{2^{k+1}} ∈[−δ,δ]}\{|X_{n}(t+ \frac{i}{2^{k+1}} )−X_{n}(t)|>(1− \frac{1}{2^{a(k+1)}} )ε\},

⋂_{\frac{j}{2^{k}} ∈[−δ,δ]}\{|X_{n}(t+ \frac{j}{2^{k}})−X_{n}(t)|≤(1− \frac{1}{2^{ak}})ε\}\}

≤ 0 + ∑_{k=1}^{∞} ∑_{\frac{i}{2^{k+1}} ∈[−δ,δ]}P\{|X_{n}(t+ \frac{2i+1}{2^{k+1}})−X_{n}(t)|−|X_{n}(t+ \frac{2i(+2)}{2^{k+1}})−X_{n}(t)| > (1− \frac{1}{2^{a}})\frac{ε}{2^{ak}}\}

≤ ∑_{k=1}^{∞}2^{k+3}δ c 2^{−(k+1)(1+γ)}/((1− \frac{1}{2^{a}})\frac{ε}{2^{ak}} )^{β} ≤ ε for δ >0 small enough. □

8. Wiener Measure and Donsker’s Theorem. Theorems 8.1-8.2 (MT 37). (Donsker) For iid. standardized r.v.’s ξ_{1}, ξ_{2}, …, {X_{n}(t)}_{t∈[0,1]} ≡ {n^{−1/2}(∑_{j=1}^{⌊nt⌋} ξ_{j} +(nt−⌊nt⌋)ξ_{⌈nt⌉})}_{t∈[0,1]} ⇒ W in P(C). Proof. For [\frac{k}{n}, \frac{k+1}{n})∋(t−δ)∨0<(t+δ)∧1∈[\frac{ℓ−1}{n}, \frac{ℓ}{n}), by Doob-Kolmogorov Inequality

P\{sup_{s∈[t−δ,t+δ]} |X_{n}(t)−X_{n}(s)|>ε\} ≤ 2P\{sup_{j∈\{k,...,ℓ\}} |X_{n}(\frac{j}{n})−X_{n}(\frac{k}{n})|> \frac{1}{2}ε\}

= 2P\{sup_{i∈\{1,...,ℓ−k\}} |∑_{j=k+1}^{k+i}ξ_{j}|> \frac{1}{2}ε\sqrt{n}\} ≤ 2E\{(∑_{j=1}^{ℓ−k}ξ_{j})^{2}\}/(\frac{1}{2}ε\sqrt{n})^{2} = 8(ℓ−k)/(ε^{2}n) ≤ 8((n(t+δ)+1) − (n(t−δ)−1))/(ε^{2}n) ≤ ε

for n ≥ n_{0}, δ ≤ δ_{0}, giving (ii′). (i′) is trivial (t = 0). So relatively compact. By example, convergent subsequences go to Gaussian with covariance min{s, t}. (We found limit chf. for continuous F ∈BV[0, 1]. Such F are enough by MT 26.) □

12. Geometry of D. Definition. • \underline{Λ} ≡ \{increasing bijective λ:[0, 1]→[0, 1]\};

• Λ ∋ λ → \underline{∥λ∥_{Λ}} ≡ sup_{s,t∈[0,1], s≠t} | ln \frac{λ(t)−λ(s)}{t−s}| ∈ [0, ∞);

• D×D ∋ (x, y) → \underline{d_{1}(x, y)} ≡ inf\{∥λ∥_{Λ}∨∥x−y∘λ∥_{C} : λ∈Λ\} ∈ [0, ∞). Theorem 12.2. \underline{Skorokhod J_{1}-space} (D, d_{1}) complete separable metric space. Proof. λ=I gives d_{1}(x, y)<∞ and d_{1}(x, x)=0. d_{1}(x, y)≠0 for x(1)≠y(1), since λ(1)=1. |x(t̂)−y(t̂)|>0, t̂<1, gives |x(t)−y(t)|≥ε, t∈[t̂, t̂+δ], some ε, δ >0, so ∥x−y∘λ∥_{C} ≥ε, or ∥λ−I∥_{C} ≥δ so sup |\frac{λ(t)−λ(s)}{t−s}−1| ≥ sup |\frac{λ(t)−t}{t}| ≥ δ, i.e., ∥λ∥_{Λ} ≥|ln(1±δ)|, so d_{1}(x, y)>0. If ∥λ̂∥_{Λ} ≤ε̂, ∥x−z∘λ̂∥_{C} ≤ ε̂ and ∥λ̃∥_{Λ} ≤ε̃, ∥y−z∘λ̃∥_{C} ≤ ε̃, then ∥x−y∘λ̂∘λ̃^{−1}∥_{C} ≤ ∥x−z∘λ̂∥_{C} +∥z∘λ̂−y∘λ̂∘λ̃^{−1}∥_{C} ≤ ε̂+ε̃, ∥λ̂∘λ̃^{−1}∥_{Λ} ≤ sup|ln \frac{λ̂∘λ̃^{−1}(t)−λ̂∘λ̃^{−1}(s)}{λ̃^{−1}(t)−λ̃^{−1}(s)}|+sup|ln \frac{λ̃^{−1}(t)−λ̃^{−1}(s)}{t−s}| ≤ ε̂+ε̃. So d_{1} metric.

\{∑_{i=1}^{n−1} e_{i}1_{[ \frac{i−1}{n−1} , \frac{i}{n−1} )}+e_{n}1_{\{1\}} : e_{1}, …, e_{n} ∈ℚ, n∈ℕ\} dense in D, so separable. Enough show Cauchy {y_{n}}_{n=1}^{∞} ⊆ D with d_{1}(y_{n}, y_{m}) < 2^{−n}, m ≥ n, converges

(MT 42). Taking λ_{n} ∈Λ with ∥λ_{n}∥_{Λ} <2^{−n} and ∥y_{n}−y_{n+1}∘λ_{n}∥_{C} <2^{−n}, we have ∥λ_{n}−I∥_{C} ≤ C2^{−n}, somce C >0. Putting μ_{k}^{n} ≡λ_{k+n}∘…∘λ_{n}, this give, for k ≤ℓ, ∥μ_{ℓ}^{n}−μ_{k}^{n}∥ ≤ ∑_{i=k}^{ℓ−1}∥λ_{i+1+n}∘…∘λ_{n}−λ_{i+n}∘…∘λ_{n}∥ = ∑_{i=k}^{ℓ−1}∥λ_{i+1+n}−I∥ ≤ C2^{−(k+n)}, so μ_{k}^{(n)} →_{C} λ̂_{n}. Here λ̂_{n} ∈Λ, since λ̂_{n}(0)=0, λ̂_{n}(1)=1, and strictly increasing

| ln \frac{λ̂_{n}(t)−λ̂_{n}(s)}{t−s}| = lim_{k} | ln \frac{μ_{k}^{n}(t)−μ_{k}^{n}(s)}{t−s}| = lim_{k}|ln \frac{λ_{n}(t)−λ_{n}(s)}{t−s} + ∑_{j=0}^{k−1} ln \frac{μ_{j+1}^{n}(t)−μ_{j+1}^{n}(s)}{μ_{j}^{n}(t)−μ_{j}^{n}(s)} |
≤ ∥λ_{n}∥_{Λ} + ∑_{j=0}^{k−1}∥λ_{n+j+1}∥_{Λ} ≤ 2^{−n}.

Since λ̂_{n} =λ̂_{n+1}λ_{n}, so λ̂_{n+1}^{−1}λ̂_{n} =λ_{n}, we get ∥y_{n}∘λ̂_{n}^{−1}−y∥_{C} →0, some y, since ∥y_{k}∘λ̂_{k}^{−1}−y_{n}∘λ̂_{n}^{−1}∥ ≤ ∑_{j=n}^{k−1}∥y_{j+1}∘λ̂_{j+1}^{−1}−y_{j} ∘λ̂_{j}^{−1}∥ = ∑_{j=n}^{k−1}∥y_{j+1}∘λ_{j} −y_{j}∥ ≤ 2^{1−n}, k >n. Here y ∈D, so y_{n} →_{D} y, since ∥λ̂_{n}^{−1}∥_{Λ} = ∥λ̂_{n}∥_{Λ} ≤ 2^{−n} → 0. □ Definition. For x:[0, 1]→ℝ,

• \underline{w_{x}(δ)} ≡ sup_{s,t∈[0,1], |s−t|≤δ} |x(s)−x(t)|;

• \underline{w′_{x}(δ)} ≡ inf_{0=t_{0}<...<t_{k}=1, t_{i}−t_{i−1}>δ} sup_{s,t∈[t_{i−1},t_{i})} |x(s)−x(t)|;

• \underline{w′′_{x}(δ)} ≡ sup_{0≤r≤s≤t≤1, t−r≤δ} |x(s)−x(r)| ∧ |x(t)−x(s)|;

• \underline{j_{x}} ≡ sup_{t∈(0,1]} |x(t^{−})−x(t)|. Lemma 12.1 (MT 40). For x:[0, 1]→ℝ,

• w′_{x}(δ/2) ≤ w_{x}(δ) ≤ 2w′_{x}(δ)+j_{x};

• w′′_{x}(δ) ∨ |x(δ)−x(0)| ∨ |x(1−0)−x(1−δ)| ≤ w′_{x}(2δ);

• w′_{x}(\frac{δ}{2}) ≤ 24(w′′_{x}(δ) ∨ |x(δ)−x(0)| ∨ |x(1−0)−x(1−δ)|);

• x∈D iff. lim_{δ↓0} w′_{x}(δ) = 0;

• x∈D is measurable;

• sup_{t∈[0,1]} |x(t)| < ∞ for x∈D;

• #{t∈(0, 1] : |x(t^{−})−x(t)|>δ} < ∞ for x∈D. Proof. Everything easy, except third inequality, which has long boring proof. □ Theorem 12.3. A⊆D is relatively compact iff. following two hold

(i) sup_{x∈A} ∥x∥_{C} < ∞; (ii) lim_{δ↓0} sup_{x∈A} w′_{x}(δ) = 0.

Proof. ⇐ Since D separable complete, enough show A totally bounded. Take ε-net e_{1}, …, e_{n} for {x(t) : x ∈ A, t ∈ [0, 1]}. Let sup_{x∈A} w′_{x}(3δ) ≤ ε, some δ > 0, and | ln \frac{δ±m^{−1}}{δ} | ≤ ε, some ℕ ∈ m ≥ δ^{−1}. For x ∈ A, ∥x−y ∘λ∥_{C} ≤ ε, some y ∈ \{∑_{i=1}^{m} e_{j_{i}}1_{[ \frac{i−1}{m} , \frac{i}{m})}+e_{j_{m+1}}1_{\{1\}}: 1≤j_{i} ≤n\}, some pice-wise linear λ∈(0)_{ε}^{Λ}. □ Theorem 12.4. A ⊆ \underline{D} ≡ D[0, 1] is relatively compact iff. following four hold (i) sup_{x∈A} ∥x∥_{C} < ∞; (ii) lim_{δ↓0} sup_{x∈A} w′′_{x}(δ) = 0; (iii) lim inf_{δ↓0} sup_{x∈A} |x(δ)−x(0)| = 0 (lim_{δ↓0} sup_{x∈A} |x(δ)−x(0)| = 0); (iv) lim inf_{δ↓0} sup_{x∈A} |x(1−0)−x(1−δ)| = 0 (lim_{δ↓0} sup_{x∈A} |x(1−0)−x(1−δ)| = 0). Proof. Theorem 12.3, previous lemma, and monotonicity of w′_{x}(·) and w′′_{x}(·). □ Theorem 12.5. π_{t_{1},...,t_{k}} measurable. π_{0}, π_{1} continuous. For t∈(0, 1), π_{t} continuous at x iff. x continuous at t. For T ∋1 dense in [0, 1], σ({π_{t}^{−1}H : t∈T, H ⊆ℝ Borel}) = σ(J_{1}) and {π_{t_{1},...,t_{k}}^{−1}H_{1}×…×H_{k} : t_{i} ∈T, H_{i} ⊆ℝ Borel, k ∈ℕ} separating. Proof. Continuity easy. D ∋ x → h_{ε}(x) ≡ \frac{1}{ε} ∫_{t}^{t+ε}x ∈ ℝ continuous, since d_{1}(x_{n}, x) →0 ⇒x_{n} →x a.e., so π_{t} =lim_{ε} h_{ε} and π_{t_{1},...,t_{k}} measurable. V_{n}∘π_{t_{0},...,t_{n}}x ≡ ∑_{i=1}^{n} x(t_{i})1_{[t_{i−1},t_{i})}+x(t_{n})1_{{t_{n}}} →_{d_{1}} x as t_{i}−t_{i−1} →0 (as for separability), so I σ({π_{t}^{−1} H : t∈T, H ⊆ℝ Borel})-measurable, since π_{t_{0},...,t_{n}} is, and V_{n} continuous. □

13. Weak Convergence in P(D). Proposition. \underline{T_{P}} ≡ \{t ∈ [0, 1] : P (π_{t} continuous at t) = 1\} ∋ 0, 1 and #T_{P}^{c} ≤ ℵ_{0} for P ∈P(D). t∈T_{P} ∩(0, 1) iff. {x∈D: x(t−0)≠x(t)} P -null. Proof. Last statement by π_{t} continuous at x iff. x is at t. 0, 1∈T by π_{0}, π_{1} continuous. #\{t : P (x ∈ D: |x(t^{−})−x(t)| > \frac{1}{n}) > \frac{1}{m}\} < ∞, so #\{t : P (x ∈ D: |x(t^{−}) −x(t)| > \frac{1}{n}) > 0\} ≤ ℵ_{0}, and same for #\{t : P (x ∈ D: |x(t^{−})−x(t)| > 0) > 0\}, by P (⋂_{k=1}^{∞}⋃_{ℓ=k}^{∞}\{x∈D: |x(t_{ℓ}^{−})−x(t_{ℓ})|> \frac{1}{n}\}) = 0 for infinite \{t_{k}\}_{k=1}^{∞} ⊆[0, 1]. □ Theorem 13.1. P_{n}⇒P in P(D) some P, iff. {P_{n}}_{n=1}^{∞} tight and P_{n}π_{t_{1},...,t_{k}}^{−1}⇒ in P(ℝ^{k}), t_{i} in some dense T ∋1 in [0, 1], k ∈ℕ. Then P π_{t_{1},...,t_{k}}^{−1}⇐P_{n}π_{t_{1},...,t_{k}}^{−1}, t_{i} ∈T. Proof. Tightness necessary, by D separable complete. Given tightness, sufficient show convergence on separating class, which {π_{t_{1},...,t_{k}}^{−1}H_{1}×…×H_{k} : t_{i} ∈ T, H_{i} ⊆ ℝ Borel, k ∈ℕ} is. Convergence on T =T_{P} necessary, by Mapping Theorem. □ Theorem 13.2. {P_{n}}_{n=1}^{∞} ⊆P(D) tight iff. following two hold

(i) lim_{M→∞} lim sup_{n} P_{n}(x∈D: ∥x∥_{C} >M) = 0; (ii) lim_{δ↓0} lim sup_{n} P_{n}(x∈D: w′_{x}(δ)>ε) = 0 for ε>0.

Proof. ⇐ Let P_{n}(x ∈ D: w′_{x}(δ_{k}) > \frac{1}{k}) ≤ 2^{−k} \frac{ε}{2} and P_{n}(x ∈ D: ∥x∥_{C} > M) ≤ \frac{ε}{2},

n ∈ ℕ, so K ≡ ⋂_{k=1}^{∞}\{x ∈ D: w′_{x}(δ_{k}) ≤ \frac{1}{k} \} ∩ \{x ∈ D: ∥x∥_{C} ≤ M\} compact, by Theorem 12.3, with P_{n}(K) ≥ 1−ε. ⇒ Immediate from Theorem 12.3. □ Corollary. (i) in Theorem 13.2 may be replaced with either of (i′) lim_{M→∞} lim sup_{n} P_{n}(x∈D: |x(t)|>M) = 0 for dense T ∋1 in [0, 1]; (i′′) lim_{M→∞} lim sup_{n} P_{n}(x∈D: j_{x} >M) = 0 and (i′) holds for t=0. Proof. By (ii), pick δ >0 with P_{n}(x∈D:w′_{x}(δ)≤ \frac{1}{2}) ≥ 1−\frac{ε}{2}, n∈ℕ. Let 0= s_{0} <… < s_{m} = 1, s_{i} ∈ T, s_{i}−s_{i−1} < δ, so for x with w′_{x}(δ) ≤ \frac{1}{2}, and 0 = t_{0} < … < t_{k} = 1, t_{j}−t_{j−1} >δ, with sup_{s,t∈[t_{j−1},t_{j})} |x(t)−x(s)|≤1, since s_{i} ∈[t_{j−1}, t_{j}) some i, ∥x∥_{C} ≤ sup_{i} |x(s_{i})|+1, so by (i′), for M >0 with P_{n}(x∈D:|x(s_{i})|>M−1) ≤ \frac{ε}{2m}, n∈ℕ, we get (i) of Theorem 13.2. Sufficiency of (i′′) from w_{x}(δ) ≤ 2w′_{x}(δ)+j_{x}. □ Corollary (MT 45). P_{n} ⇒P in P(D), some P, iff. (ii) of Theorem 13.2 holds and P_{n}π_{t_{1},...,t_{k}}^{−1}⇒ in P(ℝ^{k}) for t_{i} ∈T_{P}, k ∈ℕ (where limit must be P π_{t_{1},...,t_{k}}^{−1}). Theorem 13.3. P_{n} ⇒ P in P(D), some P, iff. P_{n}π_{t_{1},...,t_{k}}^{−1} ⇒ in P(ℝ^{k}), t_{i} in some dense T ∋1 in [0, 1], k ∈ℕ, and following two hold

(i) lim inf_{δ↓0} lim sup_{n} P_{n}(x∈D: |x(1−0)−x(1−δ)|>ε) = 0 for ε>0; (ii) lim_{δ↓0} lim sup_{n} P_{n}(x∈D: w′′_{x}(δ)>ε) = 0 for ε>0. Proof. (ii) of Theorem 13.2 can be replaced with (ii)- (iv) of Theorem 12.4. So (i)- (ii) necessary. Sufficient, since lim sup_{n} P_{n}(x∈D:|x(δ_{k})−x(0)|>ε) ≥ η, some η >0 and T_{P} ∋δ_{k} ↓0, gives P (x∈D:|x(0+0)−x(0)|≥ε) ≥ η, contradicting cádlág. □ Corollary (MT 45). If content with if, (i) in Theorem 13.3 may be replaced with (i′) lim_{δ↓0} P (x∈D: |x(1)−x(1−δ)|>ε) = 0 for ε>0. Proof. (ii) gives lim sup_{n} P{|X_{n}(1)−X_{n}(1−0)|∧|X_{n}(1−0)−X_{n}(1−δ_{k})|>ε} → 0 and (i′) lim sup_{n} P{|X_{n}(1)−X_{n}(1−δ_{k})|>ε} → 0, some δ_{k} ↓0, so that lim sup_{n} P{|X_{n}(1 −0)−X_{n}(1−δ_{k})|>ε} → 0, i.e., (i). (Billingsley's proof seems incomplete!) □ Theorem 13.4. For X_{n} ⇒X in P(D), P{X ∈C} = 1 iff. j_{X_{n}} ⇒0 in P(ℝ). Proof. By Mapping Theorem, j_{X_{n}} ⇒j_{X}, since x_{n} →_{D} x easily gives j_{x_{n}} →j_{x}. For D-valued X, by Lemma 12.1, j_{X} =0 equivalent with X C-valued. □ Corollary. P_{n} ⇒ P in P(D) with P (C) = 1 if P_{n}π_{t_{1},...,t_{k}}^{−1} ⇒ P π_{t_{1},...,t_{k}}^{−1} in P(ℝ^{k}) for t_{i} ∈[0, 1], k ∈ℕ, and (ii) [or (ii′)] of Theorem 7.3 hold. Proof. Hypothesis of Theorems 13.1-2 hold, since ∥x∥_{C} ≤|x(0)|+ \frac{2}{δ}w_{x}(δ) and w′_{x} (\frac{δ}{2})≤w_{x}(δ), so ⇒ in P(D). And Theorem 13.4 applies, since j_{x} ≤w_{x}(δ), δ >0. □

Theorem 13.5. X_{n} ⇒ X in P(D), some X, if (X_{n}(t_{1}), …, X_{n}(t_{k})) ⇒ in P(ℝ^{k}), t_{i} in some dense T ∋1 in [0, 1], k ∈ℕ, and following two hold

(i) lim inf_{δ↓0} lim sup_{n} P{|X_{n}(1−0)−X_{n}(1−δ)|>ε} = 0 for ε>0; (ii) there are constants K, β, γ >0 such that P{|X_{n}(s)−X_{n}(r)|∧|X_{n}(t)−X_{n}(s)|>λ} ≤ K|t−r|^{1+γ}/λ^{β} for 0≤r<s<t≤1, λ>0.

Proof. Enough show (ii) of Theorem 13.3. Let D_{k} ≡ {2^{−k}i: i = 0, …, 2^{k}},

A_{k} ≡ max{|X_{n}(s)−X_{n}(r)|∧|X_{n}(t)−X_{n}(s)| : r, s, t∈D_{k}, s−r =t−s=2^{−k}, t−s≤δ}
B_{k} ≡ max{|X_{n}(s)−X_{n}(r)|∧|X_{n}(t)−X_{n}(s)| : r, s, t∈D_{k}, r ≤s≤t, t−s≤δ}
C ≡ sup{|X_{n}(s)−X_{n}(r)|∧|X_{n}(t)−X_{n}(s)| : r, s, t∈[0, 1], r ≤s≤t, t−s≤δ/2}

and, for t∈D_{k}, define t′_{k}(t)∈D_{k−1} by t′_{k}(t)≡t if t∈D_{k−1} and

t′_{k}(t) ≡ \{ \begin{matrix}t−2^{−k} if t∉D_{k−1} and |X_{n}(t)−X_{n}(t−2^{−k})|≤|X_{n}(t)−X_{n}(t+2^{−k})| \\ t+2^{−k} if t∉D_{k−1} and |X_{n}(t)−X_{n}(t−2^{−k})|>|X_{n}(t)−X_{n}(t+2^{−k})| \end{matrix}.

Notice |X_{n}(t)−X_{n}(t′_{k}(t))| ≤ A_{k}, t∈D_{k}, so for t̂, t̃∈D_{k},

|X_{n}(t̂)−X_{n}(t̃)| ≤ |X_{n}(t̂)−X_{n}(t′_{k}(t̂))|+|X_{n}(t′_{k}(t̂))−X_{n}(t′_{k}(t̃))|+|X_{n}(t′_{k}(t̃))−X_{n}(t̃)|
≤ |X_{n}(t′_{k}(t̂))−X_{n}(t′_{k}(t̃))|+2A_{k}.

For r, s, t∈D_{k} with r <s<t, t′_{k}(r)≤t′_{k}(s)<t′_{k}(t), so B_{k} ≤ B_{k−1}+2A_{k}, giving B_{k} ≤ 2 ∑_{j=1}^{k} A_{j}, since A_{0} =B_{0} =0, and C ≤ 2 ∑_{j=1}^{∞} A_{j}, by right-continuity. Thus

P\{C ≥ε\} ≤ ∑_{k=1}^{∞}P\{2A_{k} ≥(1− \frac{1}{θ})θ^{k}ε\}

≤ ∑_{k=1}^{∞}∑_{i=1}^{2^{k}−1}P\{|X_{n}(\frac{i}{2^{k}})−X_{n}(\frac{i−1}{2^{k}})|∧|X_{n}(2^{−k}(\frac{i+1}{2^{k}})−X_{n}(\frac{i}{2^{k}})|≥ \frac{1−\frac{1}{θ}}{2} θ^{k}ε\} ≤ ∑_{k=1}^{∞}2^{k}K(2^{−k})^{1+γ/2}δ^{γ/2}/(\frac{1−\frac{1}{θ}}{2} θ^{k}ε)^{β} → 0 as δ ↓0, some θ >0. □

Corollary. (i) in Theorem 13.5 may be replaced with (i′) X(1)−X(1−δ) ⇒ 0 in P(ℝ) as δ ↓0. Proof. This is same exchange of (i′) to (i) we have seen before. □ Corollary (MT 45). (ii) in Theorem 13.5 may be replaced with (ii′) E{|X_{n}(s)−X_{n}(r)|^{β} ∧|X_{n}(t)−X_{n}(s)|^{β}} ≤ K|t−r|^{1+γ}.

14. Applications [Functional Central Limit Theorems]. Theorem 14.1. (Donsker) For iid. standardized r.v.’s {ξ_{j}}_{j=1}^{∞}, {X_{n}(t)}_{t∈[0,1]} ≡ {n^{−1/2} ∑_{j=1}^{⌊nt⌋} ξ_{j}}_{t∈[0,1]} ⇒ W in P(D). Theorem 14.3. For uniform iid. [0, 1]-valued r.v.’s \{η_{j}\}_{j=1}^{∞}, \{Y_{n}(t)\}_{t∈[0,1]} ≡ \{\sqrt{n} (\frac{1}{n} ∑_{j=1}^{n} 1_{[0,t]}(η_{j})−t)\}_{t∈[0,1]} ⇒ W^{∘}(s) in P(D), where \{W^{∘}(s)(t)\}_{t∈[0,1]} is \underline{Brown}- \underline{ian bridge}, i.e., zero-mean Gaussian with E\{W^{∘}(s)W^{∘}(t)\} = s∧t−st.

Proof. ∑_{i=1}^{k} a_{i}(1_{[0,t_{i}]}(η)−t_{i}) is zero-mean with variance ∑_{i,j=1}^{k} a_{i}a_{j}(t_{i}∧t_{j}− t_{i}t_{j}),

∑_{i=1}^{k}a_{i}Y_{n}(t_{i}) = \frac{1}{\sqrt{n}}∑_{j=1}^{n}∑_{i=1}^{k}a_{i}(1_{[0,t_{i}]}(η_{j})−t_{i}) ⇒ N(0, Var\{∑_{i=1}^{k}a_{i}W^{∘}(t_{i})\}) in P(ℝ), by elementary CLT. So fidi’s ⇒ in P(ℝ^{k}), by Cramér-Wold Device. (i′) of Theorem 13.5 holds since E{[W^{∘}(1)−W^{∘}(1−δ)]^{2}} = … = δ → 0 as δ ↓0. For (ii′) of Theorem 13.5, notice that for 0≤r <s<t≤1, some constant C >0,

E{(Y_{n}(s)−Y_{n}(r))^{2}(Y_{n}(t)−Y_{n}(s))^{2}} = E\{(\frac{1}{\sqrt{n}}∑_{j=1}^{n}[1_{(r,s]}(η_{j})−(s−r)])^{2}(\frac{1}{\sqrt{n}}∑_{j=1}^{n}[1_{(s,t]}(η_{j})−(t−s)])^{2}\} = \frac{1}{n}E\{[1_{(r,s]}(η)−(s−r)]^{2}[1_{(s,t]}(η)−(t−s)]^{2}\}

+ \frac{n−1}{n}E\{[1_{(r,s]}(η)−(s−r)]^{2}\} E\{[1_{(s,t]}(η)−(t−s)]^{2}\}

+ 2\frac{n−1}{n}[E\{[1_{(r,s]}(η)−(s−r)][1_{(s,t]}(η)−(t−s)]\}]^{2} ≤ C(s−r)(t−s) ≤ C(t−r)^{2}. Theorem 14.3. For iid. [0, 1]-valued r.v.’s \{ξ_{j}\}_{j=1}^{∞}, \{Y_{n}(t)\}_{t∈[0,1]} ≡ \{\sqrt{n}(\frac{1}{n} ∑_{j=1}^{n} 1_{[0,t]}(ξ_{j})−F_{ξ}(t))}_{t∈[0,1]} ⇒ Y in P(D), where {Y (t)}_{t∈[0,1]} is zero-mean Gaussian with E{Y (s)Y (t)} = F_{ξ}(s∧t)−F_{ξ}(s)F_{ξ}(t) for s, t∈[0, 1]. Proof. F^{←}(s)≡inf{t∈[0, 1] : s≤F_{ξ}(t)} has F^{←}(s)≤t iff. s≤F_{ξ}(t), so F^{←}(η)=_{d} ξ, and {Y_{n}^{η}(F_{ξ}(t))}_{t∈[0,1]} =_{d} {Y_{n}(t)}_{t∈[0,1]}. Since D ∋ x → h(x) ≡ x∘F_{ξ} ∈ D continuous on C, general result by Mapping Theorem (P{Y^{η} ∈C}=1). □

23. Euclidian Chf. [Mainly after Feller Vol. II].

ℝ^{k}-valued r.v. X has chf. ϕ_{X}(t) = E{e^{i⟨t,X⟩}} = E{e^{i ∑_{j=1}^{k}t_{j}X_{j}}}, t∈ℝ^{k}. Theorem. (Unicity) For F_{X}-continuity (−∞, x]⊆ℝ^{k},

F_{X}(x) = lim_{ε↓0} _{∫z≤x}_{∫y∈ℝ^{k}}∏_{j=1}^{k}e^{−iy_{j}z_{j}} \frac{1}{\sqrt{2π}ε}f_{N(0,ε^{−2})}(y_{j})ϕ_{X}(y) dy dz. Proof. ⇐ By weak convergence in P(ℝ^{k}), as ε↓0,

∫z≤xdF_{X}(z) ← _{∫z≤x}_{∫t∈ℝ^{k}}∏_{j=1}^{k}f_{N(0,ε^{2})}(z_{j} −t_{j}) dF_{X}(t) dz
= _{∫z≤x}_{∫t∈ℝ^{k}}∏_{j=1}^{k} \frac{1}{\sqrt{2π}ε}ϕ_{N(0,ε^{−2})}(t_{j} −z_{j}) dF_{X}(t) dz

= _{∫z≤x}_{∫y∈ℝ^{k}}∏_{j=1}^{k} \frac{1}{\sqrt{2π}ε}f_{N(0,ε^{−2})}(y_{j})(_{∫t∈ℝ^{k}}e^{iy_{j}(t_{j}−z_{j})} dF_{X}(t)) dydz. □

Corollary. (Independence) ℝ^{k}-valued r.v.’s X,Y independent iff. ϕ_{X,Y} =ϕ_{X}ϕ_{Y}. Theorem. (Inversion) ℝ^{k}-valued r.v. X with integrable chf. ϕ_{X} is absolutely continuous with bounded continuous density f_{X}(x) = \frac{1}{(2π)^{k}} _{∫t∈ℝ^{k}}e^{−i⟨t,x⟩}ϕ_{X}(t) dt. Proof. By integrability and inspection of previous proof, as ε↓0, the density

∫t∈ℝ^{k}∏_{j=1}^{k}f_{N(0,ε^{2})}(z_{j} −t_{j}) dF_{X}(t) = _{∫y∈ℝ^{k}}∏_{j=1}^{k}e^{−iy_{j}z_{j}} \frac{1}{2π} ϕ_{N(0,ε^{2})}(y_{j})ϕ_{X}(y) dy

→ _{∫y∈ℝ^{k}}e^{−i⟨y,z⟩} \frac{1}{(2π)^{k}} ϕ_{X}(y) dy ≡ f(z),

with f bounded continuous. By bounded convergence, for bounded Borel I ⊆ℝ^{k}, ∫I dF_{X} ← _{∫z∈I}_{∫t∈ℝ^{k}}∏_{j=1}^{k}f_{N(0,ε^{2})}(z_{j} −t_{j}) dF_{X}(t) dz → _{∫z∈I} f(z) dz as ε↓0. □

Theorem. (Lévy-Cramér, Convergence) X_{n} ⇒ X in P(ℝ^{k}), some X, iff. ϕ_{X_{n}} →ϕ, some ϕ that is continuous at zero. If so ϕ_{X} = ϕ. Proof. ⇐ By Unicity Theorem, enough show tight. By continuity, as K →∞,

lim sup_{n} P\{|(X_{n})_{j}|≥2K\} ≤ lim sup_{n} 2_{∫|x|≥2K}(1− \frac{sin(x/K)}{x/K}) dF_{(X_{n})_{j}}(x)
≤ lim sup_{n} 2_{∫ℝ}(1− \frac{sin(x/K)}{x/K}) dF_{(X_{n})_{j}}(x)

= lim sup_{n} K_{∫x∈ℝ}_{∫|t_{j}|≤1/K}(1−e^{it_{j}x_{j}}) dt_{j} dF_{(X_{n})_{j}}(x)

= K_{∫|t_{j}|≤1/K}(1−ϕ(…, 0, t_{j}, 0, …)) dt_{j} → 0. □

Corollary. (Cramér-Wold Device) X_{n} ⇒X in P(ℝ^{k}), some X, iff. ⟨t, X_{n}⟩ ⇒ in P(ℝ) for t∈ℝ^{n}. If so ϕ_{X}(t)= lim_{n} ϕ_{⟨t,X_{n}⟩}(1). Theorem. (Bochner) ϕ: ℝ → ℂ is chf. for some real r.v. iff. continuous, nonnegative definite i.e., ∑_{k,ℓ=1}^{n} a_{k}\overline{a_{ℓ}} ϕ(t_{k}−t_{ℓ})≥0, t_{k} ∈ℝ, a_{k} ∈ℂ, n∈ℕ], with ϕ(0)=1. Proof. Continuity and ϕ(0)=1 necessary, so enough show continuous ϕ with ϕ(0) =1 chf. iff. bounded and ∫_{ℝ^{2}} e^{ix(s−t)}ϕ(s−t) dF(s)dF(t) ≥ 0, x∈ℝ, all F, since â_{k} = a_{k}/(∑_{k=1}^{n} |a_{k}|) = p_{k}e^{ixt_{k}}, p_{k} ≥0, dF =∑_{k=1}^{n} p_{k}δ_{t_{k}} in this gives non-negative definite, and non-negative definite gives aϕ(t)+\overline{a}ϕ(−t)+1+|a|^{2} ≥ 0, so \overline{ϕ(t)}=ϕ(−t) (a =1, i) and |ϕ(t)|≤1 [a=−\overline{ϕ(t)}], so integral exists, non-negative for dF =∑_{k=1}^{n} p_{k} δ_{t_{k}} as above, and in general by approximate with such (cf. proof of Theorem 6.8).

For r.v. ξ ∼ϕ and iid. r.v.’s η, η̂ ∼ F, |ϕ_{η}(y)|^{2} = ϕ_{η−η̂}(y) = ∫_{ℝ^{2}} e^{iy(s−t)}dF(s)dF (t) ≥ 0, so 0 ≤ ∫_{ℝ^{3}} e^{iy(s−t)} dF(s)dF(t)dF_{ξ+x}(y) = ∫_{ℝ^{2}} e^{ix(s−t)}ϕ(s−t) dF(s)dF(t).

By Continuity Theorem, bounded continuous ϕ with ϕ(0)=1 chf. if ϕ ϕ_{N(0,ε^{−2})} chf., ε>0. Notice f(x) ≡ \frac{1}{2π} ∫_{ℝ} e^{−ixy}ϕ(y)ϕ_{N(0,ε^{−2})}(y) dy ≥ 0, x∈ℝ, since equal \frac{ε}{\sqrt{2π}}_{∫ℝ}e^{−ixy}ϕ(y) dF_{N(0,ε^{2})}(y) = \frac{ε}{\sqrt{2π}}_{∫ℝ^{2}}e^{−ix(s−t)}ϕ(s−t) dF_{N(0,ε^{2}/2)}(s)dF_{N(0,ε^{2}/2)}(t), and bounded continuous. f is integrable, since by Inversion Theorem,

∫ℝϕ_{N(t,δ^{2})}(x)f(x) dx = _{∫ℝ}f_{N(t,δ^{2})}(y)ϕ(y)ϕ_{N(0,ε^{−2})}(y) dy ≤ ∥ϕ∥ < ∞, so ∫_{ℝ} f ≤ lim inf_{δ↓0} ∫_{ℝ} ϕ_{N(0,δ^{2})}f < ∥ϕ∥ by Fatou’s Lemma. Now Dominated Convergence and N(t, δ^{2})⇒t in P(ℝ) as δ ↓0 show ϕ ϕ_{N(0,ε^{−2})} chf., since

∫ℝe^{ixt}f(x) dx = lim_{δ↓0} E{(ϕϕ_{N(0,ε^{−2})})(N(t, δ^{2}))} = (ϕϕ_{N(0,ε^{−2})})(t). □

24. Basic Theory for Processes. Definition. For set T ≠∅, \underline{stochastic process} \{X(t)\}_{t∈T} with parameter in T is family of r.v.’s defined on common probability space, that we assume completed. Definition. The distributions \{\{F_{X(t_{1}),...,X(t_{k})}\}_{t∈T^{k}}\}_{k∈ℕ} \underline{fidi’s} of \{X(t)\}_{t∈T}. Theorem. (Kolmogorov) For family of distributions {{F_{t}:ℝ^{k} →[0, 1]}_{t∈T^{k}}}_{k∈ℕ} there exists process {X(t)}_{t∈T} with these fidi’s iff. following two hold

(i) F_{...,t_{i−1},t_{j},t_{i+1},...,t_{j−1},t_{i},t_{j+1},...}(…, x_{i−1}, x_{j}, x_{i+1}, …, x_{j−1}, x_{i}, x_{j+1}, …) = F_{t}(x); (ii) lim_{x_{k+1}→∞} F_{t,t_{k+1}}(x, x_{k+1}) = F_{t}(x). Proof. ⇐ Enough find probability P on smallest (so called cylinder) σ-algebra of ℝ^{T} that has P π_{t}^{−1} well-defined with valued F_{t}, t∈T^{k}, since for X(ω; t)=ω(t), ω ∈ ℝ^{T}, t ∈ T, P (⋂_{i=1}^{k}{X(t_{i}) ∈ H_{i}}) = P (⋂_{i=1}^{k}{ω(t_{i}) ∈ H_{i}}) = P π_{t_{1}...t_{k}}^{−1}(H_{1} ×… ×H_{k}). Define finitely additive P on algebra 𝔄 of finite unions of π_{t_{1}...t_{k}}^{−1}(H_{1}×… ×H_{k}) by P π_{t}^{−1} =F_{t}. Enough show P σ-additive, since it then extends to σ(𝔄). Enough show P tight (MT 17). Holds by tightness of fidi’s. □ Definition. Process \{X(t)\}_{t∈T} on topological space T \underline{separable} if there is countable dense \underline{separant} D ⊆T and P-null N, such that to ω ∈Ω\N and t∈T, there is {t_{n}}_{n=1}^{∞} ⊆D with t_{n} →_{T} t and X(ω; t_{n})→X(ω; t).

Separable processes are of utmost importance since their important properties are determined by fidi’s, and they possess “all nice properties” allowed by the fidi’s. Example. If separable {X(t)}_{t∈[0,1]} has same fidi’s as C-valued r.v. Y, then

P\{X not uniformly continuous\} = P\{⋃_{k=1}^{∞}⋂_{ℓ=1}^{∞}⋃_{s,t∈[0,1]:|s−t|<\frac{1}{ℓ}}\{|X(s)−X(t)|> \frac{1}{k}\}\}

= P\{⋃_{k=1}^{∞}⋂_{ℓ=1}^{∞}⋃_{s,t∈D:|s−t|<\frac{1}{ℓ}}\{|X(s)−X(t)|> \frac{1}{k}\}\} = P{X not uniformly continuous on D} = 0,

since lim_{L↑∞} lim_{N↑∞} P\{⋂_{ℓ=1}^{L}⋃_{i,j∈\{1,...,N\}:|d_{i}−d_{j}|< \frac{1}{ℓ}}\{|X(d_{i})−X(d_{j})|> \frac{1}{k} \}\} = 0, since determined by fidi’s and Y uniformly continuous, so 1 for Y. (On non-continu-ity null-set we may change X to e.g., 0 without affecting separability or fidi’s.) Example. Process {X(t)}_{t∈ℝ} with iid. N(0, 1)-values is separable. Theorem. If separable process {X(t)}_{t∈T} on second order countable space T is P-continuous [i.e., X(s)→_{P} X(t) if s→_{T} t], each dense in T is separant. Proof. Pick base {G_{n}}_{n=1}^{∞}, separant D ⊆T and countable dense S ⊆T. To t∈D, ŝ_{n}(t) ∈ B_{n} ≡ ⋂_{k∈{1,...,n}:G_{k}∋t} G_{k} some {ŝ_{n}(t)}_{n=1}^{∞} ⊆S, so ŝ_{n}(t)→t and X(ŝ_{n}(t)) →_{P} X(t). There is P-null Ω_{t} and {ŝ′_{n}(t)}_{n=1}^{∞} ⊆{ŝ_{n}(t)}_{n=1}^{∞} with X(ŝ′_{n}(t))→X(t), ω ∉ Ω_{t}. Take |X(ŝ′_{n}(t))−X(t)| ≤ \frac{1}{n}. There is P-null Ω̂ such that to t ∈ T and ω ∉ Ω̂ there is \{t_{n}^{ω}(t)\}_{n=1}^{∞} ⊆ D with t_{n}^{ω}(t) → t and |X(t_{n}^{ω}(t))−X(t)| ≤ \frac{1}{n}. Put N ≡ Ω̂∪⋃_{t∈D} Ω_{t}. Let s_{n}^{ω}(t)≡ŝ′_{n}(t_{n}^{ω}(t)). Then X(s_{n}^{ω}(t))→X(t), t∈T, ω ∉N, since

|X(s_{n}^{ω}(t))−X(t)| ≤ |X(ŝ′_{n}(t_{n}^{ω}(t)))−X(t_{n}^{ω}(t))| + |X(t_{n}^{ω}(t))−X(t)| ≤ \frac{2}{n}. For open T ⊇G∋t, t∈G_{n_{0}} ⊆G, some n_{0} ∈ℕ, so t_{n}^{ω}(t)∈G_{n_{0}}, n≥n_{1}, some n_{1} ∈ℕ, giving s_{n}^{ω}(t) = ŝ′_{n}(t_{n}^{ω}(t)) ∈ B_{n}, n≥n_{0}∨n_{1}, by definition of B_{n}, so s_{n}^{ω}(t)→t. □ Example (MT 51). For separable {X(t)}_{t∈[0,1]} with separant D ∈[0, 1], P{X(t) ∈ G some t ∈ [0, 1]} = P{X(t) ∈ G some t ∈ D} for open G ⊆ ℝ. For X P-con-tinuous, this holds for any dense D ⊆[0, 1]. Definition. Process \{X(t)\}_{t∈[0,1]} \underline{bounded} if lim_{M→∞} P\{∥X∥_{C} >M\} = 0. Example (continued). P{∥X∥_{C} >M} = P{X(t)>M some t∈D} = lim_{n→∞} P {max_{1≤i≤n} X(d_{i})>M}. X bounded iff. some process with same fidi’s bounded. Definition. Processes {X(t)}_{t∈T} and {Y (t)}_{t∈T} on common probability space \underline{indistinguishable} if P\{X(t)=Y (t)\} = 1 for t∈T. (Cf. MT 51.) Example. Indistinguishable processes have same fidi’s.

Right- and leftcontinuous version of a given Poisson process indistinguishable. If continuously distributed r.v. ξ independent of process {X(t)}_{t∈ℝ}, then Y (t)=

X(t) for t≠ξ, Y (t)=X(t)+1 for t=ξ, is indistinguishable from X. Theorem. (Doob) Process {X(t)}_{t∈T} on second order countable topological space T has separable indistinguishable version {Y (t)}_{t∈T}. Proof. Pick countable basis 𝔊 in T, with 𝔊_{t} ={B_{n}^{t}}_{n=1}^{∞} sets of 𝔊 containing t.

Enough show for each closed F ⊆ℝ and open G⊆T (see also MT 52)

(⋆)  {Y (t)∈F for t∈G∩D} \ {Y (t)∈F for t∈G} ⊆ N:

Take ω ∉ N and t ∈ T. By (⋆) with F = \{y ∈ ℝ : |y−Y (t)| ≥ \frac{1}{n}\}, there is t_{n} ∈ ⋂_{i=1}^{n} B_{i}^{t}∩ D with |Y (t_{n})−Y (t)|< \frac{1}{n}. Now Y (t_{n})→Y (t) and t_{n} →t.

Let A_{0} be finite unions of closed intervals with rational or infinite endpoints, and A intersections of such, so A is the closed in ℝ. There is countable D ⊆T with

(⋆⋆)  _{⋃A∈A}({X(s)∈A for s∈D} \ {X(t)∈A}) P-null, for t∈T:

Notice D_{A} ≡ arginf{P(⋂_{s∈D}{X(s) ∈ A}), D ⊆ T countable} = ⋃_{n=1}^{∞} D_{n}, where P\{⋂_{s∈D_{n}}\{X(s)∈A\}\} ≤ inf+\frac{1}{n}, A∈A_{0}, so Ω_{t}(A) ≡ ⋂_{s∈D_{A}}\{X(s)∈A, X(t)∉A\} P-null, since P{⋂_{s∈D_{A}}{X(s) ∈ A}} = P{⋂_{s∈D_{A}∪{t}}{X(s) ∈ A}} by minimality of D_{A}. Put D ≡⋃_{A∈A_{0}} D_{A}, Ω_{t} ≡⋃_{A∈A_{0}} Ω_{t}(A). For B =⋂_{k} A_{k} ∈A ⋂s∈D{X(s)∈B, X(t)∉B} = _{⋃k}_{⋂s∈D}{X(s)∈B, X(t)∉A_{k}} ⊆ _{⋃k}Ω_{t}(A_{k}) ⊆ Ω_{t}. By (⋆⋆) on {X(t)}_{t∈B}, B ∈𝔊, for some countable D^{B} ⊆B, to t∈B, there is P-

null Ω_{t}^{B} with ⋂_{s∈D^{B}}{X(s) ∈ F, X(t) ∈ F } ⊆ Ω_{t}^{B}, closed F ⊆ ℝ. Let D ≡ ⋃_{B∈𝔊} D^{B}, Ω_{t} ≡∅ if t∈D, Ω_{t} ≡⋃_{B∈𝔊_{t}} Ω_{t}^{B} if t∉D, and N ≡⋃_{t∈D}Ω_{t}. Put Y (t)≡X(t) if ω ∉Ω_{t}, Y (t) ≡ lim sup_{D∋s→t} X(s) if ω ∈Ω_{t}, so X, Y indistinguishable.

By (⋆), enough show Δ_{t}(B) ≡ ⋂_{s∈B∩D}{Y (s)∈F, Y (t)∉F } ⊆ N, t∈G⊇B ∈𝔊_{t}. Let Y (t)∉F and ω ∉Ω_{t}. Since X(t) = Y (t) ∉ F and ω ∉Ω_{t}^{B}, X(s)∉F, some

s∈B∩D =D^{B}. So Y (s)=X(s)∉F, ω ∉Ω_{s}. Thus Δ_{t}(B)∩Ω_{t}^{c} ⊆ Ω_{s} ⊆ N.

Let Y (t)∉F and ω ∈Ω_{t}. If Y (s)∈F, s∈D^{B}, then Y (t) = lim sup_{D∋s→t} X(s) = lim sup_{D∋s→t} Y (s) ∈ F, ω ∉N, by F closed and Ω_{s} ∈N. So Δ_{t}(B)∩Ω_{t} ⊆N. □ Theorem. (Kolmogorov) Separable process {X(t)}_{t∈[0,1]} fits into C if P{|X(s)−X(t)|>x} ≤ c|t−s|^{1+γ}/x^{β} for s,t∈[0,1], x>0, some constants c,β,γ>0.

Proof. By Example, enough show there is C-valued r.v. Y with same fidi’s. Sample X at 0, \frac{1}{n}, …, \frac{n}{n} and interpolate; X_{n}(t) ≡ X(⌊nt⌋/n)+(nt−⌊nt⌋)[X(⌈nt⌉/n)− X(⌊nt⌋/n)]. Fidi’s of X_{n} ⇒ those of X, since by condition in theorem, |X_{n}(t)−X(t)| ≤ (1−nt+⌊nt⌋)|X(⌊nt⌋/n)−X(t)|+(nt−⌊nt⌋)|X(⌈nt⌉/n)−X(t)| →_{P} 0. Inequality for X gives it for X_{n} with other c, so tight by Criterion. □

25. Lévy Processes. Definition. \{X(t)\}_{t∈[0,1]} \underline{Lévy process} if P-continuous, X(0)=0 and independent stationary increments. R.v. ξ \underline{ID} if ξ =_{d} ∑_{i=1}^{n} η_{i} some iid. r.v.’s \{η_{i}\}_{i=1}^{n}, n∈ℕ. Theorem. (Lévy-Khintchine) ξ ID iff. for some unique constants μ∈ℝ, σ^{2} ≥ 0, and Borel measure ν on ℝ with ∫_{ℝ} 1∧|x|^{2} dν(x)<∞, denoted ξ ∼ \underline{ID(μ, σ^{2}, ν)}, ϕ_{ξ}(x) = E\{e^{ixξ}\} = exp\{iμx +_{∫ℝ}(e^{ixy} −1−ix\frac{y}{1∨|y|}) dν(y) − \frac{1}{2}σ^{2}x^{2}\} for x∈ℝ.

Remark. \frac{y}{1∨|y|} can be replaced with any bounded measurable y+O(y^{2}) as y →0.

Fidi’s of Lévy process {X(t)}_{t∈[0,1]} are determined by univariate marginal distributions F_{X(t)}, and even by F_{X(1)}, X(t) =_{d} X(1)^{⋆t}, where ϕ_{ξ^{⋆t}}(x) = (ϕ_{ξ}(x))^{t}.

By Kolmogorov’s Theorem, there is Lévy process {Y (t)}_{t∈[0,1]} with Y (t)=_{d} ID (m, σ^{2}, ν)^{⋆t}. [There is Lévy process {X(t)}_{t∈[0,1]} in C with X(t) =_{d} ID(m, σ^{2}, ν)^{⋆t} iff. ν =0. Here ⇒ is difficult, while ⇐ is MT 56.] Lemma. For Lévy process {X(t)}_{t∈[0,1]} with E{X(1)^{2}}<∞, E{X(t)^{2}} = [E{X (1)}]^{2}t^{2} + Var{X(1)}t, t∈[0, 1].

Proof. Recall E{X(1)^{2}}<∞ iff. ϕ_{X(1)}(x) two times (continuously) differentiable at zero, in which case E{X(1)} = iϕ′_{X(1)}(0) and E{X(1)^{2}} = −ϕ′′_{X(1)}(0), so ϕ_{X(t)}(x)=ϕ_{X(1)}(x)^{t} gives E{X(1)^{2}}<∞ iff. E{X(t)^{2}}<∞, t∈[0, 1], and

E\{X(t)^{2}\} = −\frac{d^{2}}{dx^{2}} ϕ_{X(1)}(x)^{t}|_{x=0} = −t(t−1)ϕ′_{X(1)}(0)^{2}ϕ_{X(1)}(0)^{t−2} − tϕ′′_{X(1)}(0)ϕ_{X(1)}(0)^{t−1}
= t(t−1)[E{X(1)}]^{2} + tE{X(1)^{2}} = claim. □

Theorem. To ξ ∼ ID(m, σ^{2}, ν) there is D-valued r.v. {X(t)}_{t∈[0,1]} that is Lévy process with X(t) =_{d} ξ^{⋆t}. Proof for E{ξ^{2}} < ∞. X_{n}(t) ≡ ∑_{i=1}^{⌊nt⌋} η_{i}^{(n)} has ϕ_{X_{n}(t)−X_{n}(s)}(x) → ϕ_{ξ^{⋆(t−s)}}(x), so X_{n}(t)−X_{n}(s) ⇒ ξ^{⋆(t−s)} in P(ℝ), 0 ≤ s < t ≤ 1. Now (X_{n}(t_{1}), …, X(t_{k})) ⇒ in P(ℝ^{k}), to right limit, 0≤t_{1} <…<t_{k} ≤1, since (X_{n}(t_{1}), X(t_{2})−X(t_{1}), …, X(t_{k})− X(t_{k−1})) does, by X_{n}(t) ⇒ and independent increments. (i′) of Theorem 13.5 holds, since lim_{δ↓0} P{|ξ^{⋆δ}|>ε} = 0 for ε>0, by MT 56. For (ii′), notice that, by independent increments and Lemma, for 0≤r ≤s≤t, with Y as above,

E{|X_{n}(s)−X_{n}(r)|^{4}∧|X_{n}(t)−X_{n}(s)|^{4}} ≤ E{|X_{n}(s)−X_{n}(r)|^{2}}E{|X_{n}(t)−X_{n}(s)|^{2}}
= E\{[Y(\frac{⌊ns⌋}{n})−Y(\frac{(⌊nr⌋+1)∧⌊ns⌋}{n})]^{2}\}

× E\{[Y(\frac{⌊nt⌋}{n})−Y(\frac{(⌊ns⌋+1)∧⌊nt⌋}{n})]^{2}\}

≤ K (s−r)(t−s) ≤ K (t−r)^{2}.

This holds also for t−r < \frac{1}{n}, since then |X_{n}(s)−X_{n}(r)|^{4}∧|X_{n}(t)−X_{n}(s)|^{4} = 0. □ Theorem. ID(μ_{n}, σ_{n}^{2}, ν_{n}) ∋ ξ_{n} ⇒ ξ in P(ℝ) iff. ξ ∈ID(μ, σ^{2}, ν) and the following (i) lim_{n} μ_{n} = μ; (ii) lim_{n} ν_{n}(A) = ν(A) for ν-continuity Borel A⊆ℝ with 0∉clos(A); (iii) lim_{ε↓0} lim sup_{n}|∫_{|y|≤ε} x^{2}y^{2} dν_{n}(y) + (σ_{n}^{2}−σ^{2})x^{2}| = 0. Proof of ⇐ . By Lévy-Khintchine, as n→∞ and ε↓0 (in that order),

ϕ_{ξ_{n}}(x) = exp\{iμ_{n}x +_{∫|y|>ε}(e^{ixy} −1−ix\frac{y}{1∨|y|}) dν_{n}(y) − \frac{1}{2}σ^{2}x^{2}\}

× exp\{_{∫|y|≤ε}(e^{ixy} −1−ixy) dν_{n}(y) − \frac{1}{2}(σ_{n}^{2}−σ^{2})x^{2}\} → ϕ_{ID(μ,σ^{2},ν)}(x). □

Theorem. D-valued Lévy processes X_{n}⇒X in P(D) iff. X_{n}(1)⇒X(1) in P(ℝ). Proof. ⇒ By Theorem 13.1 (1∈T_{X}). ⇐ for lim sup_{n}E{X_{n}(1)^{2}}<∞. X_{n}(t)=_{d} X_{n}(1)^{⋆t} ⇒ X(1)^{⋆t} =_{d} X(t), t∈[0, 1], gives (X_{n}(t_{1}), …, X(t_{k}))⇒ in P(ℝ^{k}), by independent stationary increments. (i′) of Theorem 13.5 holds, since lim_{δ} P{|X(1)^{⋆δ}| >ε} = 0, while Lemma gives (ii′), since for 0≤r ≤s≤t, E{|X_{n}(s)−X_{n}(r)|^{4}∧|X_{n}(t)−X_{n}(s)|^{4}} ≤ (E{X_{n}(1)}^{2}(t−r)^{2}+Var{X_{n}(1)}(t−r))^{2}. □

Appendix. Micro Theorems (\underline{MT}) for Exercise (1=easy, 5=difficult). MT 1 (1.5). For D countable dense in T , shew {⋂_{i=1}^{k}{x ∈ C(T ) : x(t_{i}) ∈ H_{i}}, H_{i} ⊆T open, t_{i} ∈D, k ∈ℕ} is separating for P(C(T )). MT 2 (1.5). In Theorems 3.5-3.6, it is not desirable to require that each E{|X_{n}|} < ∞, since convergence of E{|X_{n}|} do not involve all X_{n}. Give characterization of when E{|X_{n}|} → E{|X|} < ∞, using a version of uniform integrability, so that the requirement that each E{|X_{n}|}<∞ is not needed (at all). MT 3 (2.5). A version of Scheffe’s Theorem: For densities f, f_{n}:S →ℝ wrt. measure μ, with f_{n} →f a.e. (μ), and g_{n}:S →ℝ uniformly bounded measurable, shew ∫_{S} g_{n}f_{n} dμ ≈ ∫_{S} g_{n}f dμ as n→∞, in sense that should be specified. Shew lim inf_{n} ∫_{S} g_{n}f_{n} dμ ≥ ∫_{S} lim inf_{n} g_{n}f dμ and lim sup_{n} ∫_{S} g_{n}f_{n} dμ ≤ ∫_{S} lim sup_{n} g_{n}f dμ. MT 4 (3). Another: For X_{n} r.v.’s with X_{n} →X ∈𝕃^{1}(Ω) a.s., lim sup_{n} E{X_{n}^{+}}≤ E{X^{+}} and lim sup_{n} E{X_{n}^{−}}≤E{X^{−}}, shew X_{n} →_{𝕃^{1}(Ω)} X. Relation to Scheffé? MT 5 (3). For X_{n} →_{P} X, shew X_{n} →_{𝕃^{1}(Ω)} X iff. {X_{n}}_{n=1}^{∞} uniformly integrable. MT 6 (1.5). For ℤ-valued r.v.’s X_{n}, X, shew P{X_{n} =k} → P{X =k} for k ∈ℤ implies X_{n} ⇒X in P(ℝ). MT 7 (1-5). Discuss first countability of P(S) (i.e., when the open neighborhoods of each P ∈P(S) has a countable basis). MT 8 (3). State general principle of correspondance between weak convergence, in the sense of convergence wrt. imposed weak topologies, and pointwise convergence. MT 9 (4). For S-valued r.v.’s X_{n}, S separable complete, shew (X_{m},X_{n})⇒(X̂, X̂) in P(S×S), some S-valued r.v. X̂, iff. X_{n} →_{P} X̃ in S, some S-valued r.v. X̃.

MT 10 (3). Billingsley Problem 5.10.

MT 11 (1). Shew (Fε̂)ε̃⊆F_{ε̂+ε̃} and F ⊆(((Fε̂)^{c})_{ε})^{c}, ε̂>ε. Exemplify strict inclusion. MT 12 (1). For separable S′, S′′ and P ∈P(S′×S′′), shew opens in S′×S′′ countable unions of P -continuity S′_{r′}(x′)×S′′_{r′′}(x′′). How is this used in weak convergence? MT 13 (1). Shew convergence determining class is separating. Shew A_{P} ⊆ 𝔖 separating for P ∈P(S) if P_{n}(A)→P (A) for A∈A_{P} implies P_{n} ⇒P in P(S). MT 14 (1.5). Shew lim_{ε↓0} P (A_{ε}) = P (A_{0}) for P ∈P(S) and (any!) A⊆S.

MT 15 (2). For P ∈P(S) and S separable, shew opens in S are countable unions of P -continuity open balls, as well as of P -continuity closed balls. Deduce that finite intersections of open (closed) balls are convergence determining. MT 16 (2). For tight P ∈P(S), shew P(A) = sup{P(K) : A⊇K compact}, A∈𝔖.

MT 17 (3.5). Billingsley Problem 1.5.

MT 18 (1.5). Shew ℂ_{B}(S) Banach space. MT 19 (2). A real \underline{Banach space} is normed linear space (B, ∥·∥) over ℝ, that is complete (in metric from norm). Shew linear L: B → ℝ continuous iff. continuous at 0 iff. bounded i.e., ∥L∥^{⋆} ≡ sup\{|Lx| : ∥x∥ ≤ 1\} < ∞. The \underline{dual B^{⋆}} ≡ {linear continuous L:B →ℝ}, with norm ∥·∥^{⋆}. Shew (B^{⋆}, ∥·∥^{⋆}) is Banach space. MT 20 (2). Shew BL(S) Banach space. MT 21 (1.5). Starting with BL(S), explain notation ∥P ∥_{BL(S)}^{⋆} for P ∈P(S). MT 22 (4). Topological space T is metrizable iff. T_{1}, \underline{regular} [i.e., for x∈T and open G∋x, there is closed F ⊆G with x∈INT(F)], with \underline{σ-locally finite base} [i.e., open base O = ⋃_{n=1}^{∞} O_{n} such that, given n ∈ ℕ, for x ∈ T there is open G ∋ x with #{H ∈ O_{n} : G∩H ≠ ∅} < ∞]. Shew (0)_{1}^{⋆} is metrizable if B separable. (It is iff .) [Hint: Take dense {x_{i}}_{i=1}^{∞} ⊆B and O = ⋃_{i,j=1}^{∞} O_{i,j}, where O_{i,j} = ⋃_{k=1}^{K_{i,j}} \{f ∈(0)_{1}^{⋆} : (f −f_{k}^{i,j})(x_{i})< \frac{1}{j}\} covers (0)_{1}^{⋆}, some \{f_{k}^{i,j}\}_{k=1}^{K_{i,j}} ⊆(0)_{1}^{⋆}, by Alaoglu.] MT 23 (1). Shew T_{1} can be changed to “Hausdorff” in MT 24. Why isn’t it? MT 24 (4). Shew C(T ) separable. [Hint: By Ascoli, {f ∈BL(T ) : ∥f∥_{BL(T )} ≤n}, and BL(T ), are C(T )-separable, so clos(BL(T ))_{C(T )} = C(T ) by Stone-Weierstrass.] MT 25 (2). Shew C[0, 1] dense in 𝕃^{1}[0, 1]. Conclude 𝕃^{1}[0, 1] separable. Discuss 𝕃^{p}[0, 1] for general p∈(0, ∞), and more general T in C(T ) than T =[0, 1]. MT 26 (3). Shew w^{⋆}-clos({μ ∈ M[0,1] : μ = f dx, f ∈ C[0,1]}) = M[0,1]. Argue clos({f dx: f ∈C[0,1]}) ⊊ M[0,1]. MT 27 (5). For Π⊆P(S) tight, shew (Π relatively compact iff.) clos(Π) sequentially compact. For P(S) metrizable (i.e., S separable) shew Π⊆P(S) relatively compact iff. clos(Π) sequentially compact. MT 28 (3). Shew P_{n} ⇒P in P(S) if {P_{n}}_{n=1}^{∞} is relatively compact and P_{n}(A)→ P (A) for A in class that contains a separating class of continuity sets of each Q∈P(S). Shew it is iff. (and not only if) for S separable.

MT 29 (3). Billingsley Problem 6.5. [Hint: See Billingsley on ball σ-field.]
MT 30 (3). Billingsley Problem 6.10.

MT 31 (4). For S separable complete shew (P(S), ∥·∥_{BL}^{⋆}) complete. [Hint: Take compacts in tightness as continuity sets of finite Π ⊆ P(S), approximate by f ∈ BL(S), use Cauchy and Prohorov.] MT 32 (1.5). Shew η(x)=∥x∥ some η ∈(0)_{1}^{⋆} for x∈B. [Hint: Hahn-Banach.] MT 33 (3). Shew three corollaries after Theorem 7.3. MT 34 (3.5). Shew B-valued r.v. X has Bochner mean iff. E{∥X∥} < ∞, and in that case X has Pettis mean E{X} and ∥E{X}∥ ≤ E{∥X∥}. Shew B ⊆ B^{⋆⋆}. If B =B^{⋆⋆}, B is \underline{reflexive}. Discuss Pettis-means for B reflexive. MT 35 (2.5). Shew B-valued r.v. X has covariance if E{(η(X))^{2}}<∞, η ∈B^{⋆}. [Hint: Closed Graph Theorem]. MT 36 (3). Discuss correspondance between r.v. X ∼N_{B}(E, Q) and pair E ∈B^{⋆⋆}, Q:B^{⋆} →B^{⋆⋆}, with Q symmetric non-negative. MT 37 (2). Shew proof of Donsker’s Theorem not uses but proves existence of W. MT 38 (1). Shew version of criterion in Section 7 with requirements on moments. MT 39 (2). Why doesn’t criterion in Section 7 give tightness in Donsker’s Theorem? MT 40 (3). Motivate different parts of Lemma 12.1, except third inequality. MT 41 (1). Shew x=y in D if same on dense [0, 1]⊇T ∋1. Shew x_{n} →_{D} x ⇒ x_{n} →_{a.e.} x MT 42 (1). Shew Cauchy {x_{n}}_{n=1}^{∞} has subsequence {y_{n}}_{n=1}^{∞} with d(y_{n}, y_{m}) ≤ 2^{−n} for m≥n. Shew {x_{n}}_{n=1}^{∞} converges if {y_{n}}_{n=1}^{∞} does. MT 43 (3). Shew Π⊆E relatively compact iff. totally bounded. Shew Ascoli’s Theorem Π⊆C(T ) bounded and \underline{equicontinuous} [i.e., lim_{δ↓0} sup_{x∈Π} w_{x}(δ) = 0] iff. clos (Π) compact. Can bounded be replaced with sup_{x∈Π} |x(t)| < ∞ for a single t∈T ? MT 44 (4). Shew x_{n}, x∈D with ∫_{0}^{1} x_{n}f → ∫_{0}^{1} xf, f ∈C, ⇏ d_{1}(x_{n}, x)→0. MT 45 (2.5). Shew two corollaries in Section 13. MT 46 (1). Shew Independence in Section 23, and Cramér-Wold Device.

MT 47 (1). For non-negative definite r : T × T → ℝ, shew there is Gaussian process on T with this covariance. MT 48 (1). Shew separable Wiener process continuous wp. 1. MT 49 (1). Explain processes with same fidi’d need not be indistinguishable. MT 50 (1). Shew P-continuity need not imply separability. MT 51 (2). For separable {X(t)}_{t∈[0,1]}, separant D, open G⊆ℝ, shew ⋃_{t∈[0,1]} {X(t)∈G} = ⋃_{t∈D}{X(t)∈G} except on P-null. How about general G⊆ℝ? With P-continuity, shew lim_{n→∞} P\{sup_{t∈[0,...,2^{n}]} X(\frac{t}{2^{n}})>x\} = P\{sup_{t∈[0,1]} X(t)>x\}. MT 52 (4). Shew (⋆) is also necessary for separability in Doob’s Theorem. MT 53 (4). Give missing details of proof of Kolmogorov’s Theorem, Section 24. MT 54 (2). Shew Kolmogorov gives existence of Lévy process X =_{d} ID(μ, σ^{2}, ν)^{⋆·}. MT 55 (2.5). Shew ℝ^{k}-valued r.v. ξ symmetric iff. ϕ_{ξ} real. Shew ID(μ, σ^{2}, ν)^{⋆δ} ∼ ID(δμ, δσ^{2}, δν) and lim_{δ↓0} P{|ID(μ, σ^{2}, ν)^{⋆δ}|>ε} = 0 for ε>0. Shew ID(μ, σ^{2}, ν) symmetric iff. μ=0 and ν symmetric iff. ϕ_{ID(μ,σ^{2},ν)} strictly positive. MT 56 (1.5). Shew there is C-valued Lévy process X =_{d} ID(m, σ^{2}, ν)^{⋆} if ν =0.