Understanding the Central Limit Theorem Through Cauchy and Normal Distributions

Introduction: The Central Limit Theorem in 2026

The Central Limit Theorem (CLT) is a cornerstone of probability theory, explaining why many real-world phenomena follow a normal distribution. In 2026, as AI models process massive datasets and financial algorithms predict market trends, the CLT remains vital. This tutorial dives into key concepts from Math154 homework, focusing on the Cauchy and normal distributions, characteristic functions, entropy, and the de Moivre CLT. Whether you're analyzing gaming leaderboards or app user behavior, these principles underpin statistical inference.

Problem 9.1: Characteristic Function of Cauchy Distribution

a) Finding the Characteristic Function

The characteristic function of a standard Cauchy random variable X with density f(x) = 1/(π(1+x²)) is ϕ_X(t) = E[e^{itX}] = ∫_{-∞}^{∞} e^{itx}/(π(1+x²)) dx. Using residue calculus, we evaluate the integral for t>0 by closing the contour in the upper half-plane, picking the residue at x=i: Res(e^{itz}/(π(1+z²)), i) = e^{-t}/(2πi). Thus ϕ_X(t) = e^{-|t|}. For t<0, symmetry gives the same result.

b) Stability of Cauchy Under Averaging

Let X and Y be independent standard Cauchy. The characteristic function of (X+Y)/2 is ϕ_{(X+Y)/2}(t) = ϕ_X(t/2)ϕ_Y(t/2) = e^{-|t/2|}e^{-|t/2|} = e^{-|t|}, which is the characteristic function of a standard Cauchy. Hence (X+Y)/2 is also standard Cauchy. This stability contrasts with the normal distribution, where averaging reduces variance.

Problem 9.2: Entropy and Renormalized Moments

a) Differential Entropy of Cauchy

The differential entropy H(X) = -∫ f(x) log f(x) dx for Cauchy density f(x)=1/(π(1+x²)). Compute H = -∫_{-∞}^{∞} (1/(π(1+x²))) log(1/(π(1+x²))) dx = log(π) + (2/π)∫_0^∞ log(1+x²)/(1+x²) dx. Using integral ∫_0^∞ log(1+x²)/(1+x²) dx = π log 2, we get H = log(π) + log 4 = log(4π). Note: Mathematica may incorrectly output log(π) due to branch cuts.

b) Renormalized Expectation

For Cauchy, E[X] is undefined because ∫ x/(π(1+x²)) dx diverges. A renormalized expectation can be defined by symmetric truncation: lim_{a→∞} ∫_{-a}^{a} x/(π(1+x²)) dx = 0 by odd symmetry. This subtracts two infinite quantities (positive and negative tails) to yield zero.

c) Renormalized Variance

The renormalized variance is defined as lim_{a→∞} (∫_{-a}^{a} x²/(π(1+x²)) dx - (E_renorm[X])²). Since E_renorm=0, compute ∫_{-a}^{a} x²/(π(1+x²)) dx = (2/π)(a - arctan a). As a→∞, this diverges linearly. However, the renormalized variance is considered as the limit of the second moment after subtracting divergent parts? Actually, the limit does not exist in finite sense; the Cauchy distribution has infinite variance. The problem likely expects that the renormalized variance diverges.

Problem 9.3: Entropy Comparison

a) Entropy of Standard Normal

The standard normal density φ(x)=e^{-x²/2}/√(2π). Its differential entropy is H = -∫ φ(x) log φ(x) dx = (1/2)log(2πe). Specifically, H = (1/2)log(2π) + (1/2) = (1/2)log(2πe).

b) Which Entropy is Larger?

Cauchy entropy: log(4π) ≈ log(12.566) ≈ 2.531. Normal entropy: (1/2)log(2πe) ≈ (1/2)log(17.079) ≈ (1/2)*2.837 = 1.4185. Thus Cauchy entropy is larger, reflecting its heavier tails and greater uncertainty.

c) Entropy of Probability Theory Distribution

This part likely refers to a specific distribution; assuming standard Cauchy, answer is log(4π).

Problem 9.4: Convolution of Measures

a) Convolution Identity

Given dµ=f dx, dν=g dx, the convolution f*g satisfies ∫ (f*g)(z) h(z) dz = ∫∫ f(y) g(z-y) h(z) dy dz. Change variable x=z-y gives ∫∫ f(y) g(x) h(x+y) dy dx = ∫ h(x+y) dµ(x) dν(y).

b) Convolution Measure

For any Borel set A, (µ*ν)(A) = ∫∫ 1_A(x+y) dµ(x) dν(y). This follows from the identity above with h(z)=1_A(z).

c) Fixed Point of Transformation

Consider the transformation T(µ) = µ*µ on probability measures with mean zero. The unique fixed point is the Dirac delta at 0, since T(δ_0)=δ_0. Uniqueness follows from characteristic functions: ϕ_{T(µ)}(t)=ϕ_µ(t)², so fixed point requires ϕ_µ(t)=ϕ_µ(t)², implying ϕ_µ(t)=0 or 1 for all t. Continuity and ϕ_µ(0)=1 force ϕ_µ(t)=1, so µ=δ_0.

Problem 9.5: Proving de Moivre Central Limit Theorem

a) Using Stirling's Approximation

Let S_n be Binomial(n,1/2). The de Moivre CLT states that (S_n - n/2)/√(n/4) → N(0,1). Using Stirling: n! ~ √(2πn)(n/e)^n. For k near n/2, approximate P(S_n=k) = n!/(k!(n-k)!) (1/2)^n. Set k = n/2 + x√n/2. Then log P ≈ -x²/2 - (1/2)log(πn/2). Normalizing yields density approaching φ(x).

b) Using Characteristic Functions

The characteristic function of (S_n - n/2)/√(n/4) is ϕ_n(t) = (e^{it/√n} + e^{-it/√n})^n / 2^n = (cos(t/√n))^n. As n→∞, log ϕ_n(t) = n log(1 - t²/(2n) + o(1/n)) → -t²/2, the characteristic function of N(0,1). By Lévy's continuity theorem, convergence in distribution holds.

Conclusion

From Cauchy's heavy tails to normal's universality, the CLT shapes modern data science. In 2026, as we analyze streaming data or AI training distributions, these mathematical foundations remain essential. Practice these problems to master probability theory and its applications.