Distributions — Alex Beaudin

The distribution or law of a random variable is defined as follows.

If $\mu$ is the law of a random variable, then $(\mathbf{R}, \mathcal{B}, \mu)$ is a valid probability triple. It’s also common to refer to the law $\mu$ as $\mathcal{L}(X)$ or $\mathbf{P} X^{-1}$ , and we denote that $\mu$ is the law of a random variable $X$ by $X \sim \mu$ .

Notice a couple things. First, the CDF of a random variable is right continuous in the sense that if $\{x_n\} \searrow x$ , then $\lim_{n \to \infty} F_X(x_n) = F_X(x)$ . Moreover, it is a non-decreasing function of $x$ , with $\lim_{x \to -\infty } F_X(x) = 0$ and $\lim_{x \to \infty} F_X(x) = 1$ . We note the following.

Change of Variables Theorem

How do we know that a distribution tells us everything about a random variable? Of course, anyone with exposure to introductory probability theory has the impression the distribution is the random variable. However, it might be worth double checking in our formalism.

Theorem.Change of Variable Theorem

Given a probability triple $(\Sigma, \mathcal{F}, \mathbf{P})$ , let $X$ be a random variable having distribution $\mu$ . Then for any Borel-measurable function $f: \mathbf{R} \to \mathbf{R}$ , we have

\htmlId{eq-3}{ \int_\Omega f(X(\omega)) \mathbf{P}(d \omega) = \int_{-\infty }^\infty f(t) dt.} \tag{3}

In other words, $\mathbb{E}_{\mathbf{P}}[f(X)] = \mathbb{E}_\mu[f]$ , provided that either side is well-defined. The expected value of the random variable $f(X)$ with respect to the probability measure $\mathbf{P}$ on $\Omega$ is equal to the expected value of the function $f$ with respect to the measure $\mu$ on $\mathbf{R}$ .

Proof.

Suppose first that $f = \mathbf{1}_{B}$ is an indication function of a Borel set $B \subseteq \mathbf{R}$ . Then $\int_\Omega f(X(\omega)) \mathbf{P}(d\omega) = \int_{\Omega } \mathbf{1}_{\{X(\omega) \in B\}} \mathbf{P}(d \omega) = \mathbf{P}(X \in B)$ , while $\int_{-\infty}^\infty f(t) dt = \int_{-\infty }^{\infty } 1_{\{t \in B\}} \mu(dt) = \mu(B) = \mathbf{P}(X \in B)$ . Hence, equality holds in this case.

Now suppose that $f$ is a non-negative simple function. A simple function is a positive linear combination of the indicator functions we considered in the previous paragraph. Since both sides equality $\href{#eq-3}{(3)}$ linear in $f$ , the result holds.

Now suppose that $f$ is a general non-negative Borel-measurable function. We know that we can find a sequence of functions where $\{f_n\} \nearrow f$ and each $f_n$ is simple and so $\href{#eq-3}{(3)}$ holds. By the monotone convergence theorem, $\href{#eq-3}{(3)}$ holds for $f$ .

Finally, for general Borel-measurable $f$ , we can write $f = f^+ - f^-$ . We know that $\href{#eq-3}{(3)}$ is linear and holds for $f^+$ and $f^-$ separately, so it must hold for $f$ . Thus terminates the proof.

$\blacksquare$

Now would be a good time to talk about density functions. Those with prior exposure to probability theory know that a density function captures the amount of mass at a particular point. We’ve discussed laws and distributions, so how does the density function tie in?

We often say that $f$ is the density of $\mu$ with respect to $\lambda$ , sometimes written as $\frac{d\mu}{d\lambda} = f$ . In this case, we say that $\mu$ is absolutely continuous with respect to $\lambda$ .

Now, we can compute things!