Further Foundations

Chapter III of Rosenthal

Let’s get right into it. We’ve already defined probability triples, now let’s start talking about random variables. Notice that before, we didn’t have any randomness. Actually, it will still take a bit of time before we encounter any stochasticity.

This is really just a technical definition to be on common ground. Not much intuition can be or should be given to it. In essence, this just says that $X$ is measurable: the $\sigma$-algebra contains the relevant sets so that, along with $\mathbf{P}$, we can determine the probability of events. One can almost think of the function $X$ as defining the cumulative density function, but that’s not quite the case.

This gets a lot out of the way. In particular, we can perform basic algebra with random variables. Can we extend this notion to continuous functions of random variables? It turns out we can.

Essentially, a measurable function is one where the pre-images of Borel sets are still Borel sets. That is, we can apply a measure to the resulting transformation based on the size of the set that yielded the result. Note that the Borel sets is the smallest $\sigma$-algebra containing all the intervals of $\R$.

Back to our question about taking functions of random variables. For a measurable function $f$ and a random variable $X$, it turns out that $f(X)$ is still a random variable, since compositions of measurable functions are measurable.

For example, if $f(x) = x^k$, then $f$ is Borel-measurable. Hence, for if $X$ is a random variable, then so is $X^k$ for all $k \in \N$.

The book makes an additional remark about the random variable possibly being a function that maps to some other measurable space. This is nice for completeness, but in my experience, mapping to the Lebesgue measure is more than expressive enough.

Independence

Now for a classic word: independence. We know what this means intuitively: two events are independent if they do not affect each other’s probabilities. In other words, $A$ and $B$ are independent if $P(A \cap B) = P(A)P(B)$.

Three events are independent if they are pairwise independent as above and $P(A \cap B \cap C) = P(A)P(B)P(C)$. You need all these conditions, it may be good to think about why.

The definition extends naturally to random variables by associating the events $A$ and $B$ to the measurable sets $X^{-1}(S_1)$ and $Y^{-1}(S_2)$. That is, the random variables are independent if, for all Borel sets $S_1, S_2$, the events $X \in S_1$ and $Y \in S_2$ are independent. All these notions can be extended to collections of events or variables by considering all finite subsets of events in the collection.

Further, deterministic transformations preserve independence:

Continuity of Probabilities

Given a standard probability triple and events $A_1, A_2, \dotsc \in \mathcal{F}$, we write $\{A_n\} \nearrow A$ to mean that $A_1 \subseteq A_2 \subseteq \cdots,$ and $\bigcup_n A_n = A$. In words, the events $A_n$ increase to $A$. Similarly, we write $\{A_n\} \searrow A$ to mean that $\{A_n^c\} \nearrow A^c$, or equivalently that $A_1 \supseteq A_2 \supseteq \cdots$ and $\bigcap_n A_n = A$; the events decrease to $A$.

Limit Events

Given events $A_1, A_2, \dotsc \in \mathcal{F}$, we define

$$\htmlId{eq-7}{\limsup_n A_n = \{A_n \text{ infinitely often }\} = \bigcap_{n=1}^\infty \bigcup_{k=n}^\infty A_k} \tag{7}$$

and

$$\htmlId{eq-8}{\liminf_n A_n = \{A_n \text{ almost always }\} = \bigcup_{n=1}^\infty \bigcap_{k=n}^\infty A_k.} \tag{8}$$

Since $\mathcal{F}$ is a $\sigma$-algebra, these are well defined events. They correspond to those events in the collection of $A_n$ that happen either infinitely often, or the complement.

The theorem is striking: if the events are independent, the limsup is either 0 or 1 and never anything else. The key theme here is that if we can show that the sum of probabilities of a bunch of events diverges, then these events must occur infinitely often.

Tail Fields

For example, consider infinite fair coin tossing. Let $H_n$ denote the event that the $n$-th coin comes up heads. Then $\tau$ includes the event limit event $\limsup_n H_n$, $\liminf_n H_n$, and many more interesting events.

Before we can prove it, we need a corollary on independence.