Expected Values

Rosenthal — Chapter IV

Expected values are integral to probability theory. This note crystallizes the notion of expected values for random variables that will be used throughout the remainder of the text.

Simple Random Variables

Before jumping into the notion in full generality, we’ll start with a special case. For those familiar with measure theory, this should be reminiscent of simple functions. They’ll serve as a basis for extending the notion of expected values (integration) to general random variables (functions).

In other words, the random variable can only take on finitely many values. We can always write these variables in a particular form. Denote the values $X$ takes on by $x_1, \dotsc, x_n$, and their associated pre-images by $A_i$, so $X(A_i) = \{x_i\}$. We can then write $X = \sum_{i=1}^n x_i \mathbf{1}_{A_i}$. Note that the sets $\{A_i\}$ form a finite partition of $\Omega$.

For such a random variable, we define the expected value as follows.

We also see from the definition that $\mathbf{E}[ \mathbf{1}_{A_1} ] = \mathbf{P}(A)$. It also follows from the definition that $\mathbf{E}$ is both linear, and order-preserving.

Moreover, when $X$ and $Y$ are independent, we have that

$$\htmlId{eq-4}{\mathbf{E}[XY] = \mathbf{E}[X]\mathbf{E}[Y].} \tag{4}$$

Clearly, $\mathrm{Var}[X] \geq 0$ and from the linearity of expectation, we see that

$$\htmlId{eq-5}{\mathbf{E}[(X - \mu_X)^2] = \mathbf{E}[X^2] - \mu_X^2.} \tag{5}$$

And it immediately follows that

$$\htmlId{eq-6}{\mathrm{Var}[\alpha X + \beta] = \alpha^2 \mathrm{Var}[X].} \tag{6}$$

Let’s conclude with a fact, because this is all there really is to say about simple random variables.

General Non-negative Random Variables

At first glance, it’s not obvious how to extend our definition of expected values to random variables that are not simple. However, the definition itself should conjure an image of Riemann, or better yet, Lebesgue, integration. In fact, the previous proposition applies to $X$ being simple, but could serve as a definition for the expected value of $X$ when it is not simple.

In this case, it is indeed possible that $\mathbf{E}[X] = \infty$. That’s something that we’re willing to accept, and mirrors the fact that integrals may be infinite in measure theory.

Recall that for $k \in \N$, the $k$-th moment of a non-negative random variable is defined to be $\mathbf{E}[X^k]$