Class 4: Randomness
Key Ideas
Classes 1 through 3 described data we already had. Today we describe the process that produced it.
A random variable is a number whose value is determined by chance. It is fully specified by its distribution: a list of the possible outcomes and a probability for each, summing to one.
Problems of the Day
Q9: Numbers You Can’t Predict
A random variable is a variable whose value you cannot predict exactly before you observe it. For each quantity below, decide whether it is a random variable. If it is, list the possible outcomes, and say whether you could write down the probability of each outcome. If it is not, say what makes it different.
- The number showing after you roll a die.
- The hours of sleep of the next student to walk through the door.
- Whether that same student has a job.
- That student’s commute, in minutes.
- Your own height.
- A coin that has already been flipped, but is still covered by your neighbor’s hand.
Extension. Imagine rolling a fair die a million times and computing the sample mean of the results. The number you get is the expected value of the die, \(E[X] = \sum_x x \cdot \Pr(X = x)\), each possible value weighted by its probability. Calculate the expected value of a die. The answer will be a value the die can never show. In what sense is it still the right answer?
Q10: The Phone Insurance Question
Phone insurance costs $50 a year. Suppose your phone breaks during the year with probability 0.05, and a replacement costs $600. Does it make sense to buy the insurance, given expected values?
Extension. Many people would buy insurance anyway. Compare the two situations you can choose between: “lose $50 for sure” and “lose $600 with probability 0.05, otherwise lose nothing.” The main difference between them lies in their spread, which the variance measures: \(\text{Var}(X) = E\left[(X - E[X])^2\right] = \sum_x (x - E[X])^2 \cdot \Pr(X = x)\), the probability-weighted average of squared distances from the expected value. Compute the variance of each situation. How does the variance explain why people buy insurance?
Extension #2. Two random variables are independent when learning the value of one leaves the probabilities of the other unchanged. Your phone and a stranger’s phone in another city each break with probability 0.05; are they independent? Fill in a table with the probabilities of “both break,” “exactly one breaks,” and “neither breaks.” Now replace the stranger’s phone with your roommate’s, which lives in the same backpack as yours. Which entries of the table go up, which go down, and why can’t you fill it in exactly?
Q11: Ten Rolls
These four students all rolled a dice ten times. Here are the results:
| Maya | Jordan | Sofia | Ethan |
|---|---|---|---|
| 3 | 4 | 5 | 1 |
| 2 | 6 | 5 | 2 |
| 1 | 1 | 2 | 5 |
| 1 | 5 | 6 | 4 |
| 1 | 1 | 1 | 4 |
| 1 | 2 | 5 | 5 |
| 2 | 5 | 4 | 3 |
| 2 | 3 | 6 | 1 |
| 5 | 6 | 6 | 2 |
| 5 | 6 | 5 | 5 |
Compute Maya’s sample mean, Jordan’s sample mean, Sofia’s sample mean, and Ethan’s sample mean. Compare them to the true expected value, 3.5. Now pool all the rolls and recompute. What happened, and why?
Extension. Now switch from dice to the survey. Let \(X\) be the hours of sleep of a randomly chosen student at this university. Can you compute them the way you computed \(3.5\)? What is your best guess for \(E[X]\)?
Answers
Q9: Numbers You Can’t Predict
The test is whether the value is settled before you look. A random variable is defined by its distribution, the list of outcomes with a probability attached to each, and the interesting cases are the ones where you can name the outcomes but not the probabilities.
- Random variable. Outcomes 1 through 6, each with probability \(1/6\) if the die is fair. This is the one case where the distribution can be written down completely from first principles.
- Random variable. The outcomes are the sleep values students report, which in the survey run from 3 to 10 hours in half-hour steps. The probabilities are not known, but they can be estimated: 25 of 94 students reported 8 hours, so \(\Pr(X = 8) \approx 0.27\). The distribution exists; we just have to measure it rather than derive it.
- Random variable. Two outcomes, job or no job. The probability can be estimated from the survey (28 of 95 are employed, about 0.29), and the fact that the outcome is a category rather than a number is no obstacle: code “job” as 1 and “no job” as 0, as with pets in Class 1.
- Random variable. The possible outcomes are any non-negative number of miles, which is a different kind of list from the die’s: there are infinitely many outcomes, and the probability of exactly 12.0 miles is zero. For a continuous variable the distribution assigns probabilities to ranges (“between 10 and 15 miles”) rather than to single values. It is still a distribution, and it can still be estimated.
- Not a random variable, at least not to you. Your height is a fixed number that you know. It would be a random variable to someone picking a student at random and asking, which shows that randomness is a property of the process of observation, not of the quantity itself.
- Random variable, and this is the case the question is built around. The coin has landed, so physically there is nothing left to happen, but you still cannot predict what you will see when the hand lifts, and heads and tails each have probability \(1/2\) from where you sit. Randomness describes what you know, not what has happened. Item 5 and item 6 are the same point from opposite directions.
Extension. \(E[X] = 1 \cdot \tfrac{1}{6} + 2 \cdot \tfrac{1}{6} + \cdots + 6 \cdot \tfrac{1}{6} = \tfrac{21}{6} = 3.5\). The die cannot show 3.5, and that is fine, because the expected value does not claim to be a typical roll. It is the balance point of the distribution, the same role the mean played for a list of data in Class 1, and it answers a question about the long run: roll a million times and the sample mean will be within a hair of 3.5, so a bet that pays the face value is worth $3.50 a roll. The expected value is the right answer to “what does a roll cost or pay on average,” not to “what will the next roll show.” The mean of a list did not have to be a value in the list either (Class 1, Q1), and the same is true here.
Q10: The Phone Insurance Question
Without insurance, the loss is $600 with probability 0.05 and $0 with probability 0.95, so the expected loss is \(0.05 \times 600 + 0.95 \times 0 = \$30\). The insurance costs $50 for sure. By expected values the insurance is a bad deal: it costs $20 a year more than the expected loss it removes. That $20 is the insurer’s margin, and it has to be there, because the insurer is pooling thousands of phones and needs its sure $50 to exceed its expected $30 payout to stay in business. A buyer who goes by expected value alone should decline.
Extension. The two situations have the same kind of outcome (money lost) but very different spread. “Lose $50 for sure” has one outcome, so every deviation from the expected value is zero and the variance is \(0\). For the uninsured phone, \(E[X] = 30\), and
\[\text{Var}(X) = 0.05 \times (600 - 30)^2 + 0.95 \times (0 - 30)^2 = 0.05 \times 324{,}900 + 0.95 \times 900 = 17{,}100,\]
so the standard deviation is \(\sqrt{17{,}100} \approx \$131\). Most years the uninsured owner does better by $50, but one year in twenty they are out $600 at once, and for someone who cannot absorb a $600 hit, that year is a different kind of loss than twelve years of $50 each. Insurance is a trade: pay $20 in expected value to bring the variance from 17,100 down to zero. Whether that trade is worth it depends on how much the spread hurts, which is why the expected value alone cannot settle the question. People who buy insurance are not making an arithmetic mistake; they are paying for lower variance.
Extension #2. Your phone and a stranger’s phone in another city have nothing to do with each other, so learning that one broke tells you nothing about the other: the two are independent, and the joint probabilities are products of the marginals.
| Probability | |
|---|---|
| Both break | \(0.05 \times 0.05 = 0.0025\) |
| Exactly one breaks | \(2 \times 0.05 \times 0.95 = 0.095\) |
| Neither breaks | \(0.95 \times 0.95 = 0.9025\) |
The rows sum to one, as they must. Now put your roommate’s phone in the same backpack. Each phone still breaks with probability 0.05 on its own, but the events are no longer independent: the backpack gets dropped, rained on, or left on a bus, and when that happens both phones go together. “Both break” goes up, and since the marginals are fixed, “exactly one breaks” must go down by twice as much and “neither breaks” must go up to keep the total at one. How far the entries move depends on how often a single accident takes out both phones, and that is a number the problem does not give and the two marginal probabilities cannot supply. Independence is what lets you fill the table from the margins alone; without it, you need to know the joint distribution directly.
Q11: Ten Rolls
| Maya | Jordan | Sofia | Ethan | Pooled | |
|---|---|---|---|---|---|
| Sum | 23 | 39 | 45 | 32 | 139 |
| Rolls | 10 | 10 | 10 | 10 | 40 |
| Sample mean | 2.3 | 3.9 | 4.5 | 3.2 | 3.475 |
All four dice are fair, so all four have the same expected value, 3.5, yet the four sample means range from 2.3 to 4.5. None of the students did anything wrong. Ten rolls is a small sample, and a sample mean of ten rolls is itself a random variable: Maya rolled four ones and no sixes, Sofia rolled four sixes and one one, and both outcomes are perfectly ordinary runs of luck. Pooled, the 40 rolls give a sample mean of 3.475, within 0.025 of the truth. The pooling cancels the luck: Maya’s low rolls and Sofia’s high rolls are averaged against each other, and the more rolls go into the mean, the less any one run of luck can move it.
The distinction to draw is between the parameter \(E[X] = 3.5\), a fixed property of the die that does not change from student to student, and the estimate \(\bar{x}\), a number computed from data that changes with every new sample. The expected value is what the sample mean is trying to find; the sample mean is the sum of ten probability-weighted possibilities only in the limit, as the number of rolls grows without bound.
Extension. For the die, \(E[X]\) could be computed exactly because the distribution was known in advance: six outcomes, each with probability \(1/6\). For sleep, the outcomes are known (3 to 10 hours, in the values students report) but the probabilities are not. There is no symmetry argument that makes \(\Pr(X = 8)\) equal to anything in particular, so the formula \(\sum_x x \cdot \Pr(X = x)\) cannot be evaluated. What can be done is exactly what the students did with the dice: take a sample and compute its mean. The 94 students who answered the sleep question average 6.84 hours, and that is the best guess for \(E[X]\). It is an estimate, not the parameter; a different section of this course surveyed on a different day would give a different sample mean, in the same way that Maya and Sofia gave different means for the same die. The lesson of the pooled rolls applies here too: 94 students is a far better guess than any 10 of them would give, and the whole university would be better still.