Data Analysis with R

EC005 · A Practical Introduction for Economists

Introduction

Course Design

This course is for students who perhaps have never written a line of code or taken a formal statistics course, but who are curious about how economists use data to answer “real-life” questions. Class time is spent working problems: deciding whether “half the class scores below average” is always true, replotting the O-ring data the engineers had the night before Challenger, or solving a murder mystery with nothing but a database. Between classes, you learn to program in R for data analysis through koans, short guided scripts with blanks to fill in and automated tests that tell you when you have it right. By the end of the term you will have used vectors, tibbles, pipes, dplyr, ggplot2, lm(), custom functions, and map() on real data, and written simulations of your own.

The first half of the course builds the vocabulary of data. We start with summary statistics and group means, then visualization, then randomness: random variables, expected value, variance, covariance, and correlation. From there we turn to linear regression, and rather than taking the OLS formulas on faith we derive them, find out what makes an estimate differ from the truth, and prove that the estimator is unbiased under a single assumption called exogeneity. Treating the slope as a random variable with its own sampling distribution leads naturally to standard errors, confidence intervals, and hypothesis tests, each checked by simulation so that “95%” means something you have watched happen.

The last part of the course asks what data can actually tell us about cause and effect. We add controls, dummies, and interactions to our regressions, learn to sign an omitted variable bias, and meet confounders, reverse causality, and randomized experiments. When exogeneity fails, instrumental variables offer a way to recover a causal effect, and the course finishes by using a frost in Brazil as an instrument to pull the demand curve for coffee out of a cloud of market crossings, connecting the tools of data analysis to the theory of Principles of Microeconomics and Macroeconomics. Along the way you complete a data project of your own, from finding a dataset to writing up what it can and cannot say. The five course goals below show which level of each skill every class targets.

Course Calendar

Course Goals

The course works toward five goals. Each goal has a basic, intermediate, and advanced level, and every class is designed to move you to a particular level of one or more goals. Use this map to see what each class is for and how the pieces fit together.

Course goalsEach goal has three levels; the map below shows which level each class targets

1.Describe data

Reduce a dataset to honest numbers and pictures, and know when either one misleads.

BasicCompute the mean, median, mode, and spread by hand, and read a histogram, box plot, or scatterplot

IntermediateCompute group means, choose the plot that answers a question, and judge whether a chart is honest

AdvancedOrganize a raw dataset and build the summaries and plots that answer a question in code

2.Reason about randomness

Treat data as one draw from a random process and quantify what that implies.

BasicDefine a random variable and compute its expected value and variance

IntermediateDescribe the sampling distribution of an estimator and its standard error

AdvancedConstruct a confidence interval and carry out a hypothesis test

3.Estimate a regression

Fit a line to data, understand where the formula comes from, and read the output.

BasicInterpret a slope, an intercept, a residual, and an error term

IntermediateDerive the OLS estimators and explain unbiasedness, leverage, and R-squared

AdvancedSpecify a multiple regression with controls, dummies, and interactions, and read a coefficient as a marginal effect

4.Reason about causality

Say when a regression coefficient is a causal effect, and what to do when it is not.

BasicExplain why correlation is not causation and what exogeneity assumes

IntermediateSign an omitted variable bias, identify confounders and reverse causality, and explain why randomization works

AdvancedUse an instrument to estimate a causal effect and identify a supply or demand curve from market data

5.Program in R

Write reproducible R code that turns raw data into results.

BasicWork with vectors, tibbles, and pipes

IntermediateUse dplyr, ggplot2, and lm() on a real dataset

AdvancedWrite functions, iterate with map(), and run simulations

BBasicIIntermediateAAdvancedHomework rows are shown in grey.

1 Describe data  ·  2 Reason about randomness  ·  3 Estimate a regression  ·  4 Reason about causality  ·  5 Program in R

Class	1	2	3	4	5
Class 1: Summary Statistics You decide whether statements like 'half the class scores below average' are always, sometimes, or never true, compute averages from the class survey and find which ones are meaningless, and invent a number that captures why two groups with the same mean aren't equally rested. You compute and interpret center and spread by hand and read each statistic in its real context (Goal 1, basic).	B
HW0: Installing R You install R, RStudio, and the tidyverse, then compile a check document to confirm everything works. It is the first step of the programming goal (Goal 5, basic).					B
Class 2: Data Visualization You answer the dean's five questions in pictures instead of numbers, build two honest posters from the same data that tell opposite stories, and replot the O-ring data the engineers had the night before Challenger. You learn to read distributions from histograms, box plots, and scatterplots, and to choose the plot that answers the question honestly (Goal 1, intermediate).	I
HW1: Vectors, Tibbles, and Pipes Your first set of koans: guided R scripts with blanks to fill and automated tests that check your answers. They cover creating vectors, calling functions, chaining with the pipe, and building tibbles (Goal 5, basic).					B
Class 3: Group Means You guess a new student's sleep, then revise it once you learn they have a job; you check a blogger's claim that quitting your job buys half an hour of sleep; and you rebuild the overall mean from a table of group means after the raw data is lost. Group means and how they aggregate are the statistical core (Goal 1, intermediate).	I
Class 4: Randomness You sort quantities into random variables and not, decide whether phone insurance is worth it using expected value, and compare four students' ten dice rolls to separate luck from bias. Random variables, expectation, and variance arrive here (Goal 2, basic).		B
HW2: Dplyr Koans on filter, select, mutate, summarize, group_by, count, arrange, and slice, all run on the class survey, ending with a koan that combines the verbs to redo a class exercise. You organize and summarize a dataset in code (Goal 1, intermediate; Goal 5, intermediate).	I				I
Class 5: Covariance and Correlation You test pairs of variables for independence, compute a covariance from coin flips, predict the sign and strength of correlations before seeing data, and watch the correlation stay put when commute switches from miles to minutes. This extends randomness to two variables at once (Goal 2, intermediate), and the story of the 2008 mortgage models shows what a correlation is silent about (Goal 4, basic).		I		B
Class 6: Murder Mystery A whodunit solved entirely with data: you filter, join, and summarize your way through the clues until only one suspect is left. It is the first time the dplyr verbs are used together to organize a raw dataset around a question (Goal 1, advanced; Goal 5, intermediate).	A				I
HW3: Ggplot2 Koans on the grammar of graphics: mapping variables to aesthetics, choosing geoms, and adding labels and themes. You build the plots from Class 2 in code (Goal 1, advanced; Goal 5, intermediate).	A				I
Class 7: Intro to Linear Regression Five students' study hours and exam scores: you draw a line by eye, write the regression model and say what each symbol means, brainstorm everything else that lives in the error term, and compute residuals with signs. This is the vocabulary of regression, slope, intercept, residual, and error (Goal 3, basic).			B
Class 8: Deriving OLS Estimators You judge candidate lines by their sum of squared residuals, find the bottom of the bowl by setting a derivative to zero, and compare the fitted residuals to the true errors when you are told the truth. You derive the OLS formulas yourself and see why estimate and truth differ (Goal 3, intermediate).			I
HW4: lm Koans on fitting regressions with lm(), reading the output with broom, and plotting fitted lines. You run the regressions you derived by hand (Goal 3, basic; Goal 5, intermediate).			B		I
Class 9: More on Deriving OLS You show three algebraic faces of the slope formula, compute the leverage each observation has on the slope, and define R-squared as the share of variation explained. It deepens the derivation into what the estimator is actually made of (Goal 3, intermediate).			I
Class 10: Exogeneity You substitute the truth into the slope formula to find the gap between estimate and parameter, work with conditional expectations, and finish the proof that OLS is unbiased under one assumption. You see exactly what unbiasedness requires (Goal 3, intermediate) and meet exogeneity as the assumption that separates correlation from a causal reading (Goal 4, basic).			I	B
HW5: Custom Functions Koans on writing your own functions: arguments, return values, and reusing code. This is the step from running R to programming in it (Goal 5, advanced).					A
Class 11: Distribution of Regression Coefficients Starting from three normal curves, you work out what the sampling distribution of the slope estimator is, why the central limit theorem makes it normal, and what its variance depends on. The regression slope becomes a random variable with a distribution of its own (Goal 2, intermediate).		I
Class 12: Standard Errors You estimate the variance of the slope from data, learn what degrees of freedom and the residual standard error are, and read a full regression output line by line. Standard errors quantify uncertainty about an estimate (Goal 2, intermediate) and make regression output legible (Goal 3, intermediate).		I	I
HW6: map() Koans on iterating with purrr's map(): applying a function across many inputs and collecting the results. It is the tool that makes simulation possible (Goal 5, advanced).					A
Class 13: Confidence Intervals You build confidence intervals around a slope, choose critical values, and read a plot of fifty simulated intervals to see what '95%' actually promises. It is the advanced level of reasoning about randomness (Goal 2, advanced).		A
HW7: Simulations I You write simulations that draw repeated samples, estimate a regression on each, and study how the estimates spread. Simulation makes the sampling distribution something you can see (Goal 2, intermediate; Goal 5, advanced).		I			A
Class 14: Hypothesis Testing A herbalist claimed to have lived 196 years: you use the statistics of very old ages to ask whether that is believable, and from there set up null and alternative hypotheses, p-values, and significance levels for regression coefficients. It completes the inference toolkit (Goal 2, advanced).		A
Class 15: Multiple Regression and Multicollinearity You add control variables to a regression, see what happens when two regressors move together, and work out the direction of bias when an important variable is left out. Multiple regression is the advanced form of the regression goal (Goal 3, advanced), and omitted variable bias is the central idea of causal reasoning (Goal 4, intermediate).			A	I
HW8: Simulations II More simulation: you check coverage of confidence intervals and the size of hypothesis tests by repeating them many times. It ties the inference tools to their sampling distributions (Goal 2, intermediate; Goal 5, advanced).		I			A
Class 16: Interactions and Squared Terms You add dummy variables, interaction terms, and squared terms to a regression and learn to read a marginal effect that depends on where you are. It extends model specification and turns coefficients into economic quantities like the marginal effect for a particular group (Goal 3, advanced).			A
Class 17: Causal Inference You define a causal effect, meet confounders and reverse causality, and see why a randomized controlled trial answers the question a regression alone cannot. It is the conceptual heart of the causality goal (Goal 4, intermediate).				I
HW9: Data Project A full analysis of your own: you find a dataset, organize it, visualize it, run and interpret regressions, and write up what the data can and cannot say. It brings together describing data, regression, and R (Goals 1, 3, and 5, advanced).	A		A		A
Class 18: Estimating Causal Effects with IV You learn what an instrument is, what the first stage and the exclusion restriction require, and compute the instrumental variables estimator. It is the advanced level of causal reasoning: using an instrument to recover a causal effect when exogeneity fails (Goal 4, advanced).				A
Class 19: Estimating Supply and Demand with IV Five hundred seasons of coffee prices and quantities give a cloud of supply and demand crossings, not a demand curve. You use a frost in Brazil as a supply shifter to trace out demand, then ask what would identify supply instead. It is the capstone: identifying a causal relationship and connecting it to the theory of Principles courses (Goal 4, advanced).				A

Slides

• Class 1 slides

Extra Resources

This course combines three major topics: prob/stat and linear regression, programming in R, and causal inference. For each of these topics, here are extra resources I’d point you toward to learn more:

1. Probability, Statistics, and Linear Regression

   • Introduction to Econometrics (Christopher Dougherty)

2. Programming in R for Data Analysis

   • tibble
   • view
   • read_csv
   • dplyr
   • ggplot2
   • map

3. Causal Inference (exogeneity, estimating causal effects with IV)

   • The Effect (Nick Huntington-Klein)

---

Good luck, and welcome to the world of data.