My Long-Time Peers Told Me I Regressed (s**tpost)

Talking to old colleagues sucks sometimes, especially when they're cynical.

I think they were just mad, because a rizzler like me hit the skibidi toilet scuba in Ohio 6-7 times and aura farmed on them, no cap. 🚫🧢

They said: “Thommas, you’ve really regressed.”

Of course I regressed—technically it's called Regression—a statistical analysis method used to estimate the relationship between an outcome and the predictor variables. 📈📉

You probably remember from school 🏫:

$$y = mx + b$$

But you can go deeper 🕳️🪏. Let's redefine these variables:

symbol	school	description
$\hat{y}_i$	$y$	the $i$th prediction
$\beta_0$	$b$	the intercept
$\beta_1$	$m$	the slope
$x_i$	$x$	the $i$th predictor variable

Final equation for a simple linear regression:

$$\hat{y}_i = \beta_0 + \beta_1 x_i$$

You see 👀, the goal is to draw a line that minimises the residuals ($e_i$) from the data points. Some people call these errors.

$$e_i = y_i - (\beta_0 + \beta_1 x_i)$$

In the wild, you'll see people refer to linear regression as Ordinary Least Squares (OLS). This is because we are trying to minimise the Sum of Squared Errors (SSE), to penalise the big ones more than the small ones. Since real life data is imperfect (except for Me 🧍🏻‍♂️), let's append to our formula to account for error:

$$\hat{y}_i = \beta_0 + \beta_1 x_i + e_i$$ $$SSE = \sum_{i=1}^{n} (y_i - \beta_0 - \beta_1 x_i)^2$$

Solving for the intercept

The goal (of OLS and of Life itself) is to find the perfect balance in the line to minimise the SSE. We take the partial derivative of the $SSE$ with respect to $\beta_0$ and solve for zero:

$$\frac{\partial SSE}{\partial \beta_0} = -2 \sum_{i=1}^{n} (y_i - \beta_0 - \beta_1 x_i) = 0$$ $$\sum_{i=1}^{n} y_i - \sum_{i=1}^{n} \beta_0 - \beta_1 \sum_{i=1}^{n} x_i = 0$$

If you add up a constant ($\beta_0$) exactly $n$ times, you just get $n\beta_0$. Let's move that to the other side:

$$\sum_{i=1}^{n} y_i = n\beta_0 + \beta_1 \sum_{i=1}^{n} x_i$$

Divide the entire equation by $n$. When you divide a sum by $n$, you get an average (which we write as $\bar{y}$ and $\bar{x}$):

$$\bar{y} = \beta_0 + \beta_1 \bar{x}$$

Solve for $\beta_0$:

$$\hat{\beta}_0 = \bar{y} - \hat{\beta}_1 \bar{x}$$

Once you know the slope ($\beta_1$), you can find the intercept by plugging in the averages of your $x$ and $y$ data ($\bar{x}$ and $\bar{y}$).

Solving for the slope

Now we do the same thing, but taking the derivative with respect to $\beta_1$:

$$\frac{\partial SSE}{\partial \beta_1} = -2 \sum_{i=1}^{n} x_i (y_i - \beta_0 - \beta_1 x_i) = 0$$ $$\sum_{i=1}^{n} (x_i y_i - \beta_0 x_i - \beta_1 x_i^2) = 0$$

Substitute the $\beta_0$ formula we just found ($\beta_0 = \bar{y} - \beta_1 \bar{x}$) into this equation:

$$\sum_{i=1}^{n} \left( x_i y_i - (\bar{y} - \beta_1 \bar{x}) x_i - \beta_1 x_i^2 \right) = 0$$

Expand it out:

$$\sum_{i=1}^{n} (x_i y_i - \bar{y} x_i + \beta_1 \bar{x} x_i - \beta_1 x_i^2) = 0$$

Group all the terms with $\beta_1$ on one side:

$$\sum_{i=1}^{n} (x_i y_i - \bar{y} x_i) = \beta_1 \sum_{i=1}^{n} (x_i^2 - \bar{x} x_i)$$

Finally, divide to isolate $\beta_1$. With a bit of algebraic rearranging, this simplifies into its final form:

$$\hat{\beta}_1 = \frac{\sum_{i=1}^{n} (x_i - \bar{x})(y_i - \bar{y})}{\sum_{i=1}^{n} (x_i - \bar{x})^2}$$

The slope is just the covariance (more on this another time) of $x$ and $y$ (how they move together) divided by the variance of $x$ (how spread out the $x$ values are).

Congratulations to me, I just solved Regression. 🥳🥳🥳

Well, there's actually more.

You could have more than one predictor variable. Mathematically you can have up to as many predictors ($p$) as one less than the number of data points, such that $p < n - 1$, making the following formula:

$$y_i = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \dots + \beta_p x_p + e_i$$

The one above has two predictor variables, and would be solved as such:

$$y_i = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + e_i$$

So I guess what I have to say is:

I'm not delulu, I'm in my GOAT era. 🤪⏳⌛🐐 (fr)

Yes, I've regressed. But it's Simple, it's Ordinary, it's Regression.

I ghosted them, sus and mid fr. 👻👻👻