Multiple Linear Regression

machine-learning
supervised-learning
regression
In univariate linear regression, you had a single feature (the size of a house) and predicted (the price). The model was:
Published

July 6, 2026

From One Feature to Many

In univariate linear regression, you had a single feature \(x\) (the size of a house) and predicted \(y\) (the price). The model was:

\[ f_{w,b}(x) = wx + b \]

But what if you had more than just the size? Suppose you also knew the number of bedrooms, the number of floors, and the age of the home in years. That extra information should help you predict the price more accurately.

This week, you will extend linear regression to handle multiple input features. The same ideas from gradient descent still apply, but the notation grows to handle lists of numbers (vectors) instead of single numbers.

Review Questions

1. In univariate linear regression, how many input features does the model use?

One. “Univariate” means a single input variable (for example, house size only).

1. Why might adding more features (bedrooms, floors, age) improve price predictions?

Each feature carries additional information about the house. A model that can use all of these signals together may fit the data better than one that only looks at size.

Notation for Multiple Features

Suppose we have four features for each house:

Feature Meaning
\(x_1\) Size (square feet)
\(x_2\) Number of bedrooms
\(x_3\) Number of floors
\(x_4\) Age (years)

We use \(x_j\) (or “feature \(j\)”) to denote the \(j\)-th feature. Here \(j\) runs from 1 to 4 because we have four features.

We use lowercase \(n\) to denote the total number of features. In this example, \(n = 4\).

Note

Do not confuse \(n\) (number of features) with \(m\) (number of training examples). In the housing dataset, \(m = 47\) houses, but each house would need more columns if we tracked bedrooms, floors, and age as well.

As before, \(\vec{x}^{(i)}\) denotes the \(i\)-th training example. With multiple features, \(\vec{x}^{(i)}\) is no longer a single number. It is a vector (a list of \(n\) numbers) containing all features for the \(i\)-th example.

For example, the second training example might be:

\[ \vec{x}^{(2)} = \begin{bmatrix} 1416 & 3 & 2 & 40 \end{bmatrix} \]

This means: 1,416 sq ft, 3 bedrooms, 2 floors, and 40 years old. The numbers are written in a row, so this is sometimes called a row vector. If you have not studied row vs column vectors before, do not worry about the distinction for now.

To refer to one specific feature within one training example, we write \(x_j^{(i)}\). For example:

\[ x_3^{(2)} = 2 \]

This is the number of floors in the second training example.

Note

In this course, vectors are written with an arrow above the symbol (for example, \(\vec{x}^{(i)}\) and \(\vec{w}\)) to distinguish them from single numbers. Individual features \(x_j^{(i)}\) and weights \(w_j\) stay without an arrow because each one is a scalar. The model is written \(f_{\vec{w},b}(\vec{x})\) when both weights and features are vectors; in the univariate case it is \(f_{w,b}(x)\) with scalar \(w\) and \(x\). In Python code, variables like x and w are plain names (NumPy arrays); the arrow is math notation only. The same vectors are sometimes written \(\mathbf{x}\) in the linear algebra course; both mean a list of numbers.

Review Questions

1. If a dataset has 5 features per house, what is \(n\)?

\(n = 5\). The variable \(n\) counts the number of features (input variables).

1. Given the vector below, what does \(x_3^{(2)}\) represent?

\[ \vec{x}^{(2)} = \begin{bmatrix} 1416 & 3 & 2 & 40 \end{bmatrix} \]

\[ x_3^{(2)} = 2 \]

This is the value of the third feature (number of floors) in the second training example.

1. Does \(\vec{x}^{(i)}\) refer to a single number or a list of numbers when we have multiple features?

A list of numbers (a vector). It contains all \(n\) feature values for the \(i\)-th training example.

Model with Multiple Features

With a single feature, the model was \(f_{w,b}(x) = wx + b\), where \(w\) and \(x\) are scalars. With multiple features, \(w\) becomes the vector \(\vec{w}\) and \(x\) becomes \(\vec{x}\). We write \(f_{\vec{w},b}(\vec{x})\) to reflect that. Expanded out, we give each feature its own weight:

\[ f_{\vec{w},b}(\vec{x}) = w_1 x_1 + w_2 x_2 + w_3 x_3 + w_4 x_4 + b \]

Each \(w_j\) is a parameter that controls how strongly feature \(j\) influences the prediction. The bias \(b\) is still a single number.

In general, with \(n\) features:

\[ f_{\vec{w},b}(\vec{x}) = w_1 x_1 + w_2 x_2 + \cdots + w_n x_n + b \]

Here \(\vec{x}\) is the vector of input features \([x_1, x_2, \ldots, x_n]\), and \(w_1, w_2, \ldots, w_n\) are the corresponding weights.

Review Questions

1. How does the multi-feature model differ from the univariate model below?

\[ f_{w,b}(x) = wx + b \]

Instead of one weight \(w\) times one feature \(x\), the multi-feature model uses a separate weight \(w_j\) for each feature \(x_j\):

\[ f_{\vec{w},b}(\vec{x}) = w_1 x_1 + w_2 x_2 + \cdots + w_n x_n + b \]

1. Consider the model below. How many parameters are in the weight list (not counting \(b\))?

\[ f_{\vec{w},b}(\vec{x}) = w_1 x_1 + w_2 x_2 + w_3 x_3 + w_4 x_4 + b \]

Four:

\[ w_1, \; w_2, \; w_3, \; w_4 \]

One weight for each of the four features.

Interpreting Parameters

Here is one possible model for predicting house prices (in thousands of dollars):

\[ f_{\vec{w},b}(\vec{x}) = 0.1\, x_1 + 4\, x_2 + 10\, x_3 - 2\, x_4 + 80 \]

How do we read these numbers?

  • \(b = 80\): The base price starts at $80,000 (before adding contributions from size, bedrooms, floors, or age). This is a mathematical baseline, not a price you would see for a real house with zero size.
  • \(w_1 = 0.1\): Each additional square foot adds \(0.1 \times \$1{,}000 = \$100\) to the predicted price.
  • \(w_2 = 4\): Each additional bedroom adds $4,000.
  • \(w_3 = 10\): Each additional floor adds $10,000.
  • \(w_4 = -2\): Each additional year of age subtracts $2,000 from the price (because the weight is negative).
Predicted price for x^(2): $173.6k

For \(\vec{x}^{(2)} = [1416, 3, 2, 40]\), this model predicts a price of about $173.6k. You can verify by hand: \(0.1(1416) + 4(3) + 10(2) - 2(40) + 80 = 173.6\).

Review Questions

1. If prices are in thousands of dollars, what happens to the predicted price when the house is one year older?

\[ w_4 = -2 \]

The prediction decreases by $2,000. A negative weight means that feature pushes the prediction down as its value increases.

1. If prices are in thousands of dollars, how much does the prediction change for each additional square foot?

\[ w_1 = 0.1 \]

It increases by $100 per square foot:

\[ 0.1 \times \$1{,}000 = \$100 \]

1. What does the bias term represent in this example?

\[ b = 80 \]

It is the starting baseline of $80,000 before the model adds contributions from the four features. It sets the intercept of the model.

Compact Form with Vectors and Dot Product

Writing out \(w_1 x_1 + w_2 x_2 + \cdots + w_n x_n\) every time gets tedious. We can pack the weights and features into vectors.

Let \(\vec{w}\) be the vector of parameters:

\[ \vec{w} = \begin{bmatrix} w_1 & w_2 & \cdots & w_n \end{bmatrix} \]

Let \(\vec{x}\) be the vector of features:

\[ \vec{x} = \begin{bmatrix} x_1 & x_2 & \cdots & x_n \end{bmatrix} \]

The bias \(b\) is still a single number (not a vector). Together, \(\vec{w}\) and \(b\) are the parameters of the model.

Now the model can be written compactly using the dot product (also called the inner product) from vector algebra:

\[ f_{\vec{w},b}(\vec{x}) = \vec{w} \cdot \vec{x} + b \]

The dot product \(\vec{w} \cdot \vec{x}\) multiplies corresponding pairs and sums the results:

\[ \vec{w} \cdot \vec{x} = w_1 x_1 + w_2 x_2 + w_3 x_3 + \cdots + w_n x_n \]

So \(\vec{w} \cdot \vec{x} + b\) is exactly the same as the long form above. The dot product notation is just a shorter way to write the same thing.

Long form:  173.6
Dot product: 173.6
Equal? True

Review Questions

1. What does the dot product below compute?

\[ \vec{w} \cdot \vec{x} \]

\[ \vec{w} \cdot \vec{x} = w_1 x_1 + w_2 x_2 + w_3 x_3 + \cdots + w_n x_n \]

It multiplies each pair of corresponding entries and sums all of those products.

1. Are these two expressions different models?

\[ f_{\vec{w},b}(\vec{x}) = \vec{w} \cdot \vec{x} + b \]

\[ f_{\vec{w},b}(\vec{x}) = w_1 x_1 + w_2 x_2 + \cdots + w_n x_n + b \]

No. They are the same model written in two equivalent ways. The dot product version is just more compact.

1. Which of the following is a vector, and which is a scalar?

\[ \vec{w}, \quad b, \quad \vec{x} \]

\(\vec{w}\) and \(\vec{x}\) are vectors (lists of numbers). \(b\) is a scalar (a single number).

Multiple Linear Regression

The name for linear regression with multiple input features is multiple linear regression. This is different from univariate linear regression, which uses just one feature.

You might hear the term “multivariate regression” in other contexts, but in this course that term refers to something else (predicting multiple output variables at once). We will stick with multiple linear regression for predicting one output \(y\) from many input features.

Everything you learned so far still applies:

  • The model predicts \(\hat{y} = f_{\vec{w},b}(\vec{x})\)
  • You train it by minimizing a cost function (like the squared error from the cost function section)
  • Gradient descent can find good values for all of the \(w_j\) parameters and \(b\)

The main change is that you now have many weights to update instead of just one.

In the next section, you will learn about vectorization, a technique that lets you implement multiple linear regression (and many other algorithms) much more efficiently in code.

Review Questions

1. What is the difference between univariate and multiple linear regression?

Univariate regression uses one input feature. Multiple linear regression uses two or more input features, with a separate weight \(w_j\) for each feature.

1. Why is “multivariate regression” not the name used in this course for this model?

In other fields, “multivariate regression” often means predicting multiple output variables at once. Here we still predict a single output \(y\) (price) from many inputs, so the course uses “multiple linear regression” instead.

1. Does gradient descent still work when the model has many features?

Yes. Gradient descent can minimize the cost function for any number of parameters:

\[ w_1, \; w_2, \; \ldots, \; w_n, \; b \]

You just update more weights at each step.