A Comprehensive Guide to Interaction Terms in Linear Regression

Introduction

This post provides an intuitive explanation of interaction terms in the context of linear regression. Interaction terms are a crucial component of regression analysis, and understanding how they work can help practitioners better train models and interpret their data.

Model Specification with Two Features

The model specification with two features can be described by the following equation: Y = β0 + β1X1 + β2X2 + ε, where Y is the dependent variable, X1 and X2 are the independent variables, β0 is the intercept, β1 and β2 are the coefficients of the linear model, and ε is the error term.

Example

To make the explanation easier to understand, the post provides an example of the apartment pricing in two different locations. After gathering data and estimating a linear regression model, the post obtains the following coefficients:

β0 = 100, β1 = 50, β2 = 20

Knowing that X2 is a Boolean feature, the post writes out the two possible scenarios depending on the value of X2. Interaction terms represent joint effects.

Interpreting the Coefficients with Interaction Terms

Adding interaction terms to a model changes the interpretation of all the coefficients. Without an interaction term, the coefficients are interpreted as the unique effect of a predictor on the dependent variable. However, with an interaction term, the effect of the predictor varies depending on the value of another feature.

Interpreting the Coefficients with Interaction Terms - Continued

In the apartment pricing example, an additional square meter in an apartment in the city center costs more than an additional square meter in an apartment on the outskirts. Therefore, not only the intercept but also the slope of the lines should be different between the two scenarios. The effect of the apartment’s size is different for different values of X2.

Constructing Interaction Terms

To test for the joint effect of two variables, an interaction term is added to the regression model. For example, in the car pricing example, an interaction term is constructed by multiplying the mileage of the car by a dummy variable that denotes whether it has an automatic or manual transmission.

Predictive Performance

By using interaction terms, the specification of a linear model becomes more flexible, which can result in a better fit to the data and better predictive performance.

Conclusion

Interaction terms enable the examination of whether the relationship between the target and a feature changes depending on the value of another feature. By using interaction terms, the specification of a linear model becomes more flexible, resulting in a better fit to the data and better predictive performance.