Concrete example to elaborate on the relationship between `linear function` and `affine function` in the context of machine learning

Question

I am learning the deeplearningbook

Section 5.1.4 says:

so the mapping from parameters to predictions is still a linear function but the mapping from features to predictions is now an affine function

Can any one give a concrete example to elaborate on the relationship between linear function and affine function in the context of machine learning?

Answer 1

There's a great answer on the Mathematics stack exchange about the distinction between linear and affine functions.

A linear function fixes the origin, whereas an affine function need not do so. An affine function is the composition of a linear function with a translation, so while the linear part fixes the origin, the translation can map it somewhere else.

In section 5.1.4, the book introduces linear regression: $$\hat{y} = \pmb{\omega}^{T} \pmb{x} + b$$

In this example, the $\hat{y} = \pmb{\omega}^{T} \pmb{x}$ part is a linear function. It has to pass through the origin. The addition of a vertical translation $b$ causes $\hat{y}$ to become an affine transformation, since it no longer has to pass through the origin.

I think the authors point out this distinction because it's technically incorrect from a purely mathematical point-of-view to describe $$\hat{y} = \pmb{\omega}^{T} \pmb{x} + b$$ as a linear function of the parameters $\pmb{x}$. It's actually an affine function, so "linear regression" may appear to be a misnomer if you have a strong background in linear algebra.

For what it's worth, I don't think this is very helpful information to include in the main text of an introductory chapter on machine learning. If you still don't understand, then just keep reading! Don't let this block you from progressing through the book.

Answer 2

For a purely mathematical discussion, here is a good post. For a specific concrete example, I use the Affine Edit Distance to determine the difference in two strings.

A common edit distance metric is the Levenshtein distance, which calculates the minimum number of character edits (including insertions or deletions) to convert one string into another string.

The Affine distance, in contrast, penalizes differences in adjacent edits with smaller distance counts. Specifically, Affine edit distance counts one edit for the first difference in two comparison strings, but then for subsequent sequential differences it counts as less than one full edit distance count.

For example suppose there are 3 characters appended to the first comparison string, and that it is otherwise identical to the second comparison string. Suppose also that the Affine distance is counting subsequent sequential edits as 0.5 increments instead of 1.

cat
catsup

Using the Levenshtein edit distance formula, we would calculate an edit distance of 3 between cat and catsup, summing 1 for each of s, u, and p.

Using Affine edit distance formula, we would calculate an edit distance of 2 between cat and catsup, summing 1 for the first change s, and 0.5 for u, and p.