# Meaning of axes in Linear Discrimination Analysis

Looking at the LDA of the Iris Dataset, which looks like this:

It's understandable that the 3 types of flowers have succesfuly been seperated into categories, with versicolor and virginica slightly overlapping. My question is, how does this help us in a practical way? What can we do with the information, now that we've seperated the 3 categories like this. It's great to know that Setosa is apparent for values between 5 and 10, but what do those values mean?

You could be asking yourself the same questions using PCA! Usually these linear transformations (or even non-linear for that matter, PCA, LDA, ICA, UMAP etc.) are used to reduce the high dimensional features space to serve one or multiple purposes. For example in PCA one could use it to compress the high-dimensional feature space to lower (usually 2 or 3) by finding the directions of maximal variance, either for better separation or a simple compression or better explainability via projected visualization for human eyes. Sometimes they are served for better class separation like the case of LDA, in a supervised fashion (in contrast to PCA). In case you missed it check out Sebastian Raschka's post on the difference between LDA and PCA for dimensionality reduction (be careful with LDA assumptions).

In order to answer the question about the axes, we better touch briefly on what LDA is aiming to do, borrowing the explanation from this blog post:

LDA aims to find a new feature space to project the data in order to maximize classes separability. The first step is to find a way to measure this capacity of separation of each new feature space candidate. In 1988, a statistician called Ronald Fisher proposed the following solution: Maximize the function that represents the difference between the means, normalized by a measure of the within-class variability. The Fisher’s propose is basically to maximize the distance between the mean of each class and minimize the spreading within the class itself. Thus, we come up with two measures: the within-class and the between-class.

[The Answer]: As in the case of PCA: the first axis (by convention PCA 1) represents the axis holding the maximum variance in the data, PCA 2 the second in the rank and so forth. The same analogy holds true to LDA. Here though, LDA 1 (your x-axis in the graph, again by convention and the visual judgement) accounts for the maximum variation among the classes, and LDA 2 the second and so forth (still confused, you might one to check this video). You can define a simple linear model to separate them, that is it, that perhaps you couldn't using the original features.

[Bonus]: Although I am finished with the answer, and the following it is not super relevant, but I would like to bring a super nice example what these techniques might be able to answer from data. In the case of, again PCA, Tableau team is showing such capabilities, see this video. In real world problems, not a simple IRIS, these will come handy, and surely these possible even outside Tableau, but requires more efforts.

Intrinsically the values don't mean anything specific: it is the result of the axis combination. And most of the time, the components hold more information than the projection of the points on the new axis system.

We can easily think of an example where LDA components are informative. Imagine you have multiple features for human bodies (weight, height, length of arms, etc) and you want to separate the obese people (1) from not obese people (0). Weight is not very helpful to separate people, nor is Height on.

But LDA would probably give you a first axis having a strong composition of weight and height. What does that mean? That combination of height and weight gives you a better way to determine if a person is obese or not.

It could be informative for you, as an analyst or for a classification algorithm later on.

• After playing around with it a bit more, I came to understand it pretty well. Basically to create an LDA Model, one would need to know the features and categories of objects beforehand. We can then use LDA to seperate the objects. Then, we can use the LDA Model to predict how to categorize new objects, that we don't know the category of, but do know the features of. Commented Apr 3, 2020 at 9:23