I have been looking at feature agglomeration in Python's scikit-learn. According to the user guide, feature agglomeration "applies Hierarchical clustering to group together features that behave similarly". Does this mean it is testing for interactions between features and groups features that do interact together? What does "behave similarly" mean in this context?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
From the documentation:
Similar to AgglomerativeClustering, but recursively merges features instead of samples.
In standard agglomerative clustering you receive a matrix $M^{n \times m}$ representing $n$ samples of dimension $m$ that you want to cluster. In feature agglomeration the algorithm clusters the transpose of the matrix, i.e. $M^T$ so it clusters $m$ samples of dimension $n$, these samples represent the features.
The default distance used to cluster the features (samples in the transpose matrix) is the euclidean distance, but you can also use l1, cosine and others.
For example suppose you have 3 samples of dimension 3 (a matrix 3x3 matrix)::
----------------------------------
| feature1 | feature2 | feature3 |
----------------------------------
| 1 | 1.05 | 10 |
----------------------------------
| 1 | 1.05 | 10 |
----------------------------------
| 2 | 2.05 | 20 |
----------------------------------
If you want to reduce the dimension of your dataset to 2 dimensions, the algorithm clusters together feature1 and feature2, and leaves feature3 unchanged, the new matrix is:
----------------------------------
| feature1 - feature2 | feature3 |
----------------------------------
| 1.025 | 10 |
----------------------------------
| 1.025 | 10 |
----------------------------------
| 2.025 | 20 |
----------------------------------
The resulting feature is determined by the pooling function in the case above it computes the arithmetic mean. See this for a more in depth usage example.
-
3$\begingroup$ Thanks for the explanation. I feel python should have this explanation because someone who is new to feature cluster can easily get lost $\endgroup$ Oct 11, 2018 at 0:17
-
$\begingroup$ There is a "Read more in the User Guide" link in the docs that has good additional detail $\endgroup$– PaulOct 13, 2022 at 2:18