24
votes
Accepted
Feature selection vs Feature extraction. Which to use when?
Adding to The answer given by Toros,
These(see below bullets) three are quite similar but with a subtle differences-:(concise and easy to remember)
feature extraction and feature engineering: ...
- 2,510
20
votes
Accepted
Why are autoencoders for dimension reduction symmetrical?
There is no specific constraint on the symmetry of an autoencoder.
At the beginning, people tended to enforce such symmetry to the maximum: not only the layers were symmetrical, but also the weights ...
- 27.5k
18
votes
Accepted
Are t-sne dimensions meaningful?
The dimensions of the low dimensional space have no meaning. Note that the t-SNE loss function is solely based on the distances between points ($y_i$ and $y_j$) and probability distributions over ...
- 971
15
votes
What is the difference between and Embedding Layer and an Autoencoder?
Actually they are 3 different things (embedding layer, word2vec, autoencoder), though they can be used to solve similar problems. (i.e. dense representation of data)
Autoencoder is a type of neural ...
- 860
14
votes
Accepted
Improve the speed of t-sne implementation in python for huge data
You must look at this Multicore implementation of t-SNE.
I actually tried it and can vouch for its superior performance.
14
votes
Accepted
One hot encoding alternatives for large categorical values
One option is to map rare values to 'other'. This is commonly done in e.g. natural language processing - the intuition being that very rare labels don't carry much statistical power.
I have also ...
- 2,258
13
votes
Accepted
Efficient dimensionality reduction for large dataset
Have you heard of Uniform Manifold Approximation and Projection (UMAP)?
UMAP (Uniform Manifold Approximation and Projection) is a novel
manifold learning technique for non-linear dimension ...
- 4,399
9
votes
Why are autoencoders for dimension reduction symmetrical?
Your question is definitely in place, however I found that any question in the format of "should I do X or Y in deep learning ?" has only one answer.
Try them both
Deep learning is a very empirical ...
- 116
9
votes
Accepted
Why it is recommended to use T SNE to reduce to 2-3 dims and not higher dim?
Big Alarm!
T-SNE is NOT a dimensionality reduction algorithm (like PCA, LLE, UMAP, etc.). It is ONLY for visualization, and for that sake, more than 3 dimensions does not make sense.
T-SNE is not a ...
- 6,712
8
votes
Improve the speed of t-sne implementation in python for huge data
Try UMAP.
It's significantly faster than t-SNE.
8
votes
Improve the speed of t-sne implementation in python for huge data
Check out FFT-accelerated Interpolation-based t-SNE (paper, code, and Python package).
From the abstract:
We present Fast
Fourier Transform-accelerated Interpolation-based t-SNE (FIt-SNE),
...
- 180
7
votes
Reducing the dimensionality of word embeddings
There is a paper on this subject called
Simple and Effective Dimensionality Reduction for Word Embeddings, Vikas Raunak
You can read it here
You can also find the implementation here
In my opinion ...
- 171
7
votes
How to create interactive plot of thousands of images as output of t-SNE?
It is certainly possible to create interactive plots of many thousands of images, as Google has done in their interactive art t-SNE Map.
However, as far as I have found, there is not any canned way ...
- 231
7
votes
How to create interactive plot of thousands of images as output of t-SNE?
Datashader is a Python visualization library designed to handle large datasets. A tutorial to plot t-SNE with datashader can be found here.
- 22.3k
6
votes
Accepted
What does it mean by “t-SNE retains the structure of the data”?
You should break this down one step further: retaining local structure and retaining global structure.
Other well-understood methods, such as Principal Component Analysis are great at retaining ...
- 15.1k
6
votes
Accepted
Proper Understanding of Condensed Nearest Neighbor
Condensed Nearest Neighbors algorithm helps to reduce the dataset X for k-NN classification. It constructs a subset of examples which are able to correctly classify the original data set using a 1-NN ...
6
votes
Elimination of features based on high covariance without affecting performance?
$|d| \gg 0$ means there is a very strong correlation between $x_1$ and $x_2$. This means one can be expressed (almost completely) in terms of the other, thus one of two is almost redundant.
A simple ...
- 2,413
5
votes
Reducing the dimensionality of word embeddings
t-distributed stochastic neighbor embedding (t-SNE) is often used for dimensionality reduction in word embeddings. t-SNE maintains the relative relationships between the vectors.
Most often t-SNE is ...
- 22.3k
5
votes
Accepted
What is a good explanation of Non Negative Matrix Factorization?
Non-Negative Matrix Factorization (NMF) is described well in the paper by Lee and Seung, 1999.
Simply Put
NMF takes as an input a term-document matrix and generates a set of topics that represent ...
5
votes
t-SNE: Why equal data values are visually not close?
You're correct that the same values in T-SNE can be distributed across different points, the reason this happens is clear if you take a look at the algorithm that T-SNE runs across.
To address your ...
- 712
5
votes
Feature selection vs Feature extraction. Which to use when?
As Aditya said, there are 3 feature-related terms that sometimes are confused with each other. I will try and give summary explanation to each one of them:
Feature extraction: Generation of features ...
- 584
5
votes
Are dimensionality reduction techniques useful in deep learning
Deep learning does not use dimensionality reduction because deep learning itself is a useful dimensionality reduction technique. Deep learning learns a compressed, nonlinear representation of the data ...
- 22.3k
5
votes
Many things behave differently in high dimensional space
Along each direction for a unit cube, we have $2$ boundaries. To be less than $0.01$ from a bounday in a $d$-dimensional unit cube, it is not inside the cube of side length $1-2\times 0.001$ sharing ...
- 3,128
5
votes
Accepted
PCA for complex-valued data
Apparently this functionality is left out intentionally, see here. I'm afraid you have to use SVD, but that should be fairly straightforward:
...
- 1,555
5
votes
Accepted
What is major difference between different dimensionality reduction algorithms?
A detailed answer would require many pages of explanation, but I think a brief answer may point to the right direction for further research.
First of all the choice of dimensionality reduction ...
- 2,413
4
votes
Why are autoencoders for dimension reduction symmetrical?
I did some extensive experimenting to address the asked question. My experiments indicated that the encoding path (left leg of the NN) should have less, but wider layers. I usually take half as many ...
- 41
4
votes
Accepted
Applying dimensionality reduction on OneHotEncoded array
Following your example, you have different points in a 4-dimensional space. So, yes! you can use any dimensionality reduction technique, from PCA to UMAP.
In general, if your data is in a numeric ...
- 1,774
4
votes
Feature selection vs Feature extraction. Which to use when?
I think they are 2 different things,
Lets start with Feature Selection:
This technique is used for selecting the features which explain the most of the target variable(has a correlation with the ...
- 2,392
4
votes
How to handle large number of features in machine learning?
There are 4 ways I know in Python. In the following I copied the code I wrote for regression purposes. Classification would be very similar :
First: SelectKBest:
...
- 277
4
votes
Using random forest for selecting variables returns the entire dataframe
I guess the problem is in the for loop you have used
...
- 41
Only top scored, non community-wiki answers of a minimum length are eligible