24
votes
Accepted
Feature selection vs Feature extraction. Which to use when?
Adding to The answer given by Toros,
These(see below bullets) three are quite similar but with a subtle differences-:(concise and easy to remember)
feature extraction and feature engineering: ...
- 2,460
20
votes
Accepted
Why are autoencoders for dimension reduction symmetrical?
There is no specific constraint on the symmetry of an autoencoder.
At the beginning, people tended to enforce such symmetry to the maximum: not only the layers were symmetrical, but also the weights ...
- 23.8k
18
votes
Accepted
Are t-sne dimensions meaningful?
The dimensions of the low dimensional space have no meaning. Note that the t-SNE loss function is solely based on the distances between points ($y_i$ and $y_j$) and probability distributions over ...
- 961
15
votes
What is the difference between and Embedding Layer and an Autoencoder?
Actually they are 3 different things (embedding layer, word2vec, autoencoder), though they can be used to solve similar problems. (i.e. dense representation of data)
Autoencoder is a type of neural ...
- 850
14
votes
Accepted
Improve the speed of t-sne implementation in python for huge data
You must look at this Multicore implementation of t-SNE.
I actually tried it and can vouch for its superior performance.
14
votes
Accepted
Purpose of visualizing high dimensional data?
I take Natural Language Processing as an example because that's the field that I have more experience in so I encourage others to share their insights in other fields like in Computer Vision, ...
- 3,390
14
votes
Accepted
One hot encoding alternatives for large categorical values
One option is to map rare values to 'other'. This is commonly done in e.g. natural language processing - the intuition being that very rare labels don't carry much statistical power.
I have also ...
- 2,238
13
votes
Accepted
Efficient dimensionality reduction for large dataset
Have you heard of Uniform Manifold Approximation and Projection (UMAP)?
UMAP (Uniform Manifold Approximation and Projection) is a novel
manifold learning technique for non-linear dimension ...
- 4,207
11
votes
Purpose of visualizing high dimensional data?
First of all your explanation about the methods are right. The point is that Embedding algorithms are not to only visualize but basically reducing the dimentionality to cope with two main problems in ...
- 6,532
11
votes
Purpose of visualizing high dimensional data?
Richard Hamming is attributed with the sentence: "The purpose of computing is insight, not numbers." In this 1973 academic paper (see discussion in What is the famous data set that looks totally ...
- 1,467
9
votes
Why are autoencoders for dimension reduction symmetrical?
Your question is definitely in place, however I found that any question in the format of "should I do X or Y in deep learning ?" has only one answer.
Try them both
Deep learning is a very empirical ...
- 116
9
votes
Accepted
Why it is recommended to use T SNE to reduce to 2-3 dims and not higher dim?
Big Alarm!
T-SNE is NOT a dimensionality reduction algorithm (like PCA, LLE, UMAP, etc.). It is ONLY for visualization, and for that sake, more than 3 dimensions does not make sense.
T-SNE is not a ...
- 6,532
8
votes
Improve the speed of t-sne implementation in python for huge data
Try UMAP.
It's significantly faster than t-SNE.
8
votes
Improve the speed of t-sne implementation in python for huge data
Check out FFT-accelerated Interpolation-based t-SNE (paper, code, and Python package).
From the abstract:
We present Fast
Fourier Transform-accelerated Interpolation-based t-SNE (FIt-SNE),
...
- 180
7
votes
Reducing the dimensionality of word embeddings
There is a paper on this subject called
Simple and Effective Dimensionality Reduction for Word Embeddings, Vikas Raunak
You can read it here
You can also find the implementation here
In my opinion ...
- 171
7
votes
How to create interactive plot of thousands of images as output of t-SNE?
It is certainly possible to create interactive plots of many thousands of images, as Google has done in their interactive art t-SNE Map.
However, as far as I have found, there is not any canned way ...
- 231
7
votes
How to create interactive plot of thousands of images as output of t-SNE?
Datashader is a Python visualization library designed to handle large datasets. A tutorial to plot t-SNE with datashader can be found here.
- 20.4k
6
votes
Improve the speed of t-sne implementation in python for huge data
Since, there are no answers in SO, I have asked myself in github page and the issue has been closed by stating the following reply by GaelVaroquaux..
If you only want to parallelise vector ...
- 1,954
6
votes
Accepted
What does it mean by “t-SNE retains the structure of the data”?
You should break this down one step further: retaining local structure and retaining global structure.
Other well-understood methods, such as Principal Component Analysis are great at retaining ...
- 14.7k
6
votes
Accepted
Proper Understanding of Condensed Nearest Neighbor
Condensed Nearest Neighbors algorithm helps to reduce the dataset X for k-NN classification. It constructs a subset of examples which are able to correctly classify the original data set using a 1-NN ...
6
votes
Elimination of features based on high covariance without affecting performance?
$|d| \gg 0$ means there is a very strong correlation between $x_1$ and $x_2$. This means one can be expressed (almost completely) in terms of the other, thus one of two is almost redundant.
A simple ...
- 2,301
5
votes
Accepted
What is a good explanation of Non Negative Matrix Factorization?
Non-Negative Matrix Factorization (NMF) is described well in the paper by Lee and Seung, 1999.
Simply Put
NMF takes as an input a term-document matrix and generates a set of topics that represent ...
5
votes
What is dimensionality reduction? What is the difference between feature selection and extraction?
Feature selection is about choosing some of features based on some statistical score but feature extraction is using techniques to extract some second layer information from the data e.g. interesting ...
- 163
5
votes
Accepted
Can closer points be considered more similar in T-SNE visualization?
I would present t-SNE as a smart probabilistic adaptation of the Locally-linear embedding. In both cases, we attempt to project points from a high dimensional space to a small one. This projection is ...
- 1,307
5
votes
Is mutual information symmetric?
Given the definition for mutual information
$$I(X;Y) = \sum_{y \in Y} \sum_{x \in X}
p(x,y) \log{ \left(\frac{p(x,y)}{p(x)\,p(y)}
\right) },$$
it follows from rearrangement of the summands
$$I(...
- 206
5
votes
Why does "Depth = Semantic representation" in convolutional neural networks?
The point of the layer depth and gradual pyramidal reduction is to build up a hierarchy of spatially invariant representations, each more complex than those of the prior levels. For example, at the ...
- 371
5
votes
t-SNE: Why equal data values are visually not close?
You're correct that the same values in T-SNE can be distributed across different points, the reason this happens is clear if you take a look at the algorithm that T-SNE runs across.
To address your ...
- 712
5
votes
Feature selection vs Feature extraction. Which to use when?
As Aditya said, there are 3 feature-related terms that sometimes are confused with each other. I will try and give summary explanation to each one of them:
Feature extraction: Generation of features ...
- 568
5
votes
Are dimensionality reduction techniques useful in deep learning
Deep learning does not use dimensionality reduction because deep learning itself is a useful dimensionality reduction technique. Deep learning learns a compressed, nonlinear representation of the data ...
- 20.4k
5
votes
Accepted
PCA for complex-valued data
Apparently this functionality is left out intentionally, see here. I'm afraid you have to use SVD, but that should be fairly straightforward:
...
- 1,515
Only top scored, non community-wiki answers of a minimum length are eligible