57 votes
Accepted

What is dimensionality reduction? What is the difference between feature selection and extraction?

Simply put: feature selection: you select a subset of the original feature set; while feature extraction: you build a new set of features from the original feature set. Examples of feature ...
user avatar
45 votes
Accepted

How to do SVD and PCA with big data?

First of all, dimensionality reduction is used when you have many covariated dimensions and want to reduce problem size by rotating data points into new orthogonal basis and taking only axes with ...
user avatar
  • 2,771
36 votes
Accepted

Dimensionality and Manifold

What is a dimension? To put it simply, if you have a tabular data set with m rows and n columns, then the dimensionality of your data is n: What is a manifold? The simplest example is our planet ...
user avatar
24 votes
Accepted

Feature selection vs Feature extraction. Which to use when?

Adding to The answer given by Toros, These(see below bullets) three are quite similar but with a subtle differences-:(concise and easy to remember) feature extraction and feature engineering: ...
user avatar
  • 2,400
18 votes

What is dimensionality reduction? What is the difference between feature selection and extraction?

Dimensionality reduction is typically choosing a basis or mathematical representation within which you can describe most but not all of the variance within your data, thereby retaining the relevant ...
user avatar
  • 1,804
18 votes
Accepted

Are t-sne dimensions meaningful?

The dimensions of the low dimensional space have no meaning. Note that the t-SNE loss function is solely based on the distances between points ($y_i$ and $y_j$) and probability distributions over ...
user avatar
  • 961
18 votes
Accepted

Why are autoencoders for dimension reduction symmetrical?

There is no specific constraint on the symmetry of an autoencoder. At the beginning, people tended to enforce such symmetry to the maximum: not only the layers were symmetrical, but also the weights ...
user avatar
  • 15.4k
17 votes

Nearest neighbors search for very high dimensional data

I hope that the following resources might get you additional ideas toward solving the problem: 1) Research paper "Efficient K-Nearest Neighbor Join Algorithms for High Dimensional Sparse Data": http:/...
user avatar
16 votes
Accepted

Machine learning techniques for estimating users' age based on Facebook sites they like

One thing to start off with would be k-NN. The idea here is that you have a user/item matrix and for some of the users you have a reported age. The age for a person in the user item matrix might be ...
user avatar
  • 951
14 votes
Accepted

Improve the speed of t-sne implementation in python for huge data

You must look at this Multicore implementation of t-SNE. I actually tried it and can vouch for its superior performance.
user avatar
13 votes

High-dimensional data: What are useful techniques to know?

This is very broad question, which I think it's impossible to cover comprehensively in a single answer. Therefore, I think that it would be more beneficial to provide some pointers to relevant answers ...
user avatar
13 votes
Accepted

Purpose of visualizing high dimensional data?

I take Natural Language Processing as an example because that's the field that I have more experience in so I encourage others to share their insights in other fields like in Computer Vision, ...
user avatar
  • 3,260
13 votes
Accepted

One hot encoding alternatives for large categorical values

One option is to map rare values to 'other'. This is commonly done in e.g. natural language processing - the intuition being that very rare labels don't carry much statistical power. I have also ...
user avatar
  • 2,118
13 votes
Accepted

Efficient dimensionality reduction for large dataset

Have you heard of Uniform Manifold Approximation and Projection (UMAP)? UMAP (Uniform Manifold Approximation and Projection) is a novel manifold learning technique for non-linear dimension ...
user avatar
  • 3,966
12 votes

What is the difference between and Embedding Layer and an Autoencoder?

Actually they are 3 different things (embedding layer, word2vec, autoencoder), though they can be used to solve similar problems. (i.e. dense representation of data) Autoencoder is a type of neural ...
user avatar
  • 820
11 votes

Purpose of visualizing high dimensional data?

First of all your explanation about the methods are right. The point is that Embedding algorithms are not to only visualize but basically reducing the dimentionality to cope with two main problems in ...
user avatar
11 votes

Purpose of visualizing high dimensional data?

Richard Hamming is attributed with the sentence: "The purpose of computing is insight, not numbers." In this 1973 academic paper (see discussion in What is the famous data set that looks totally ...
user avatar
9 votes

Why are autoencoders for dimension reduction symmetrical?

Your question is definitely in place, however I found that any question in the format of "should I do X or Y in deep learning ?" has only one answer. Try them both Deep learning is a very empirical ...
user avatar
9 votes
Accepted

Why it is recommended to use T SNE to reduce to 2-3 dims and not higher dim?

Big Alarm! T-SNE is NOT a dimensionality reduction algorithm (like PCA, LLE, UMAP, etc.). It is ONLY for visualization, and for that sake, more than 3 dimensions does not make sense. T-SNE is not a ...
user avatar
8 votes

Dimensionality and Manifold

The dimensionality of a dataset is the number of variables used to represent it. For example, if we were interested in describing people in terms of their height and weight, our "people" dataset would ...
user avatar
  • 577
8 votes

Improve the speed of t-sne implementation in python for huge data

Check out FFT-accelerated Interpolation-based t-SNE (paper, code, and Python package). From the abstract: We present Fast Fourier Transform-accelerated Interpolation-based t-SNE (FIt-SNE), ...
user avatar
7 votes

What is dimensionality reduction? What is the difference between feature selection and extraction?

As in @damienfrancois answer feature selection is about selecting a subset of features. So in NLP it would be selecting a set of specific words (the typical in NLP is that each word represents a ...
user avatar
  • 599
7 votes

Machine learning techniques for estimating users' age based on Facebook sites they like

I recently did a similar project in Python (predicting opinions using FB like data), and had good results with the following basic process: Read in the training set (n = N) by iterating over comma-...
user avatar
7 votes

How to do SVD and PCA with big data?

Don't bother. First rule of programming- which also applies to data science: get everything working on a small test problem. so take a random sample of your data of say 100,000 rows. try different ...
user avatar
  • 731
7 votes

Improve the speed of t-sne implementation in python for huge data

Try UMAP. It's significantly faster than t-SNE.
user avatar
7 votes

How to create interactive plot of thousands of images as output of t-SNE?

It is certainly possible to create interactive plots of many thousands of images, as Google has done in their interactive art t-SNE Map. However, as far as I have found, there is not any canned way ...
user avatar
  • 231
6 votes

How to do SVD and PCA with big data?

PCA is usually implemented by computing SVD on the covariance matrix. Computing the covariance matrix is an embarrassingly parallel task, so it scales linear with the number of records, and is ...
user avatar
6 votes

Reducing the dimensionality of word embeddings

There is a paper on this subject calle Simple and Effective Dimensionality Reduction for Word Embeddings, Vikas Raunak You can read it here You can also find the implementation here In my ...
user avatar
  • 161
6 votes

Improve the speed of t-sne implementation in python for huge data

Since, there are no answers in SO, I have asked myself in github page and the issue has been closed by stating the following reply by GaelVaroquaux.. If you only want to parallelise vector ...
user avatar
  • 1,924
6 votes

How to create interactive plot of thousands of images as output of t-SNE?

Datashader is a Python visualization library designed to handle large datasets. A tutorial to plot t-SNE with datashader can be found here.
user avatar

Only top scored, non community-wiki answers of a minimum length are eligible