Tag Info

Accepted

What is dimensionality reduction? What is the difference between feature selection and extraction?

Simply put: feature selection: you select a subset of the original feature set; while feature extraction: you build a new set of features from the original feature set. Examples of feature ...
• 1,476
Accepted

How to do SVD and PCA with big data?

First of all, dimensionality reduction is used when you have many covariated dimensions and want to reduce problem size by rotating data points into new orthogonal basis and taking only axes with ...
• 2,771
Accepted

Dimensionality and Manifold

What is a dimension? To put it simply, if you have a tabular data set with m rows and n columns, then the dimensionality of your data is n: What is a manifold? The simplest example is our planet ...
• 2,750
Accepted

Feature selection vs Feature extraction. Which to use when?

Adding to The answer given by Toros, These(see below bullets) three are quite similar but with a subtle differences-:(concise and easy to remember) feature extraction and feature engineering: ...
• 2,400

What is dimensionality reduction? What is the difference between feature selection and extraction?

Dimensionality reduction is typically choosing a basis or mathematical representation within which you can describe most but not all of the variance within your data, thereby retaining the relevant ...
• 1,804
Accepted

Are t-sne dimensions meaningful?

The dimensions of the low dimensional space have no meaning. Note that the t-SNE loss function is solely based on the distances between points ($y_i$ and $y_j$) and probability distributions over ...
• 961
Accepted

Why are autoencoders for dimension reduction symmetrical?

There is no specific constraint on the symmetry of an autoencoder. At the beginning, people tended to enforce such symmetry to the maximum: not only the layers were symmetrical, but also the weights ...
• 15.4k

Nearest neighbors search for very high dimensional data

I hope that the following resources might get you additional ideas toward solving the problem: 1) Research paper "Efficient K-Nearest Neighbor Join Algorithms for High Dimensional Sparse Data": http:/...
• 6,508
Accepted

Machine learning techniques for estimating users' age based on Facebook sites they like

One thing to start off with would be k-NN. The idea here is that you have a user/item matrix and for some of the users you have a reported age. The age for a person in the user item matrix might be ...
• 951
Accepted

Improve the speed of t-sne implementation in python for huge data

You must look at this Multicore implementation of t-SNE. I actually tried it and can vouch for its superior performance.

High-dimensional data: What are useful techniques to know?

This is very broad question, which I think it's impossible to cover comprehensively in a single answer. Therefore, I think that it would be more beneficial to provide some pointers to relevant answers ...
• 6,508
Accepted

Purpose of visualizing high dimensional data?

I take Natural Language Processing as an example because that's the field that I have more experience in so I encourage others to share their insights in other fields like in Computer Vision, ...
• 3,260
Accepted

One hot encoding alternatives for large categorical values

One option is to map rare values to 'other'. This is commonly done in e.g. natural language processing - the intuition being that very rare labels don't carry much statistical power. I have also ...
• 2,118
Accepted

Efficient dimensionality reduction for large dataset

Have you heard of Uniform Manifold Approximation and Projection (UMAP)? UMAP (Uniform Manifold Approximation and Projection) is a novel manifold learning technique for non-linear dimension ...
• 3,966

What is the difference between and Embedding Layer and an Autoencoder?

Actually they are 3 different things (embedding layer, word2vec, autoencoder), though they can be used to solve similar problems. (i.e. dense representation of data) Autoencoder is a type of neural ...
• 820

Purpose of visualizing high dimensional data?

First of all your explanation about the methods are right. The point is that Embedding algorithms are not to only visualize but basically reducing the dimentionality to cope with two main problems in ...
• 6,287

Purpose of visualizing high dimensional data?

Richard Hamming is attributed with the sentence: "The purpose of computing is insight, not numbers." In this 1973 academic paper (see discussion in What is the famous data set that looks totally ...
• 1,377

Why are autoencoders for dimension reduction symmetrical?

Your question is definitely in place, however I found that any question in the format of "should I do X or Y in deep learning ?" has only one answer. Try them both Deep learning is a very empirical ...
• 116
Accepted

Why it is recommended to use T SNE to reduce to 2-3 dims and not higher dim?

Big Alarm! T-SNE is NOT a dimensionality reduction algorithm (like PCA, LLE, UMAP, etc.). It is ONLY for visualization, and for that sake, more than 3 dimensions does not make sense. T-SNE is not a ...
• 6,287

Dimensionality and Manifold

The dimensionality of a dataset is the number of variables used to represent it. For example, if we were interested in describing people in terms of their height and weight, our "people" dataset would ...
• 577

Improve the speed of t-sne implementation in python for huge data

Check out FFT-accelerated Interpolation-based t-SNE (paper, code, and Python package). From the abstract: We present Fast Fourier Transform-accelerated Interpolation-based t-SNE (FIt-SNE), ...
• 180

What is dimensionality reduction? What is the difference between feature selection and extraction?

As in @damienfrancois answer feature selection is about selecting a subset of features. So in NLP it would be selecting a set of specific words (the typical in NLP is that each word represents a ...
• 599

Machine learning techniques for estimating users' age based on Facebook sites they like

I recently did a similar project in Python (predicting opinions using FB like data), and had good results with the following basic process: Read in the training set (n = N) by iterating over comma-...
• 871

How to do SVD and PCA with big data?

Don't bother. First rule of programming- which also applies to data science: get everything working on a small test problem. so take a random sample of your data of say 100,000 rows. try different ...
• 731

Improve the speed of t-sne implementation in python for huge data

Try UMAP. It's significantly faster than t-SNE.

How to create interactive plot of thousands of images as output of t-SNE?

It is certainly possible to create interactive plots of many thousands of images, as Google has done in their interactive art t-SNE Map. However, as far as I have found, there is not any canned way ...
• 231

How to do SVD and PCA with big data?

PCA is usually implemented by computing SVD on the covariance matrix. Computing the covariance matrix is an embarrassingly parallel task, so it scales linear with the number of records, and is ...

Reducing the dimensionality of word embeddings

There is a paper on this subject calle Simple and Effective Dimensionality Reduction for Word Embeddings, Vikas Raunak You can read it here You can also find the implementation here In my ...
• 161