Questions tagged [dimensionality-reduction]

Dimensionality reduction refers to techniques for reducing many variables into a smaller number while keeping as much information as possible. One prominent method is [tag pca]

270 questions
Filter by
Sorted by
Tagged with
58 views

Bandwidth of the gaussian kernels in t-sne

I'm trying to understand t-SNE better and I was hoping someone could elaborate on how the $\sigma _i$'s are chosen. I was also wondering why they aren't just calculated in the normal way standard ...
142 views

Dimensionality reduction with prior knowledge of colinearity between features

Let's say that I have sparse feature vectors and I'd like to use dimensionality reduction in order to visualize them more easily. Dimensionality reduction techniques like PCA will estimate ...
9k views

Why are autoencoders for dimension reduction symmetrical?

I'm not an expert in autoencoders or neural networks by any means, so forgive me if this is a silly question. For the purpose of dimension reduction or visualizing clusters in high dimensional data, ...
553 views

How to deal with disconnected components in isomap?

While creating a nearest neighbor graph for isomap, there is a possibility that the graph is disconnected. In this case finding graph distances between all pairs of points will not be possible. Are ...
114 views

Could data from a test set 'leak' into predictor during PCA?

After reading this article I have got a question about PCA. Author was talking about whether to use test set while computing PCA. But, few important points to understand: 1) We should not ...
493 views

Dimensionality reduction with PCA limitations

What are the cases when we should not use PCA for dimensionality reduction and what to use in such cases?
1k views

Tensor Decomposition in TensorFlow for multinomial time series dimensionality reduction

I'm doing unsupervised learning (clustering and DR) on multinomial time series. I need to reduce dimensions for my data, which is sparse and has a lot of dimensions. I realized that some form of ...
2k views

t-SNE: Why equal data values are visually not close?

I have 200 data points that have the same values on all features. After t-SNE dimension reduction they doesn't look so equal anymore, just like this: Why aren't they on the same point in the ...
408 views

How are the positions of the output nodes determined in the Kohonen - Self Organizing Maps algorithm?

In the Cooperative stage of Kohonen's SOM, the neighborhood for a winning neuron(output node). In most cases, the neighborhood function happens to be the Gaussian Function. For example, h_j,_i = exp(...
174 views

I have n dimensional data and I want to check integrity, can I downgrade to 2 dimensional feature space via PCA and do so?

Say I have n dimensional data samples. I want to check the integrity of the features, if they are good representation of the respective classes, i.e. these features are good or not. My plan is: I ...
602 views

What are 2D dimensionality reduction algorithms good for?

It seems to me that t-SNE and other dimensionality reduction algorithms which reduce the dimensionality to two dimensions are mainly used to get an impression of the dataset. If done well, they look ...
246 views

How to Interpret the output of PCA?

I have dataset of 50000 values (rows) and 1000 variables (columns). Since this is high dimensional, I am unable to work with just DBSCAN. So I am trying to use PCA (principle component analysis). ...
1k views

PCA on Neural Networks dimensions reduction? [closed]

The dataset which was extracted from the database consists of more than 50 columns, I call these columns dimensions, can I call them dimensions? Obviously, I have to do dimension reduction on ...
450 views

Issue with Spark SVD

I have the following dataset with the dimensions: Rows: 41174 Columns: 439316 The matrix is very sparse and on this, I want to perform Dimensionality Reduction. I am using Spark's computeSVD ...
690 views

Can I do incremental learning with the sklearn implementation of Linear Discriminant Analysis

I have a large number of pictures that I would like to use LDA on. However, it requires too much memory, so I was wondering if it would be possible to make the learning incremental, using a sklearn ...
2k views

Are t-sne dimensions meaningful?

Are there any meanings for the dimensions of a t-sne embedding? Like with PCA we have this sense of linearly transformed variance maximizations but for t-sne is there intuition besides just the space ...
79 views

Finding the relation between two dimensions in a multi-dimensional problem

I have a collection of data points. Each point has 6 dimensions (x1, x2,...x6). I want to find a relation between two dimension (e.g. ...
6k views

Multi-class text classification with LSTM in Keras

I'm quite new to Deep Learning and trying to solve the problem of Multi-Class, multi-label text classification using Deep Learning. https://github.com/fchollet/keras/blob/master/examples/...
37 views

Modelling query in regression

I'm trying to build a regression model, where I see which attributes are influencing the margin. My data set looks like something below. ...
661 views

Preserving explained variance while reducing dimensionality

We have a function $f:R^N \rightarrow R$ and a set of points $D=\{ x\in R^N\}$. How is it possible to linearly lower the dimension of points to $M \ll N$ such that the fraction of explained variance* ...
200 views

Principal Component Analysis, Eigenvectors lying in the span of the observed data points?

I have been reading several papers and articles related to Principal Component Analysis (PCA) and in some of them, there is one step which is quite unclear to me (in particular (3) in [Schölkopf 1996])...
163 views

PCA on matrix with large M and N

Based on this answer, we know that we can perform build covariance matrix incrementally when there are too many observations, whereas we can perform randomised SVD when there are too many variables. ...
301 views

How to equalize the pairwise affinity perplexities when implementing t-SNE?

I'm trying to implement the t-SNE algorithm: I found that to compute the pairwise affinities, I have to follow this: My problem is computing $\sigma_i$. In the Wikipedia I found: The bandwidth of ...
2k views

How is dimensionality reduction achieved in Deep Belief Networks with Restricted Boltzmann Machines?

In neural networks and old classification methods, we usually construct an objective function to achieve dimensionality reduction. But Deep Belief Networks (DBN) with Restricted Boltzmann Machines (...
323 views

Difference between MDS and other manifold learning algorithms

From sklearn docs: Note that the purpose of the MDS is to find a low-dimensional representation of the data (here 2D) in which the distances respect well the distances in the original high-...
814 views

Dimension reduction techniques in R that do not use the full distance matrix

I try to apply non-linear dimension reduction in R. As usual in machine learning I have a large data set (100 K rows). I tried the packages RDRToolbox and ...
3k views

Multidimensional Scaling with Categorical Data

I have read the following about MDS in a book: using MDS requires an understanding of the individual feature's units; maybe we are using features that cannot be compared using the Euclidean ...
611 views

Why does "Depth = Semantic representation" in convolutional neural networks?

I was watching some videos online about convolutional networks, and the speaker was discussing the concept of running a filter over an image. He said, and it is also shown in the image below, that "...
1k views

Alternative Hunspell dictionary for stemming

I am using Hunspell to spellcheck and stem the words in my documents to reduct dimensionality. For spellchecking Hunspell works great with the default en_US dictionary by SCOWL (and friends), but not ...
5k views

Dimension Reduction - After or Before Train-Test Split

Should one apply dimensionality reduction methods to the data set before or after train-test splitting? Anyway, in case of training a model with preprocessing by dim-red, one should apply the same dim-...
263 views

Is there a particular order in which to do feature selection and sampling?

I want to use feature selection and observation subsampling on my data, for several reasons: feature selection for the usual motivations (reduce noise, decrease running time, etc.) observation ...
309 views

I am doing experiments on the high-dimensional regression. However, it is hard to obtain the practical or synthetic high-dimensional data. I have checked on UCI website as well as some papers with ...
151 views

Feature selection with linear interaction between variables and correlation with categorical response variable

I am searching for a feature selection algorithm able to select the minimum number (minimum redundancy) of relevant variables (maximum relevance) with respect to a categorical response variable. I ...
770 views

Is t-SNE just for visualization?

I have used the t-SNE algorithm to visualize my high dimensional data. However, I was wondering if this is a practical method for inference?
73 views

Could principle component decomposed coordinates value be correlated to each other?

I am wondering if we have a A= n*p matrix of samples and we run a PC decomposition on it. Say the eigenvector matrix is E, so the samples in the eigenvector space ...
2k views

Does PCA change the values of the data?

Principal Component Analysis is a means to reduce the dimensionality of data, if I understand correctly. So if I have a 1000 sample point 12 dimensional matrix and reduce it to a 1000 sample point 2 ...
3k views

Is mutual information symmetric?

Why is mutual information symmetric, meaning why does I(A,B) = I(B,A)? Isnt the definition of mutual information, I(A,B), something like "the reduction of entropy in A when given B"? P(A|B) doesnt ...
4k views

Pruning and parameter reduction for decision trees

I am trying to perform a classification using a decision tree classifier. I was wondering whether using a Feature reduction method is relevant for decision trees since they automatically use pruning? ...
2k views

feature redundancy

Why exactly does features being dependent on each other, features having high correlation with one another, mean that they would be redundant? Also, does PCA help get rid of redundant/irrelevant ...
195 views

Principal components analysis with compositional data

Another beginner question: I'm trying to do PCA on compositional data. In other words, all the variables in the group add up to 100%. I've since learned on this forum that compositional data poses a ...
582 views

Compute angle of vector in word2vec models

If I understand correctly, the most_similar function computes the cosine similarity of the vector with all other vectors and finds the closest one. The vectors ...
345 views

Can I apply Clustering algorithms to the result of Manifold Visualization Methods?

Some methods related to manifold-learning are commonly stated as good-for-visualization, such as T-SNE and self-organizing-maps (SOM). I understand that when referring specifically to "visualization" ...
4k views

Can closer points be considered more similar in T-SNE visualization?

I understand from Hinton's paper that T-SNE does a good job in keeping local similarities and a decent job in preserving global structure (clusterization). However I'm not clear if points appearing ...
60 views

Free/open interactive softwares/plugins for end-users' high-dimensional data visualization

Aside from questions about How to visualize data of a multidimensional dataset (TIMIT), the Purpose of visualizing high dimensional data? and High-dimensional data: What are useful techniques to know?,...
660 views

How to reduce dimensionality of audio data that comes in form of matrices and vectors?

I'm working on a project involved with identifying different types of sounds (such as screams, singing, and bangs) from each other. We've got our data a reasonable number of different transformations ...
7k views

What is a good explanation of Non Negative Matrix Factorization?

I am trying to find a resource to understand non-negative matrix factorization. Apart from Wikipedia, I couldn't find anything useful.
188 views

How exactly dependent variable is expressed in terms of independent variables using Partial Least Square Regression Method?

I understand the working of NIPALS algorithm but while doing the regression using PLS how exactly the relation between known and unknown is established using Principle Component Analysis. The idea is ...
16k views

Improve the speed of t-sne implementation in python for huge data

I would like to do dimensionality reduction on nearly 1 million vectors each with 200 dimensions(doc2vec). I am using TSNE ...