# Questions tagged [dimensionality-reduction]

Dimensionality reduction refers to techniques for reducing many variables into a smaller number while keeping as much information as possible. One prominent method is [tag pca]

270 questions
Filter by
Sorted by
Tagged with
58 views

### Bandwidth of the gaussian kernels in t-sne

I'm trying to understand t-SNE better and I was hoping someone could elaborate on how the $\sigma _i$'s are chosen. I was also wondering why they aren't just calculated in the normal way standard ...
142 views

### Dimensionality reduction with prior knowledge of colinearity between features

Let's say that I have sparse feature vectors and I'd like to use dimensionality reduction in order to visualize them more easily. Dimensionality reduction techniques like PCA will estimate ...
9k views

### Why are autoencoders for dimension reduction symmetrical?

I'm not an expert in autoencoders or neural networks by any means, so forgive me if this is a silly question. For the purpose of dimension reduction or visualizing clusters in high dimensional data, ...
553 views

### How to deal with disconnected components in isomap?

While creating a nearest neighbor graph for isomap, there is a possibility that the graph is disconnected. In this case finding graph distances between all pairs of points will not be possible. Are ...
114 views

### Could data from a test set 'leak' into predictor during PCA?

After reading this article I have got a question about PCA. Author was talking about whether to use test set while computing PCA. But, few important points to understand: 1) We should not ...
493 views

### Dimensionality reduction with PCA limitations

What are the cases when we should not use PCA for dimensionality reduction and what to use in such cases?
1k views

### Tensor Decomposition in TensorFlow for multinomial time series dimensionality reduction

I'm doing unsupervised learning (clustering and DR) on multinomial time series. I need to reduce dimensions for my data, which is sparse and has a lot of dimensions. I realized that some form of ...
2k views

### t-SNE: Why equal data values are visually not close?

I have 200 data points that have the same values on all features. After t-SNE dimension reduction they doesn't look so equal anymore, just like this: Why aren't they on the same point in the ...
408 views

### How are the positions of the output nodes determined in the Kohonen - Self Organizing Maps algorithm?

In the Cooperative stage of Kohonen's SOM, the neighborhood for a winning neuron(output node). In most cases, the neighborhood function happens to be the Gaussian Function. For example, h_j,_i = exp(...
174 views

### I have n dimensional data and I want to check integrity, can I downgrade to 2 dimensional feature space via PCA and do so?

Say I have n dimensional data samples. I want to check the integrity of the features, if they are good representation of the respective classes, i.e. these features are good or not. My plan is: I ...
602 views

### What are 2D dimensionality reduction algorithms good for?

It seems to me that t-SNE and other dimensionality reduction algorithms which reduce the dimensionality to two dimensions are mainly used to get an impression of the dataset. If done well, they look ...
246 views

### How to Interpret the output of PCA?

I have dataset of 50000 values (rows) and 1000 variables (columns). Since this is high dimensional, I am unable to work with just DBSCAN. So I am trying to use PCA (principle component analysis). ...
1k views

### PCA on Neural Networks dimensions reduction? [closed]

The dataset which was extracted from the database consists of more than 50 columns, I call these columns dimensions, can I call them dimensions? Obviously, I have to do dimension reduction on ...
450 views

### Issue with Spark SVD

I have the following dataset with the dimensions: Rows: 41174 Columns: 439316 The matrix is very sparse and on this, I want to perform Dimensionality Reduction. I am using Spark's computeSVD ...
690 views

### Can I do incremental learning with the sklearn implementation of Linear Discriminant Analysis

I have a large number of pictures that I would like to use LDA on. However, it requires too much memory, so I was wondering if it would be possible to make the learning incremental, using a sklearn ...
2k views

### Are t-sne dimensions meaningful?

Are there any meanings for the dimensions of a t-sne embedding? Like with PCA we have this sense of linearly transformed variance maximizations but for t-sne is there intuition besides just the space ...
79 views

### Finding the relation between two dimensions in a multi-dimensional problem

I have a collection of data points. Each point has 6 dimensions (x1, x2,...x6). I want to find a relation between two dimension (e.g. ...
6k views

### Multi-class text classification with LSTM in Keras

I'm quite new to Deep Learning and trying to solve the problem of Multi-Class, multi-label text classification using Deep Learning. https://github.com/fchollet/keras/blob/master/examples/...
37 views

### Modelling query in regression

I'm trying to build a regression model, where I see which attributes are influencing the margin. My data set looks like something below. ...
661 views

### Preserving explained variance while reducing dimensionality

We have a function $f:R^N \rightarrow R$ and a set of points $D=\{ x\in R^N\}$. How is it possible to linearly lower the dimension of points to $M \ll N$ such that the fraction of explained variance* ...
200 views

### Principal Component Analysis, Eigenvectors lying in the span of the observed data points?

I have been reading several papers and articles related to Principal Component Analysis (PCA) and in some of them, there is one step which is quite unclear to me (in particular (3) in [Schölkopf 1996])...
163 views

### PCA on matrix with large M and N

Based on this answer, we know that we can perform build covariance matrix incrementally when there are too many observations, whereas we can perform randomised SVD when there are too many variables. ...
301 views

### How to equalize the pairwise affinity perplexities when implementing t-SNE?

I'm trying to implement the t-SNE algorithm: I found that to compute the pairwise affinities, I have to follow this: My problem is computing $\sigma_i$. In the Wikipedia I found: The bandwidth of ...
2k views

### How is dimensionality reduction achieved in Deep Belief Networks with Restricted Boltzmann Machines?

In neural networks and old classification methods, we usually construct an objective function to achieve dimensionality reduction. But Deep Belief Networks (DBN) with Restricted Boltzmann Machines (...
323 views

### Difference between MDS and other manifold learning algorithms

From sklearn docs: Note that the purpose of the MDS is to find a low-dimensional representation of the data (here 2D) in which the distances respect well the distances in the original high-...
814 views

### Dimension reduction techniques in R that do not use the full distance matrix

I try to apply non-linear dimension reduction in R. As usual in machine learning I have a large data set (100 K rows). I tried the packages RDRToolbox and ...
3k views

### Multidimensional Scaling with Categorical Data

I have read the following about MDS in a book: using MDS requires an understanding of the individual feature's units; maybe we are using features that cannot be compared using the Euclidean ...
611 views

### Why does "Depth = Semantic representation" in convolutional neural networks?

I was watching some videos online about convolutional networks, and the speaker was discussing the concept of running a filter over an image. He said, and it is also shown in the image below, that "...
1k views

### Alternative Hunspell dictionary for stemming

I am using Hunspell to spellcheck and stem the words in my documents to reduct dimensionality. For spellchecking Hunspell works great with the default en_US dictionary by SCOWL (and friends), but not ...
5k views

### Dimension Reduction - After or Before Train-Test Split

Should one apply dimensionality reduction methods to the data set before or after train-test splitting? Anyway, in case of training a model with preprocessing by dim-red, one should apply the same dim-...
263 views

### Is there a particular order in which to do feature selection and sampling?

I want to use feature selection and observation subsampling on my data, for several reasons: feature selection for the usual motivations (reduce noise, decrease running time, etc.) observation ...
309 views

I am doing experiments on the high-dimensional regression. However, it is hard to obtain the practical or synthetic high-dimensional data. I have checked on UCI website as well as some papers with ...
151 views

### Feature selection with linear interaction between variables and correlation with categorical response variable

I am searching for a feature selection algorithm able to select the minimum number (minimum redundancy) of relevant variables (maximum relevance) with respect to a categorical response variable. I ...
770 views

### Is t-SNE just for visualization?

I have used the t-SNE algorithm to visualize my high dimensional data. However, I was wondering if this is a practical method for inference?
73 views

### Could principle component decomposed coordinates value be correlated to each other?

I am wondering if we have a A= n*p matrix of samples and we run a PC decomposition on it. Say the eigenvector matrix is E, so the samples in the eigenvector space ...
2k views

### Does PCA change the values of the data?

Principal Component Analysis is a means to reduce the dimensionality of data, if I understand correctly. So if I have a 1000 sample point 12 dimensional matrix and reduce it to a 1000 sample point 2 ...
3k views

### Is mutual information symmetric?

Why is mutual information symmetric, meaning why does I(A,B) = I(B,A)? Isnt the definition of mutual information, I(A,B), something like "the reduction of entropy in A when given B"? P(A|B) doesnt ...
4k views

### Pruning and parameter reduction for decision trees

I am trying to perform a classification using a decision tree classifier. I was wondering whether using a Feature reduction method is relevant for decision trees since they automatically use pruning? ...
2k views

### feature redundancy

Why exactly does features being dependent on each other, features having high correlation with one another, mean that they would be redundant? Also, does PCA help get rid of redundant/irrelevant ...
195 views

### Principal components analysis with compositional data

Another beginner question: I'm trying to do PCA on compositional data. In other words, all the variables in the group add up to 100%. I've since learned on this forum that compositional data poses a ...
582 views

### Compute angle of vector in word2vec models

If I understand correctly, the most_similar function computes the cosine similarity of the vector with all other vectors and finds the closest one. The vectors ...
345 views

### Can I apply Clustering algorithms to the result of Manifold Visualization Methods?

Some methods related to manifold-learning are commonly stated as good-for-visualization, such as T-SNE and self-organizing-maps (SOM). I understand that when referring specifically to "visualization" ...
4k views

### Can closer points be considered more similar in T-SNE visualization?

I understand from Hinton's paper that T-SNE does a good job in keeping local similarities and a decent job in preserving global structure (clusterization). However I'm not clear if points appearing ...
60 views

### Free/open interactive softwares/plugins for end-users' high-dimensional data visualization

Aside from questions about How to visualize data of a multidimensional dataset (TIMIT), the Purpose of visualizing high dimensional data? and High-dimensional data: What are useful techniques to know?,...
660 views

### How to reduce dimensionality of audio data that comes in form of matrices and vectors?

I'm working on a project involved with identifying different types of sounds (such as screams, singing, and bangs) from each other. We've got our data a reasonable number of different transformations ...
7k views

### What is a good explanation of Non Negative Matrix Factorization?

I am trying to find a resource to understand non-negative matrix factorization. Apart from Wikipedia, I couldn't find anything useful.
188 views

### How exactly dependent variable is expressed in terms of independent variables using Partial Least Square Regression Method?

I understand the working of NIPALS algorithm but while doing the regression using PLS how exactly the relation between known and unknown is established using Principle Component Analysis. The idea is ...
16k views

### Improve the speed of t-sne implementation in python for huge data

I would like to do dimensionality reduction on nearly 1 million vectors each with 200 dimensions(doc2vec). I am using TSNE ...
415 views

### Deciding about dimensionality reduction, classification and clustering?

Could you please help me to understand it because I'm not sure if I got it correctly. Let's say I have a dataset, of persons, with 100 features, various characteristics like height, weight, age, etc. ...
3k views

### Purpose of visualizing high dimensional data?

There are many techniques for visualizing high dimension datasets, such as T-SNE, isomap, PCA, supervised PCA, etc. And we go through the motions of projecting the data down to a 2D or 3D space, so we ...