Questions tagged [dimensionality-reduction]

Dimensionality reduction refers to techniques for reducing many variables into a smaller number while keeping as much information as possible. One prominent method is [tag pca]

Filter by
Sorted by
Tagged with
3
votes
0answers
58 views

Bandwidth of the gaussian kernels in t-sne

I'm trying to understand t-SNE better and I was hoping someone could elaborate on how the $\sigma _i$'s are chosen. I was also wondering why they aren't just calculated in the normal way standard ...
3
votes
1answer
142 views

Dimensionality reduction with prior knowledge of colinearity between features

Let's say that I have sparse feature vectors and I'd like to use dimensionality reduction in order to visualize them more easily. Dimensionality reduction techniques like PCA will estimate ...
16
votes
3answers
9k views

Why are autoencoders for dimension reduction symmetrical?

I'm not an expert in autoencoders or neural networks by any means, so forgive me if this is a silly question. For the purpose of dimension reduction or visualizing clusters in high dimensional data, ...
3
votes
1answer
553 views

How to deal with disconnected components in isomap?

While creating a nearest neighbor graph for isomap, there is a possibility that the graph is disconnected. In this case finding graph distances between all pairs of points will not be possible. Are ...
1
vote
2answers
114 views

Could data from a test set 'leak' into predictor during PCA?

After reading this article I have got a question about PCA. Author was talking about whether to use test set while computing PCA. But, few important points to understand: 1) We should not ...
3
votes
2answers
493 views

Dimensionality reduction with PCA limitations

What are the cases when we should not use PCA for dimensionality reduction and what to use in such cases?
2
votes
1answer
1k views

Tensor Decomposition in TensorFlow for multinomial time series dimensionality reduction

I'm doing unsupervised learning (clustering and DR) on multinomial time series. I need to reduce dimensions for my data, which is sparse and has a lot of dimensions. I realized that some form of ...
9
votes
1answer
2k views

t-SNE: Why equal data values are visually not close?

I have 200 data points that have the same values on all features. After t-SNE dimension reduction they doesn't look so equal anymore, just like this: Why aren't they on the same point in the ...
1
vote
2answers
408 views

How are the positions of the output nodes determined in the Kohonen - Self Organizing Maps algorithm?

In the Cooperative stage of Kohonen's SOM, the neighborhood for a winning neuron(output node). In most cases, the neighborhood function happens to be the Gaussian Function. For example, $$h_j,_i = exp(...
2
votes
4answers
174 views

I have n dimensional data and I want to check integrity, can I downgrade to 2 dimensional feature space via PCA and do so?

Say I have n dimensional data samples. I want to check the integrity of the features, if they are good representation of the respective classes, i.e. these features are good or not. My plan is: I ...
2
votes
1answer
602 views

What are 2D dimensionality reduction algorithms good for?

It seems to me that t-SNE and other dimensionality reduction algorithms which reduce the dimensionality to two dimensions are mainly used to get an impression of the dataset. If done well, they look ...
0
votes
1answer
246 views

How to Interpret the output of PCA?

I have dataset of 50000 values (rows) and 1000 variables (columns). Since this is high dimensional, I am unable to work with just DBSCAN. So I am trying to use PCA (principle component analysis). ...
1
vote
3answers
1k views

PCA on Neural Networks dimensions reduction? [closed]

The dataset which was extracted from the database consists of more than 50 columns, I call these columns dimensions, can I call them dimensions? Obviously, I have to do dimension reduction on ...
1
vote
1answer
450 views

Issue with Spark SVD

I have the following dataset with the dimensions: Rows: 41174 Columns: 439316 The matrix is very sparse and on this, I want to perform Dimensionality Reduction. I am using Spark's computeSVD ...
4
votes
2answers
690 views

Can I do incremental learning with the sklearn implementation of Linear Discriminant Analysis

I have a large number of pictures that I would like to use LDA on. However, it requires too much memory, so I was wondering if it would be possible to make the learning incremental, using a sklearn ...
18
votes
1answer
2k views

Are t-sne dimensions meaningful?

Are there any meanings for the dimensions of a t-sne embedding? Like with PCA we have this sense of linearly transformed variance maximizations but for t-sne is there intuition besides just the space ...
2
votes
1answer
79 views

Finding the relation between two dimensions in a multi-dimensional problem

I have a collection of data points. Each point has 6 dimensions (x1, x2,...x6). I want to find a relation between two dimension (e.g. ...
3
votes
2answers
6k views

Multi-class text classification with LSTM in Keras

I'm quite new to Deep Learning and trying to solve the problem of Multi-Class, multi-label text classification using Deep Learning. https://github.com/fchollet/keras/blob/master/examples/...
1
vote
0answers
37 views

Modelling query in regression

I'm trying to build a regression model, where I see which attributes are influencing the margin. My data set looks like something below. ...
2
votes
2answers
661 views

Preserving explained variance while reducing dimensionality

We have a function $f:R^N \rightarrow R$ and a set of points $D=\{ x\in R^N\}$. How is it possible to linearly lower the dimension of points to $M \ll N$ such that the fraction of explained variance* ...
2
votes
1answer
200 views

Principal Component Analysis, Eigenvectors lying in the span of the observed data points?

I have been reading several papers and articles related to Principal Component Analysis (PCA) and in some of them, there is one step which is quite unclear to me (in particular (3) in [Schölkopf 1996])...
1
vote
1answer
163 views

PCA on matrix with large M and N

Based on this answer, we know that we can perform build covariance matrix incrementally when there are too many observations, whereas we can perform randomised SVD when there are too many variables. ...
3
votes
2answers
301 views

How to equalize the pairwise affinity perplexities when implementing t-SNE?

I'm trying to implement the t-SNE algorithm: I found that to compute the pairwise affinities, I have to follow this: My problem is computing $\sigma_i$. In the Wikipedia I found: The bandwidth of ...
2
votes
1answer
2k views

How is dimensionality reduction achieved in Deep Belief Networks with Restricted Boltzmann Machines?

In neural networks and old classification methods, we usually construct an objective function to achieve dimensionality reduction. But Deep Belief Networks (DBN) with Restricted Boltzmann Machines (...
0
votes
1answer
323 views

Difference between MDS and other manifold learning algorithms

From sklearn docs: Note that the purpose of the MDS is to find a low-dimensional representation of the data (here 2D) in which the distances respect well the distances in the original high-...
0
votes
1answer
814 views

Dimension reduction techniques in R that do not use the full distance matrix

I try to apply non-linear dimension reduction in R. As usual in machine learning I have a large data set (100 K rows). I tried the packages RDRToolbox and ...
2
votes
1answer
3k views

Multidimensional Scaling with Categorical Data

I have read the following about MDS in a book: using MDS requires an understanding of the individual feature's units; maybe we are using features that cannot be compared using the Euclidean ...
8
votes
1answer
611 views

Why does "Depth = Semantic representation" in convolutional neural networks?

I was watching some videos online about convolutional networks, and the speaker was discussing the concept of running a filter over an image. He said, and it is also shown in the image below, that "...
1
vote
1answer
1k views

Alternative Hunspell dictionary for stemming

I am using Hunspell to spellcheck and stem the words in my documents to reduct dimensionality. For spellchecking Hunspell works great with the default en_US dictionary by SCOWL (and friends), but not ...
3
votes
2answers
5k views

Dimension Reduction - After or Before Train-Test Split

Should one apply dimensionality reduction methods to the data set before or after train-test splitting? Anyway, in case of training a model with preprocessing by dim-red, one should apply the same dim-...
5
votes
2answers
263 views

Is there a particular order in which to do feature selection and sampling?

I want to use feature selection and observation subsampling on my data, for several reasons: feature selection for the usual motivations (reduce noise, decrease running time, etc.) observation ...
1
vote
2answers
309 views

about high-dimensional regression data

I am doing experiments on the high-dimensional regression. However, it is hard to obtain the practical or synthetic high-dimensional data. I have checked on UCI website as well as some papers with ...
1
vote
0answers
151 views

Feature selection with linear interaction between variables and correlation with categorical response variable

I am searching for a feature selection algorithm able to select the minimum number (minimum redundancy) of relevant variables (maximum relevance) with respect to a categorical response variable. I ...
8
votes
4answers
770 views

Is t-SNE just for visualization?

I have used the t-SNE algorithm to visualize my high dimensional data. However, I was wondering if this is a practical method for inference?
0
votes
2answers
73 views

Could principle component decomposed coordinates value be correlated to each other?

I am wondering if we have a A= n*p matrix of samples and we run a PC decomposition on it. Say the eigenvector matrix is E, so the samples in the eigenvector space ...
1
vote
3answers
2k views

Does PCA change the values of the data?

Principal Component Analysis is a means to reduce the dimensionality of data, if I understand correctly. So if I have a 1000 sample point 12 dimensional matrix and reduce it to a 1000 sample point 2 ...
5
votes
2answers
3k views

Is mutual information symmetric?

Why is mutual information symmetric, meaning why does I(A,B) = I(B,A)? Isnt the definition of mutual information, I(A,B), something like "the reduction of entropy in A when given B"? P(A|B) doesnt ...
1
vote
1answer
4k views

Pruning and parameter reduction for decision trees

I am trying to perform a classification using a decision tree classifier. I was wondering whether using a Feature reduction method is relevant for decision trees since they automatically use pruning? ...
2
votes
2answers
2k views

feature redundancy

Why exactly does features being dependent on each other, features having high correlation with one another, mean that they would be redundant? Also, does PCA help get rid of redundant/irrelevant ...
0
votes
2answers
195 views

Principal components analysis with compositional data

Another beginner question: I'm trying to do PCA on compositional data. In other words, all the variables in the group add up to 100%. I've since learned on this forum that compositional data poses a ...
1
vote
0answers
582 views

Compute angle of vector in word2vec models

If I understand correctly, the most_similar function computes the cosine similarity of the vector with all other vectors and finds the closest one. The vectors ...
4
votes
1answer
345 views

Can I apply Clustering algorithms to the result of Manifold Visualization Methods?

Some methods related to manifold-learning are commonly stated as good-for-visualization, such as T-SNE and self-organizing-maps (SOM). I understand that when referring specifically to "visualization" ...
15
votes
1answer
4k views

Can closer points be considered more similar in T-SNE visualization?

I understand from Hinton's paper that T-SNE does a good job in keeping local similarities and a decent job in preserving global structure (clusterization). However I'm not clear if points appearing ...
2
votes
0answers
60 views

Free/open interactive softwares/plugins for end-users' high-dimensional data visualization

Aside from questions about How to visualize data of a multidimensional dataset (TIMIT), the Purpose of visualizing high dimensional data? and High-dimensional data: What are useful techniques to know?,...
4
votes
2answers
660 views

How to reduce dimensionality of audio data that comes in form of matrices and vectors?

I'm working on a project involved with identifying different types of sounds (such as screams, singing, and bangs) from each other. We've got our data a reasonable number of different transformations ...
2
votes
1answer
7k views

What is a good explanation of Non Negative Matrix Factorization?

I am trying to find a resource to understand non-negative matrix factorization. Apart from Wikipedia, I couldn't find anything useful.
4
votes
1answer
188 views

How exactly dependent variable is expressed in terms of independent variables using Partial Least Square Regression Method?

I understand the working of NIPALS algorithm but while doing the regression using PLS how exactly the relation between known and unknown is established using Principle Component Analysis. The idea is ...
25
votes
5answers
16k views

Improve the speed of t-sne implementation in python for huge data

I would like to do dimensionality reduction on nearly 1 million vectors each with 200 dimensions(doc2vec). I am using TSNE ...
2
votes
2answers
415 views

Deciding about dimensionality reduction, classification and clustering?

Could you please help me to understand it because I'm not sure if I got it correctly. Let's say I have a dataset, of persons, with 100 features, various characteristics like height, weight, age, etc. ...
27
votes
8answers
3k views

Purpose of visualizing high dimensional data?

There are many techniques for visualizing high dimension datasets, such as T-SNE, isomap, PCA, supervised PCA, etc. And we go through the motions of projecting the data down to a 2D or 3D space, so we ...