Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [dimensionality-reduction]

Dimensionality reduction refers to techniques for reducing many variables into a smaller number while keeping as much information as possible. One prominent method is [tag pca]

3
votes
1answer
28 views

What are the cases in which Isomap fails to do a good job?

As above, what is a possible scenario/ dataset/ case in which Isomap fails to do a decent dimensionality reduction?
0
votes
1answer
31 views

Curse of Dimensionality : How many columns is too many columns?

Say I have a dataset with 1000 columns and 3M rows. I know that this is will definitely suffer from Curse of Dimensionality and that I need to reduce the number of dimensions. But to what extent am I ...
0
votes
0answers
26 views

Autoencoder in R

I am new in neural network and trying to replicate autoencoder in R from https://statslab.eighty20.co.za/posts/autoencoders_keras_r/ I received an error when executing this line: ...
1
vote
1answer
24 views

Proper Understanding of Condensed Nearest Neighbor

I have a question regarding the Condensed Nearest Neighbors algorithm: Why am I returning Z, which if I understand correctly, is the array of all of the ...
0
votes
1answer
25 views

target in cluster analisys (PCA)

i m doing dimensionaly reduction using PCA. I don't understand why some dataset already had a target ad example in Iris database or other like this (https://scikit-learn.org/stable/datasets/index.html)...
0
votes
1answer
26 views

PCA in visual Analytics

I m studying visual analytics and i have a theoretical question about this topic. My professor introduced this schema in him slide For connect data to visualisation. Some topic is very easy to ...
1
vote
1answer
17 views

Is dimension reduction helpful to select features for a classification problem?

Let's say I have a data set but I don't know what features are relevant to solve a classification/regression problem. In this case, is it worth/good to use a dimension reduction algorithm and then ...
3
votes
1answer
41 views

Many things behave differently in high dimensional space

It turns out that many things behave very differently in high dimensional space. The below paragraph is picked from a book. I need extra help to understand. The book says, if you pick a random ...
1
vote
0answers
13 views

How to compare Factor and Principal Component Analysis results?

I am currently working on an assignment where I am to perform a comparison of different Dimension Reduction techniques in Python. I am using the Scikit-learn functions to perform PCA and FA. However I ...
-1
votes
2answers
28 views

Similarity measure before and after dimensionality reduction or clustering

I have a dataset with 500 000 samples, each sample contains 30 features. The values of the features are in the range 0.0 to 1.0. ...
1
vote
0answers
19 views

How does the hashing trick work in NLP

If I have 1000 words and want to reduce the dimension to 500, does that mean per hash there will be two words on average? In the course NLP specialization on courser, 40 million unique word matrix was ...
0
votes
0answers
21 views

Dimensionality reduction categories

According to what I found, dimensionality reduction has two types feature selection and feature extraction . In feature extraction, we find PCA, LDA ,LLE , ISOMAP, etc.. In other works i find random ...
0
votes
1answer
15 views

Given a 12x12 binary image (only black and white pixels) what is its dimensionality? And how can I define dimensionality of a data space?

Suppose I have a grid 12x12 of pixels that can be only black or white. I can't understand if the dimensionality is 2 or 3. I mean... Is dimension given by 12x12 or 12x12x2 ?
1
vote
0answers
22 views

When projecting data with UMAP, should I use only the samples I need projected or the entire dataset?

I have a neural network that maps my data samples to a 64-dimensional embedding. I wish to visualize a few of these embeddings (between 30 and 600) through a 2-dimensional projection, and I plan to ...
2
votes
0answers
24 views

Dimension of the manifold on which my data sits

Suppose that I have data points, in the form of vectors with binary entries. We create a metric space, or Vietoris-Rips complex, using the Hamming distance between the data points. I would like to ...
0
votes
1answer
53 views

Guidance needed with dimension reduction for clustering - some numerical, lots of categorical data

I've my data in a Pandas df with 25.000 rows and 1.500 columns without any NaNs. Of the columns about 30 contain numerical data which I standardized with StandardScaler(). The rest are cols with ...
0
votes
2answers
142 views

multivariate clustering, dimensionality reduction and data scalling for regression

I have a dataset with approximately 20000 observations consisting of 40 independent and 1 dependent variable. My initial objective is to develop a model that will predict the dependent variable. I ...
0
votes
1answer
20 views

Change values from nominal to numeric

I want to change the values of the class labels from nominal into numeric. e.g if the values of a class are {iris-setosa,iris-virginica,iris-versicolor} i want to make them {0,1,2} so the instances ...
0
votes
2answers
35 views

Extracting Useful features from large convolutional layers

I have been training a convolutional neural network on emotion detection. Now, I would like to extract features for my data to train an LSTM layer. In my case, the top convolutional layers in the ...
0
votes
1answer
30 views

How to choose PCA or KernelPCA a priori?

I am learning about dimensionality reduction and I understood that one of the most used techniques in ML is PCA. If I understood correctly, I use PCA whenever I want to reduce the number of features ...
0
votes
3answers
84 views

Spectral clustering with heat kernel weight matrix

I am studying normalized graph cuts, and one of the way to define weight matrix is using heat kernel, which is $W_{ij} = e^{\frac{−∥x_i − x_j∥^2}{σ^2}}$. I want to ask: what's the meaning of sigma? ...
1
vote
2answers
55 views

Differences between applying KMeans over PCA and applying PCA over KMeans

Short question: As stated in the title, I'm interested in the differences between applying KMeans over PCA-ed vectors and applying PCA over KMean-ed vectors. Long question: Let's suppose we have a ...
1
vote
1answer
37 views

Can anyone explain me the difference between Factor Anaysis and PCA?

Is PCA and Factor Analysis same? Both are used for Data dimension reduction but theoretically I am not able to find the difference between them? I did FA in SPSS to reduce number of variables in my ...
2
votes
0answers
84 views

Do I need to fit on train data for truncated SVD and then transform the test data on fitted train data?

Regarding truncated SVD(single value decomposition) do I need to fit on train data and transform the test data on fitted train data? or can I fit on test and transform on test fit?
3
votes
3answers
156 views

How to handle large number of features in machine learning?

I try to do normal classification on high dimensional traditional columnar data (several hundred columns). The features are of different type. In this case, it's clearly out of question to examine ...
7
votes
2answers
633 views

Efficient dimensionality reduction for large dataset

I have a dataset with ~1M rows and ~500K sparse features. I want to reduce the dimensionality to somewhere in the order of 1K-5K dense features. ...
5
votes
1answer
127 views

What does it mean by “t-SNE retains the structure of the data”?

I was learning about t-SNE when I was told that t-SNE retains the structure of the data in the embeddings. What exactly does this mean ? How does the algorithm achieve this ? So far I have ...
4
votes
3answers
118 views

Are dimensionality reduction techniques useful in deep learning

I have been working on machine learning and noticed that most of the time, dimensionality reduction techniques like PCA and t-SNE...
0
votes
1answer
123 views

Reconstructing original data points from t-SNE output

I have been trying to understand t-SNE for a while now and I have this very basic question on the comparison of PCA and t-SNE, on which I would really appreciate some help. In case of PCA suppose the ...
5
votes
2answers
276 views

Distributed PCA or an equivalent

We normally have fairly large datasets to model on, just to give you an idea: over 1M features (sparse, average population of features is around 12%); over 60M rows. A lot of modeling algorithms ...
6
votes
2answers
181 views

What does the long curve-shape t-SNE mean?

I use 1-D CNN input 1*512 size time series data which randomly fragment segment, the output will classify input into 10 classes. After training the CNN, I apply t-SNE to the prediction which I fed in ...
0
votes
0answers
5 views

How does multidimensional scaling achieve information reduction?

How does multidimensional scaling achieve information reduction? My notes only give that it transforms the data points into a new coordinate system, $\mathbb{R}^n \rightarrow \mathbb{R}^2$. But how ...
1
vote
2answers
188 views

Accuracy reduces drastically when using TruncatedSVD with hashingvector

I have around 0.8 million product description with categories. There are around 280 categories. I want to train a model with given dataset so that in future I can predict Category for the given ...
1
vote
0answers
111 views

Decision trees and Curse of Dimensionality

Since decision tree algorithm splits the training dataset one feature at a time, how the heck is possibly that it suffers from curse of dimensionality ?
3
votes
1answer
143 views

Discarding correlation among inputs in a neural network

I am working on a problem with 4 inputs and 1 continuous output variable. The sum of all values of the 4 input variables is always 1. a1+a2+a3+a4=1 So, they are correlated. My question is: ...
1
vote
1answer
86 views

Principal Component Analysis and abnormal data

I know that PCA is good in differentiating between anomalies and normal data and it helps to differentiate between them when it tries to transfer the data to another dimension. I mean it can somehow ...
4
votes
2answers
2k views

How to create interactive plot of thousands of images as output of t-SNE?

I have many images that I want to plot as a result of running t-SNE and I want to be able to interactively explore them. matplotlib does not allow enough interactivity to explore, and plotly is too ...
11
votes
4answers
11k views

Feature selection vs Feature extraction. Which to use when?

Feature extraction and feature selection essentially reduce the dimensionality of the data, but feature extraction also makes the data more separable, if I am right. Which technique would be ...
0
votes
1answer
65 views

Performace of Fischer projection as dimension reduction compared to other LDA methods

How is the performance of Fischer projection compared to other LDA methods of dimension reduction? I thought that Fischer projection was a great method of dimension reduction by maximizing class ...
1
vote
2answers
90 views

How to use pca results for linear regression

I have a data set of 11 variables with allot of observations for each one. I want to make linear regression on the variables with the observed $\vec{y}=\alpha +\beta*\vec{X}$ when X is matrix. I'm ...
1
vote
1answer
83 views

Applying machine learning algorithms to subset of attributes in dataframe

I have this huge mixed data set consisting of both numerical and categorical attributes which upon OneHotEncoding results into a data set with very high dimensionality. Is it wise to apply machine ...
4
votes
1answer
1k views

Applying dimensionality reduction on OneHotEncoded array

I have a really large data set with mixed variables. I have converted categorical variables to numerical using OneHotEncoding and it has resulted in more than a ...
3
votes
1answer
55 views

Non Deterministc Dimensionality reduction [closed]

could you please suggest me a nondeterministic algorithm for dimensionality reduction except t-SNE.
0
votes
1answer
140 views

Including the dependent variable in your data to perform principal component analysis?

Let's say you have a data set with GPA (dependent variable) and Amount of alcohol, Amount of study, IQ, and SAT score as the independent variables. And you want to perform the principal component ...
1
vote
1answer
155 views

How to choose variables for regression

I have a dataset of long/short equity hedge funds returns and their associated benchmarks (market indices). I need to form multiple regression on the fund returns using the benchmarks returns as ...
2
votes
1answer
508 views

Which dissimilarity/similarity measure use after a dimension reduction ( PCA / AutoEncoder / … )?

Each problem required its own similarity/dissimilarity measure. Imagine we are dealing with dataset composed with vector of real. I suppose that we mostly use the euclidean distance especially in low ...
0
votes
1answer
81 views

What is local-NMF? How is it better than original NMF?

I am reading this paper, but don't really understand. Do the words "part-based" or "local" for non-negative matrix factorization (NMF) mean that the algorithm aims to factorize some specific parts ...
0
votes
1answer
37 views

Some questions about feature hashing in the context of document classification

I'm trying to understand feature hashing, specifically in the context of document classification. I'm under the impression that it is useful because: it allows us to easily deal with 'new' words/...
2
votes
0answers
43 views

What should we do with the construction of a classifier (e.g., NN) if we have more number of input features?

For example, I consider an NN with n number of input units. What should the construction of NN be changed if the number of input units increases to 10*n? I think the number of hidden units and hidden ...
2
votes
3answers
64 views

How to test trained PCA used for compression?

I am working on an exercise for using PCA for compression of images and I don't quite understand how to use it on the test data: I have 300 images of hand drawn sixes, represented by 28x28 matrices, ...