Questions tagged [pca]

Principal component analysis, a technique for dimensionality reduction.

Filter by
Sorted by
Tagged with
13
votes
2answers
3k views

How many dimensions to reduce to when doing PCA?

How to choose K for PCA? K is the number of dimensions to project down to. The only requirement is to not lose too much information. I understand it depends on the data, but I'm looking more for a ...
11
votes
3answers
5k views

How do I make an interactive PCA scatterplot in Python?

The matplotlib library is very capable but lacks interactiveness, especially inside Jupyter Notebook. I would like a good offline plotting tool like plot.ly.
10
votes
4answers
7k views

Is PCA considered a machine learning algorithm

I've understood that principal component analysis is a dimensionality reduction technique i.e. given 10 input features, it will produce a smaller number of independent features that are orthogonal and ...
9
votes
4answers
8k views

Classify multivariate time series

I have a set of data composed of time series (8 points) with about 40 dimensions (so each time series is 8 by 40). The corresponding ouput (the possible outcomes for the categories ) is eitheir 0 or 1....
7
votes
2answers
679 views

Understanding how distributed PCA works

As part of big data analysis project, I'm working on, I need to perform PCA on some data, using cloud computing system. In my case, I'm using Amazon EMR for the job and Spark in particular. Leaving ...
7
votes
1answer
407 views

Is it OK to try to find the best PCA k parameter as we do with other hyperparameters?

Principal Component Analysis (PCA) is used to reduce n-dimensional data to k-dimensional data to speed things up in machine learning. After PCA is applied, one can check how much of the variance of ...
7
votes
1answer
827 views

Interpreting the results of randomized PCA in scikit-learn

I'm using scikit-learn to do a genome-wide association study with a feature vector of about 100K SNPs. My goal is to tell the biologists which SNPs are "interesting". RandomizedPCA really improved ...
5
votes
2answers
2k views

Does it make sense to combine PCA with an artificial neural network?

I have a Dataset of around 200 features. Most of them are categorical and only a few are numerical. It seems that an artificial neural network with an Autoencoder has some problems with that kind and ...
5
votes
2answers
3k views

Is Overfitting a problem in Unsupervised learning?

I come to this question as I read the use of PCA to reduce overfitting is a bad practice. That is because PCA does not consider labels/output classes and so Regularization is always preferred. That ...
5
votes
1answer
112 views

Intuition behind PCA eigenvectors

For undergraduate students who understand the definition of eigenvectors and eigenvalues, $$A v = \lambda v \;,$$ what is the intuition behind why the eigenvectors of the covariance (or correlation) ...
5
votes
1answer
183 views

Sklearn PCA with zero components example

I'm simply trying to repeat a benchmark from the sklearn's docs. The unclear part is: n_components = np.arange(0, n_features, 5). They are applying a PCA transform ...
5
votes
2answers
393 views

Distributed PCA or an equivalent

We normally have fairly large datasets to model on, just to give you an idea: over 1M features (sparse, average population of features is around 12%); over 60M rows. A lot of modeling algorithms ...
5
votes
1answer
63 views

How do I interpret my result of clustering?

I am working on a clustering problem. I have 11 features. My complete data frame has 70-80% zeros. The data had outliers that I capped at 0.5 and 0.95 percentile. However, I tried k-means (python) on ...
5
votes
2answers
1k views

Converting non-numeric data values into equivalent rank scores

Consider a data-frame similar to the one shown (the actual data-frame is much larger) ...
4
votes
3answers
148 views

Are dimensionality reduction techniques useful in deep learning

I have been working on machine learning and noticed that most of the time, dimensionality reduction techniques like PCA and t-SNE...
4
votes
1answer
790 views

clustering before or after PCA?

I'm newbie into data science, and I had some problems dealing with my project. I'm trying to visualize multidimensional data into 2D after clustering with using a lot of methods. (kmeans, DBSCAN, ...
4
votes
1answer
3k views

How to interpret the loading values of a pca?

Imagine I've the following matrix, which gives the grades of students in the subjects German, Philosophy, Math and Physics: ...
4
votes
3answers
3k views

tsne for prediction

I have a traditional prediction setting, with a training data set train and a test data set test. I do not know the outcome <...
3
votes
2answers
2k views

How can give weight to feature before PCA

I wonder how can I give weight to my feature before employing PCA. I mean somehow weighted PCA. Because I know that one of the features is better than others and want to give importance to it in ...
3
votes
3answers
401 views

What statistical method can be applied in my case?

I want to study the impact of 26 parameters on one variable, and therefore determine the 3 or 4 which have to most influence on it. For that, I have constructed a 10 x 26 matrix: 26 parameters with 10 ...
3
votes
2answers
5k views

Why does PCA assume Gaussian Distribution?

From Jon Shlens's A Tutorial on Principal Component Analysis - version 1, page 7, section 4.5, II: The formalism of sufficient statistics captures the notion that the mean and the variance ...
3
votes
2answers
262 views

Why do we choose principal components based on maximum variance explained?

I've seen many people choose # of principal components for PCA based on maximum variance explained. So my question is do we always have to choose principal components based on maximum variance ...
3
votes
1answer
2k views

Data scaling before PCA: how to deal with categorical values?

I have to apply PCA on a dataset, which contains both numerical and categorical values. In the preprocessing phase, I converted all the categorical values in numerical, so that the software can deal ...
3
votes
2answers
180 views

Best ML technique to suggest predictor variables

In the following dataset, the first 4 columns are predictor variables and the engine running index is the response variable. ...
3
votes
2answers
444 views

Dimensionality reduction with PCA limitations

What are the cases when we should not use PCA for dimensionality reduction and what to use in such cases?
3
votes
2answers
176 views

What are some methods for clustering individuals into distinct groups based on Features?

I started with a dataset that contained many dimensions for individuals (each id is a separate individual), and extracted three Features/Attributes columns for each ...
3
votes
1answer
1k views

PCA, SMOTE and cross validation- how to combine them together?

I was reading a lot recently about PCA and cross validation and it seems that the majority call it malpractice to do PCA before cross validation. I would also like to perform SMOTE, but there is a ...
3
votes
1answer
82 views

Many things behave differently in high dimensional space

It turns out that many things behave very differently in high dimensional space. The below paragraph is picked from a book. I need extra help to understand. The book says, if you pick a random ...
3
votes
1answer
30 views

Ploting eigenvectors

I've generated two clouds of 3d points from multivariate_normal ...
3
votes
1answer
1k views

How to implement PCA color augmentation as discussed in AlexNet

I read through "ImageNet Classification with Deep Convolutional Neural Networks" again specifically for details on how to implement PCA color augmentation. I am unsure if I have it right. Here is how ...
3
votes
1answer
90 views

PCA formulation - Deep Learning book by Ian Goodfellow

I am reading this deep learning book by Ian goodfellow. In the PCA formulation in the first chapter i.e Linear Algebra, he mentions the following: we need to choose the encoding matrix D. To do so,...
3
votes
1answer
2k views

ValueError: operands could not be broadcast together with shapes (60002,39) (38,) during pca.transform

I am trying to solve the San Francisco Crime Problem on Kaggle. To begin with, here is my code: ...
3
votes
1answer
127 views

Dimensionality reduction with prior knowledge of colinearity between features

Let's say that I have sparse feature vectors and I'd like to use dimensionality reduction in order to visualize them more easily. Dimensionality reduction techniques like PCA will estimate ...
3
votes
1answer
120 views

stable set PCA while adding features

Is it possible to have a PCA setup (or any other dimensionality reduction technique) in a way that adding new features wouldn't require retrain downstream models that were trained on that particular ...
3
votes
0answers
344 views

How to use PCA in CNN for image recognition using Keras?

I created a CNN model for image classification and I want to use Principal Component Analysis (PCA) but when I run pca.fit() code, the code still running for hours ...
3
votes
0answers
137 views

PCA and FastICA in scikit-learn giving near identical results

So after importing my data, transforming it, and splitting into training and test sets I tried running this script for PCA: ...
3
votes
0answers
349 views

Examples for predict.FAMD?

I am doing a study on unsupervised data with various categorical variables. So I have found the FactoMineR package to be really handy for this, particularly with the FAMD functions. I can get to a ...
2
votes
3answers
6k views

PCA before K-mean clustering

If I applied PCA on feature vectors and then I do clustering, such like following: ...
2
votes
4answers
166 views

I have n dimensional data and I want to check integrity, can I downgrade to 2 dimensional feature space via PCA and do so?

Say I have n dimensional data samples. I want to check the integrity of the features, if they are good representation of the respective classes, i.e. these features are good or not. My plan is: I ...
2
votes
1answer
643 views

Can PCA be applied to reduce dimensionality of only a subset of features?

Lets say I have a feature set of f0 to f1000. I am thinking of applying PCA on f500 to f1000 reducing their dimensionality. Can ...
2
votes
2answers
284 views

PCA algorithm problems - Python

I have implemented PCA algorithm and I understood it very well but still I have some questions. My code is below and it's very simple implementation. ...
2
votes
3answers
477 views

Feature Selection and PCA

I have a classification problem. I want to reduce number of features to 4 (I have 30). I'm wondering why I get better result in classification when I use correlation based feature selection(cfs) first ...
2
votes
2answers
2k views

Which variables matter most for prediction of another variable?

I have a dataset and need to predict, out of 9 variables, which ones matter most to predict number 10. I first tried using the selectKBestmethod from ...
2
votes
1answer
70 views

PCA: projection of positive data on negative side of plane

I did PCA on my data and projected the data on first two eigen vectors. After projection I see that the scatter plot of the data starts from [-1,-1]. My data is all positive. Is it correct for the ...
2
votes
2answers
113 views

What are possible approaches to deal with unseparable data even after PCA?

Greetings data scientists, I am dealing with a complex classification/prediction problem and I am finding very difficult to separate the classes at all. Even after PCA, my data (over the two PCs), ...
2
votes
1answer
160 views

Anomoly detection method selection

I need to decide between SVM (One-Class Support Vector Machine) and PCA (PCA-Based Anomaly Detection) as anomaly detection methods. Azure ML is used and provides SVM and PCA as methods - hence the ...
2
votes
1answer
72 views

When I should use PCA?

I have a data set with 60000 rows and 32 columns. I want to use SVM (with some more constraints that make it more complicated)and I think 32 columns are too large. So I decided to use PCA. But when I ...
2
votes
1answer
86 views

Deciding on the number of components in PCA

I have been running my model several times now. Each time i get different results based on what number i put in my PCA component number range (I used raw numbers in the code instead of the range ...
2
votes
2answers
421 views

Measuring distance preservation in dimensionality reduction

I am looking to compare the distance preserved during dimension reductions for several techniques. I have read some papers on similar topics here and here. For example, I would like to use the ...
2
votes
4answers
91 views

Is dimension reduction helpful to select features for a classification problem?

Let's say I have a data set but I don't know what features are relevant to solve a classification/regression problem. In this case, is it worth/good to use a dimension reduction algorithm and then ...

1
2 3 4 5