Hot answers tagged

4

Since the students know about eigenvectors and eigenvalues, I will assume that they know about Lagrangian multipliers as well. Let's start with computing the variance of data $\vec{x}$ along a given direction $\vec{e}_a$. Since we only care about the direction, $\vec{e}_a$ is a vector of unit length, i.e. $$\vec{e}_a^T \vec{e}_a = 1$$ So first, the ...


4

What can I do with this information? You can do a lot of things with this data. You can visualize it, you can use the vectors for prediction or regression, whatever the task at hand. However, there are a few restrictions of PCA that you need to keep in mind. For eg. its very memory intensive, so you need to have a "lot" of RAM to use PCA on certain data-...


3

This could be many things. Since you want to calssify the data into three classes, I would use one-hot-encoding, rather than the binary enumeration, because you kind of introduce the information to the model that [1,0] and [1,1] are "closer" than [0,0] and [1,1] for example. Unless this is actually the case, better encode the classes like [1,0,0], [0,1,0] ...


3

I think you may have forgotten to subtract the mean. As far as I know you have to center the data, otherwise you will compute variance with respect to the origin, rather than the variance within the data. Your data points have a mean vector $$\mu = \left[0, \frac{2}{3}\right] $$ Let's subtract the mean from the data and put it into a matrix $$ \tilde{X} =...


3

No. t-Distributed Stochastic Neighbor Embedding (t-SNE) and Principal Component Analysis (PCA) are dimension reduction techniques, aka fewer columns of a tidy dataframe. Clustering will reduce the number of observations, aka fewer rows of a tidy dataframe. In particular, you might be looking for hierarchical clustering.


2

The accuracy is varying and not linear, meaning that if the number of components increase, the accuracy will not necessarily increase. This is not unexpected. Choosing the right number of components requires balancing the extra information given by the additional dimensions and the useless noise and redundancy present therein. Even though PCA is a linear ...


2

If you can, go for some non-linear dimensionality reduction technique. The most powerful are Autoencoders, but you can also use t-SNE or other manifold techniques. The problem of PCA is that it can extract only latent factors that are linearly associated with your variables. Using non-linear techniques, less variables can let you capture more of the ...


2

In simple words, PCA (Principal Component Analysis) is a dimension-reduction method where you try to reduce the number of variables from a larger number to a smaller one without any loss of the important information available in the data. Mathematically, it is a projection of a higher-dimensional object in a lower-dimensional vector space. A great example ...


2

Do not apply PCA to categorical data PCA attempts to find the dimensions which contains the most variance in a dataset. When you have categorical variables, the distance between points and the variance captured by a variable are ill-defined. First of all, you have no proper distance measure to tell how far apart two categories are, and secondly, the ...


2

Usually PCA already returns standardized components. Did you compute the variance of each component? Usually, it will be 1. The more tricky question is whether to use standardization before doing PCA. I don't think there is a general answer for that.


2

There are two ways to perform the PCA: Compute the eigenvalue decomposition of the covariance matrix $\Sigma$ Compute the singular value decomposition of the data matrix $X$ Numerically, you can do both by calling svd() on either of them, as for positive semi-definite matrices (like $\Sigma$) svd() gives you the eigenvalue decomposition. There is a ...


2

What is important is that the covariance matrix eigenvalues represent the variance explained by each component of the PCA. PCA can be considered useful when it allows to reduce the dimension without removing too much variance from the dataset; that is, when there are a lot of low eigenvalues. As the sum of the eigenvalues is fixed because it is the total ...


2

Independent of the number of features, you will obviously need much more than 3 * 100.000 * 100.000 * 8 bytes of ram (with double precision floats). That is 240.000.000.000 or about 240 GB. Not only is this a lot of RAM to have, but AP will have to do many passes over this RAM, so this will take forever, even after computing the distance matrix. So ...


2

PCA. Suppose that you have n data points comprised of d numbers (or dimensions) each. If you center this data (subtract the mean data point $\mu$ from each data vector $x_i$) you can stack the data to make a matrix $$ X = \left( \begin{array}{ccccc} && x_1^T - \mu^T && \\ \hline && x_2^T - \mu^T && \\ \hline &&...


2

K-means don't modify the underlying structure of your data. K-means will just provide the 'color' part of your graph. To answer the question about why do you get a cuboid, it's because your underlying data are a cuboid. Not necessarily by construction, but that's what happen when you cap your data. As an exemple, look at the following code : X1 = c(rnorm(...


2

If you want to reduce the number of classes you are predicting over, then you could manually map them to a simpler set (i.e. map poodle, greyhound to dog ) OR if you don't have the domain knowledge you can cluster your data and predict the cluster instead of their original labels. You could use PCA or t-SNE to reduce the number of dimensions before ...


2

Instead of using the score method on your trained model, you should use the predict method. You can then pass the results into the confusion matrix function from sklearn: from sklearn.metrics import confusion_matrix y_pred = svmObject.predict(X) cm = confusion_matrix(y_true, y_pred, sample_weight=sample_weight, labels=labels, ...


1

In general (or common cases) you would find those lines when applying PCA, which are the result of finding those Principal Components Analysis, meaning the directions of maximum variance of your original datatset, i.e., the components which give you more information out of your data. As explained with examples in scikit-learn docu: PCA finds orthogonal ...


1

My guess is the issue is a combination of whiten=True and the largest value equal to the largest possible value of a float64. From scikit-learn's PCA docs: When True (False by default) the components_ vectors are multiplied by the square root of n_samples… That creates an overflow issue which results in the ValueError exception.


1

If the variables you are using for k-Means clustering are on different scales the variables with the higher variance will dominate the algorithm, by driving the convergence of the k centroids. Is this something that you can allow, based on your research goals? If, instead, you want all the factors to have equal weight in the clustering, then you should ...


1

The main difference is that PCA is a dimensionality reduction technique, while Factorization Machines are classifiers. You can use PCA to simplify/compress a given dataset, while you can use FMs to classify your observations. The other difference is that PCA is a linear technique, while FMs are non-linear. PCA extracts later factors that are linearly ...


1

It can be a good idea, but if you want to use PCA, you will have to use it carefully. First of all, PCA will reduce dimension depending on the data observed in your dataset. Consequently, if it is biaised somehow, the projection will not work with different datasets. For instance, if you have a strong correlation between two features and a third one is ...


1

PCA will change your data and you will not be able to interpret it in sane sense, you can just slice and dice the data and do many things by hand, PCA would be usefull if you would want to find "neighbouring" players in terms of raw statistics but it can be deceptive because PCA don't know which stats are important, if you want to decrease dimensionality of ...


1

I see. Yes PCA is a good tool that you can use on your choice of attributes. The tricky part is that PCA is based on variance. Hence, if you have 2 attributes which are more important and have low variance and then 5 more with a lot of variances then the latter will be predominant on the first PC (I guess some standardisation would help). Hence, maybe it ...


1

Take a look at this research paper. It mentions two methods, a Minhash Encoding technique and Gamma-Poisson Matrix Factorization technique for high cardinality categorical data.


1

Pincode I feel very very likely is not a predictor at all for the target variable you have. If you feel geography is an important predictor and has bearing on you target variable then what you can do is use city or state or other relevant geographical unit like sub-city etc. you feel have homogeneous characterstics. Say if state X n Y each has 2 cities P,...


1

one idea is to craft some features of this feature - when you classify cars you don't have data like strings "Ferrari 991 year 2014 red" "BMW z4 year 1999 2.0L blue" but you would like to have columns like "Manufacturer", "Type(SUV/Cabrio...)", "Year", "Engine" etc. you could transform it or just discard


1

Each PCA component is a projection of your centered data onto a line. Centering puts your origin into the center of the multidimensional data. Then each component line direction is chosen so that the projection has the greatest variability. Successive component directions are orthogonal by construction. Generally, PCA is intended for continuous data (as ...


1

Well, mathematically speaking, applying PCA on a bunch of data points usually means (there are some variants) removing the mean and rotate to get uncorrelated features. Then you reduce dimensionality by discarding some of the uncorrelated features. You may get the corresponding points in the original feature space by doing the inverse rotation and adding ...


1

t-SNE is computationally expensive, more than PCA. Many examples might use PCA just to simplify the problem. Moreover, it is explained here: If the data set is high dimensional, doing principal component analysis is recommended because otherwise the curse of dimensionality can be an issue. TSNE makes the assumption of local linearity which might ...


Only top voted, non community-wiki answers of a minimum length are eligible