Questions tagged [dimensionality-reduction]

Dimensionality reduction refers to techniques for reducing many variables into a smaller number while keeping as much information as possible. One prominent method is [tag pca]

Filter by
Sorted by
Tagged with
1
vote
1answer
9 views

Dimensionality reduction of vectors with null values

I have vectors of same length where each entry can have the value 0, 1 or null. V = {[0,1,1,1,null,0], [null,1,0,null,0,1], ...} How can I perform a dimensionality ...
0
votes
0answers
7 views

Embedding data with a graphical structure

I have an $n\times p$ dataset and wish to embed each observation in a $d$ dimensional space. The trouble is, my predictors are derived from a DAG. For a simplified example, suppose the DAG is as ...
0
votes
0answers
14 views

feature selection for categorical variables

I have been working on this issue for quite a while and going nowhere. If I have categorical features in my dataset and some of them have high dimensions, if I OHE them, I get a dataset with high ...
2
votes
0answers
28 views

Theoretical differences between KPCA and t-SNE?

I (think I) understand the underlying principles of most dimensionality reduction methods (MDS, IsoMap, t-SNE, Spectral Embedding, Diffusion maps, etc...). Some of the algorithms I use the most are ...
0
votes
0answers
89 views

Why am I getting a different answer in Principal Component Analysis dimensional reduction?

Problem-: Consider the two dimensional patterns (2, 1), (3, 5), (4, 3), (5, 6), (6, 7), (7, 8). Compute the principal component using PCA Algorithm. Use PCA Algorithm to transform the pattern (2, 1) ...
1
vote
1answer
13 views

Best metric and hyperparameters in dimension reduction with UMAP for binary sparse data

I am playing with a dimensionality reduction step prior to clustering for a pretty large sparse binary matrix of almost 3000 columns and 50k rows. My idea is to embed the 3000 dimensions into a two-...
0
votes
0answers
11 views

Low dimensional manifold in a high dimensional space and Geodesic distance

It is a common assumption that high-dimensional objects are lying in low-dimensional manifolds. And this constitutes a foundation for manifold learning or dimensional reduction techniques or (a way to ...
0
votes
0answers
8 views

How do you aggregate features of lists (pooling alternatives)?

Is it possible to reduce non-correlated multi-dimensional data over features to 1D data? A working option is pooling (mean/min/max) over an embedding vector (n samples of embeddings of m dimensions). ...
0
votes
2answers
18 views

Concerns regarding small dataset with too many features

I have dataframe with 322 observations with 224 features. The observations has two classes, 0 or 1,which i'm trying to predict. class 0 has 168 observations and class 1 has 154 observations. I was ...
1
vote
0answers
28 views

Hashing trick for dimensionality reduction

I am building a model that uses TF-IDF NLP features in Spark Mllib. The TF-IDF HashingTF function in Mllib uses the 'hashing trick' to efficiently allocate terms to features. My question is: does the ...
0
votes
1answer
27 views

Principal Components of PCA

I have a dataset combined with normals samples and fault samples. We use PCA on normal samples and faults samples separately. I observe that the last PCs help us to have better separation than for ...
0
votes
0answers
13 views

single layer autoencoder performing a lot worse than pca

I am trying to use a single layer autoencoder with linear activation function to perform dimensionality reduction on a dataset before clustering. The data consists of 5000 samples with 2000 features ...
2
votes
1answer
38 views

What can we learn from PCA on non linear data?

Suppose we have dataset with 10 features which are not linear: ...
6
votes
1answer
250 views

Why it is recommended to use T SNE to reduce to 2-3 dims and not higher dim?

According to wiki it is recommenced to use T-SNE to map to 2-3 dimensional. I can understand this , if we want to visualizing the data. If we want to reduce the ...
0
votes
2answers
52 views

T-SNE with high number of features

If we have high number of features (more than 50), should we use T-SNE ? According to https://scikit-learn.org/stable/modules/generated/sklearn.manifold.TSNE.html: It is highly recommended to use ...
0
votes
0answers
26 views

How to run PCA when data contains some categorial features?

Assume that we have a dataset with various features, and some of the features are categorial. And PCA dosn't work good on categorical features. How should I handle such datasets using PCA, what is ...
1
vote
2answers
19 views

What we can learn from the data if PCA scree plot bins are almost the same?

Suppose we have a data-set with 4 features. Suppose we calculate the PCA for this dataset and we plot the scree-plot: What we can learn from the features? Can we ...
0
votes
2answers
36 views

How to split and train a model for data in biology

I am using gene expression data that are float numbers and want to train classifiers in view of binary classification. Since I am a novice in this field I have some questions: The first classifier I ...
1
vote
1answer
31 views

Does it make sense to use UMAP for dimensionality reduction for modeling (rather then presentation/exploration)?

Reducing dimensionality via PCA before training is a common practice, but PCA cannot makes use of nonlinear relations between features. I read about UMAP (e.g. https://adanayak.medium.com/...
1
vote
0answers
19 views

Statistical significance of SVD least squares

I was not able to find any info on how least squares using singular value decomposition should be statistically evaluated. I have a dataset for which I did both multivariate regression and regression ...
2
votes
1answer
33 views

How can I reduce the number of dimensions using a Clustering algorithm in a mixed dataset?

I am working with a mixed data set, corresponding to TV consumption data, with the aim of reducing the number of features to only those relevant to detect TV consumption patterns (or consumption ...
1
vote
2answers
30 views

Dimensionality reduction for feature extraction when missing some feature values

I have two questions: 1-Which method is appropriate for dimensionality reduction for feature extraction when missing some feature values? 2-Which textbook is the best source for the answer in (1)?
1
vote
1answer
84 views

Can someone clarify what the linear assumption of PCA is?

0 For the past few hours I've been trying to search what this linear assumption is. Some of the articles states that that your independent variables have to be linear in relationship and need some ...
0
votes
1answer
28 views

Understanding clusters after applying PCA then K-means

I have a dataset grouped by customer level, and the rows are sum_mexico, sum_uk, ... etc to indicate if the customer has spent money at stores in those countries..similarily counts for these as well. ...
0
votes
2answers
46 views

How can I reduce the volume of data (No column to be dropped)?

I have a data set of 80,000 samples (40k 3 axis accelerometer and 40k Gyro data). I am trying to implement KNN and Random Forest for activity recognition on ESP8266 Node MCU. The limited memory of the ...
0
votes
1answer
14 views

Can I cluster an aggregated data-set (grouped by) and apply dimensionality reduction?

I have data of sales, however it is in the millions, about 500M rows. I aggregate this data by factors such as location, shoptype, country_of_shop, cardtype, and then the aggregated statistic is: ...
1
vote
0answers
24 views

Compare distance between embeddings in different dimensions

I am working on a problem with CNNs. After the convolutional layers, comes a "flatten". One could interpret that as a representation of the input image in some high-dimensional continuous ...
0
votes
0answers
23 views

MCA and FAMD bad scores for UCI census dataset

Attempting dimensionality reduction on the Census-Income (KDD) Data Set. The dataset is a mixed dataset with continuous and categorical features. PCA works fine for continuous variables, reduced down ...
0
votes
0answers
22 views

Dimensionality reduction for geometric curves using an autoencoder - what is wrong?

I am trying to play with toy models in order to study autoencoders. In particular, I want to do dimensionality reduction for simple geometric curves in 3D. First, I take a toroidal helix. ...
0
votes
0answers
14 views

Difficulty understanding the dimension differences in kernel PCA

In Kernel PCA, the kernel trick works because we can show that there is an equivalency between eigenvectors of the kernel matrix and eigenvectors of the covariance matrix. I know the math to go from ...
0
votes
0answers
13 views

Clustering pre-defined groups of data points under dimensionality reduction

I have a dateset of around a million observations, and each observation (300 features) belongs to one of around 300 groups. The set of observations of one group does not directly correspond to the ...
2
votes
1answer
35 views

Using the curse of dimensionality for encoding non-ordered (nominal) categorical variables of high cardinality

When the dimension is high, all data are approximately at the same distance away from each other. This makes distance-based methods such as k-nearest neighbors less useful if the data are more or less ...
1
vote
1answer
26 views

Dimensionality reduction to correlate large number of variables

So I have this dataset with about 750 variables (columns) and 50,000 rows of entries. I would like to reduce the dimensionality of the dataset to say 25-50-100 dimensions and then compute a ...
0
votes
0answers
14 views

dimensionality reduction of (partly) categorical metadata

I have several datasets, each with with hundreds of samples. I have different metadata for each data set, which contains about 50 variables per sample. Some of this metadata is clearly redundant. For ...
0
votes
0answers
13 views

Discrete Wavelet Transform Time Series

My problem is to cluster some time series together. But due to a huge length I was interested in using some methods to reduce the dimensionality. I was thinking of Discrete Wavelet Transform since the ...
1
vote
0answers
24 views

Search for redundant filters(channels) in CNN

When training a CNN one specifies in each layer the number of channels. In the input we have 1 channel for grayscale image and 3 for RGB image, and then usually the ...
1
vote
0answers
15 views

Utilizing 1x1(x1) convolutions as a learned max pooling (3D)?

I have a semantic segmentation network that ingests 3D images (hyperspectral $(x, y, b)$) and predicts 2D images (semantic map $(x, y)$). This network takes the form of a classic UNet, though it ...
1
vote
0answers
22 views

Is there a way to use UMAP on a weighted digraph?

I know If I have a weighted graph $G$(not a directed graph), I can define a metric on its vertices: $$d(i,j) = \text{weight of the minimum path between vertex i and j}$$ So I think It makes sense to ...
0
votes
0answers
22 views

How do I find the eigenvectors corresponding to the largest eigenvalue of a matrix in scikit?

Im trying to determine the principal component 1 and 2 of a symmetric matrix using sklearn. Id appreciate any help. Thank you.
1
vote
1answer
141 views

Dot product and linear regression

I'm studying PCA and my professor said something about finding the linear regression by doing the dot product of both axis. Could someone explain to me why? The dot product returns a number. What's ...
0
votes
0answers
17 views

What are some good techniques to decrease the size of Image Embeddings returned by CNN model?

I want to extract features from pre trained ResNet model for over 2M data. Problem? Even with the average pooling applied on the last layer's result, it provides a ...
1
vote
1answer
44 views

How to use embedding to reduce features for a regression problem

I’m working on a regression problem in which I’d like to predict demand of different items. I have used holidays as a feature in my model, in a hot encoded format, i.e. I have 11 binary features each ...
4
votes
2answers
238 views

Elimination of features based on high covariance without affecting performance?

I ran into a question where the answer ran me into a big doubt. Suppose we have a dataset $A=${$x1,x2,y$} in which $x1$ and $x2$ are our features and $y$ is the label. Also, suppose that the ...
1
vote
0answers
9 views

Can an Isomap be embedded in a manifold of higher dimension than the corresponding MDS?

I am using the Isomap algorithm to operate a dimension reduction on a distance matrix $M_{dist}$. For a given choice of nearest neighbors k to compute the geodesic distance, I use the following method ...
0
votes
0answers
8 views

MSE errors on autoencoder for dim reduction decreases in a weird patteren and I would love some help to dechyper it

I'm training a denoising autoencoder right now to reduce the dimension of a feature vector I designed of dim 58 to a latent space of dim 10, or less hopefully. I'm having a hard time understanding ...
1
vote
1answer
25 views

Modern methods for reducing dimensions and feature engineering

I am training a binary classifier in Python to estimate the level of risk of credit applicants. I extracted a little over a thousand independent variables to model the observed behavior of four ...
0
votes
0answers
22 views

Hierarchical clustering in R filtering variable

I would like to test the added value of features compared to currently used predictors. First, I checked if features were not correlated to the predictors (volume and intensity) I already use, and for ...
1
vote
1answer
34 views

Neural networks with not-fixed dimension for input and output

I would like to know if it exists a model/method which can deal with input and output of different dimension. For example, let us say that the maximum number of info we could have is 6 features and 5 ...
1
vote
0answers
20 views

Machine learning on graphs

I'm looking for some method/model to help me with my current problem: I have a geometry, consisting of points, and eges. For each point I take information about itself and its neighbours. For now I ...
1
vote
1answer
33 views

Can we use feature selection and dimensionality reduction together?

I have a dataset having about 10,000s of features. The features have a hierarchy inherent to them. I found an algorithm performing feature engineering, taking the hierarchy of the features into ...

1
2 3 4 5 6