Questions tagged [feature-extraction]

Variables (used for prediction or explication) used in regression or regression-like models (like clustering, discrimination). Use this tag for questions about constructing such variables or selecting the best among them.

Filter by
Sorted by
Tagged with
1 vote
1 answer
318 views

Feature extraction - wavelet transformation + autoregression

I am working on a feature extraction problem (ECG signal). Within my literature review I stumbled across the following text: "The wavelet transform is used to extract the coefficients of the ...
0 votes
1 answer
622 views

Should I normalise image pixel wise for pretrained VGG16 model

My goal is to use pretrained VGG16 to compute the feature vectors excluding the top layer. I want to compute embedding(no training involved) per image one by one rather than feeding batches to the ...
0 votes
0 answers
22 views

How can "profiling" be done in a dataset? What are the different techniques?

I am currently doing an analysis in which I need to "profile" each record. For example, let's say I have a dataset of accounts with customer information (name, id, address, money spent, ...
1 vote
1 answer
41 views

High level-Low Level features in U-NET

Why do the first layers of U-Net or CNN generate low-level features? Why not the last layers? What is the logic behind getting low-level features at the beginning of architecture? And yes, high-level ...
0 votes
2 answers
29 views

How to group categorical columns into similar types?

(Forgive me if the question is ill put. I am a novice in data science. Please comment or edit so that the question can be improved) I have a dataset where we have to predict the future sale of a shop....
1 vote
1 answer
45 views

How to select relevant columns from a dataset with many features

I have a dataset with a large number of potential features (>100) and I am interested in finding a relatively small subset of these (maybe on the order of 5, or 20) features which is best suited to ...
0 votes
0 answers
17 views

How to know which layers should i freeze in deep learning architectures for transfer learning?

I am trying to make generalizations about which layers to freeze. I know that I must freeze feature extraction layers but some feature extraction layers should not be frozen (for example in ...
0 votes
1 answer
31 views

Automated feature selection - Best practice to avoid data leakage?

This question relates generally to all automated feature selection approaches. In my particular scenario, we have a python package called tsfresh and multiclass classification. What has been done so ...
0 votes
1 answer
10k views

How can I fix this error : Python Error ""ValueError: could not broadcast input array from shape (4) into shape (1000)""

I have the following python code: ...
1 vote
1 answer
46 views

Feature extraction with mixed categorical and numerical variables

I've been reading up on feature extraction methods - but the ones I have come across all seem to be numerical. To do feature extraction on a mixed numerical/ categorical dataset are there techniques ...
0 votes
1 answer
24 views

What can we learn from visualizing Feature Maps

I have the following classification model (dogs vs cats): ...
2 votes
0 answers
76 views

Are pipelines capable of cacheing intermediate results?

I use pandas to do feature extraction for machine learning. I hope to achieve the following: Consider I have five data processing steps done sequentially, and if I execute them once, the results will ...
0 votes
1 answer
43 views

Features of the fourier transform for machine learning

i intend to extract features from time-domain measurement data. I feed the features to machine learning algorithms to detect anomalies. In the time-domain, i extract mean, RMS, skew and standard ...
1 vote
3 answers
933 views

How do I use TF*IDF scores for my machine learning model?

I have applied TF*IDF on the 'Ad-topic line' column of my dataset. For every ad-topic line, I get the same output: Firstly, I am unable to make sense of the output. The TF*IDF values are mentioned to ...
0 votes
0 answers
53 views

What is the suggested way to create features (Mel-Spectograms) from speech signal for classification with ResNet?

At the moment I have this piece of code which cuts a Spectogram into fixed length tensors: ...
0 votes
0 answers
20 views

extract features from parts of one image

I have several parts of one image that have one caption... I need to do image captioning by evaluating every part of the image to which the caption will belong so do I need to extract the features ...
0 votes
0 answers
20 views

log mel energies

I want to convert mel spectogram to log mel energies what I used is ...
0 votes
0 answers
4 views

Calculate features on stationary time-series data

I am trying to create a deep learning model that predicts the future price of crypto currencies based on past data. I downloaded the Open, High, Low, Close and Volume (OHLCV) data from yahoo finance ...
0 votes
1 answer
24 views

extract features from low resolution

I have medical images and need to extract features from the layer before the classification layer using VGG for example but the ...
0 votes
1 answer
21 views

For feature selection, do we use Chi-squared with Mutual Information together?

Or do we only choose one out of two for categorical data.
1 vote
2 answers
569 views

What are some good methods to forecast future revenue on categorical and value based data?

I have monthly snapshots (3 years) of all the contract data. It includes following information: Contract status [Categorical]: Proposed, tracked, submitted, won, lost, etc Contract stages [...
2 votes
0 answers
27 views

Task of regression on graphs

Which tools are available to extract features from a graph. After that, I would like to perform regressions on those features. Initially, I was thinking about using the adjacency matrix of the graph. ...
1 vote
1 answer
471 views

MR images segmentation for feature extraction

I have datasets of brain MR images with tumours, the tumours are already selected manually by a physicist using Image J. I have read about segmentation, but I still couldn't understand how do they ...
1 vote
1 answer
215 views

Mapping between original feature space and an interpretable feature space

I'm reading the following really interesting paper https://arxiv.org/pdf/1602.04938.pdf on local interpretable model explanations on page 3 however particularly section 3.3 Sampling for Local ...
0 votes
0 answers
7 views

how to align sliding window to extract features from multi modal timeseries data?

I have two datasets that are collected at different frequencies at the same time. One is recorded at 128Hz and another one is recorded at 512 Hz. I am trying to extract some features using the moving ...
0 votes
1 answer
59 views

Single image feature reduction at inference time : SVM

I am trying to train a SVM classifier using scikit-learn.. At training time I want to reduce the feature vector dimension. I have used PCA to reduce the dimension. ...
0 votes
0 answers
10 views

Neural Network One-hot Feature concatenation

I'm trying to add features to a model with two one hot encoded features. The features are defined like this. ...
1 vote
1 answer
128 views

statistical significance test between binary label features

I have 667 features and I want to find features that have a significant boundary between a binary class label before I apply a classification model (e.g Naive Bayes/ SVM) to improve classification ...
0 votes
1 answer
12 views

I am looking for general image-based clustering methods

My task is to cluster some images, I decided to use the VGG model to extract the features and then use K-Means to cluster these features. But my question: When I use a VGG as a feature extractor, I ...
2 votes
1 answer
363 views

Feature extraction from relational database

In order to build a classifier, I need to extract a few features from the data stored on a MySQL database. I need to join multiple tables and it is taking a lot of ...
0 votes
0 answers
7 views

Is Self-Supervised Learning a task of Representation Learning?

Maybe a weird question but: Currently, I'm writing a seminar paper about Self Supervised Learning for time series data. For this paper, I have to find methods to prepare unlabelled time series data ...
32 votes
6 answers
10k views

Are there any tools for feature engineering?

Specifically what I am looking for are tools with some functionality, which is specific to feature engineering. I would like to be able to easily smooth, visualize, fill gaps, etc. Something similar ...
0 votes
2 answers
19 views

do feature selection and model selection must share the same ratio between development set and test set?

As the title, after I performed a Feature Selection, is it mandatory to respect the same ratio (between development set and test set) in Model Selection?
1 vote
0 answers
12 views

Methods for combining instance observations for classification

I am working on a project where I classify tiny moving particles into a few classes (fibers, hairs, glass shards, bubbles). The particles are only a few pixels large and are observed in a few frames ...
0 votes
1 answer
273 views

Selecting only a certain number of top features using tsfresh

How can I select the top n features of time series dataset using tsfresh? Can I decide the number of top features I would like to extract?
1 vote
1 answer
90 views

Algorithms for casual feature selection for continuous Y

Currently I have been trying to find some good algorithms for feature selection. Using correlation or other non casual type of method will not be the right way to do a feature selection. I'm am ...
1 vote
2 answers
68 views

How to extract features from long chemical names?

I have an interesting problem that I am uncertain about how to even get started. I am working on a binary classifier that will take a chemical name, encoded as a string, and predict whether it is a '...
3 votes
1 answer
167 views

Extract features from a survey

I need to use the answers from a questionnaire for training a classifier. I discovered that some questions can have nested sub-questions.. Let's say (just an example) that I want to predict whether a ...
2 votes
1 answer
428 views

How to tune parameters for Time Series Analysis, when forecasting is only dominated by one feature and error is not getting reduced?

I am trying to predict time series based on 150 features. When I plot correlation of these features, I am getting 20 features with more or less importance but every model I use, it is completely ...
2 votes
2 answers
59 views

Is it wise to always `StandardScaler()` features? [SOLVED]

My current investigations point to the sklearn.preprocessing.StandardScaler() not always being the right choice for certain types of feature extractions for neural ...
0 votes
1 answer
148 views

Feature Engineering - correlation with binary outcome - Titanic Dataset - Ticket feature

I am currently building my first machine learning model using the titanic dataset. After the data exploration, I decided to focus my attention on the 'Ticket' feature. One thing I have noticed about ...
2 votes
1 answer
317 views

extract document topic vectors from lda model

how can I extract document-topic matrix from LDA model and use it as input features an svm classifier? I am using gensim for implementation
0 votes
1 answer
81 views

General question on EDA, correlations, classification, ML

I am looking for a general best practices regarding classification and correlations. I created a new predictor feature call it B, based on a certain threshold in a feature A. Now I started to do EDA ...
5 votes
1 answer
695 views

Time Series pattern recognition and classification problem

I have some labeled sensor data. Now, I would like to know how to extract features from time series using DFT, DWT, and HAAR transforms. I know that the transformations above transform a signal to ...
3 votes
1 answer
29 views

NLP text representation techniques that preserve word order in sentence?

I see people are talking mostly about bag-of-words, td-idf and word embeddings. But these are at word levels. BoW and tf-idf fail to represent word orders, and word embeddings are not meant to ...
0 votes
0 answers
8 views

How to detect features in a 3D point cloud with a simple neural network?

I have a question and hope that you can help me. I am looking for a simple algorithm to detect features from unstructured 3D data. These features can have holes and they are an important part of the ...
0 votes
0 answers
17 views

Automatic feature extraction from EMG, autoencoder vs variational autoencoder?

Thanks for checking my question! From the EMG signals of Parkinon's patients, I want to extract rigidity and bradykinesia information. In order to do that, we need feature engineering. But, I would ...
1 vote
1 answer
20 views

How should I handle time-duration-based columns in classification?

For example, say I am trying to predict whether I will win my next pickleball game. Some features I have are the number of hits, how much water I’ve drinken, etc, and the duration of the match. I’m ...
0 votes
2 answers
37 views

Dimensionality reduction for feature extraction when missing some feature values

I have two questions: 1-Which method is appropriate for dimensionality reduction for feature extraction when missing some feature values? 2-Which textbook is the best source for the answer in (1)?
0 votes
1 answer
201 views

How do I get my Neural network to ignore certain values?

I was wondering if there was a way that I can get my CNN encoder-decoder neural network to completely ignore certain values in my data (2d images). There are some pixel values of 0 that never change ...

1
2 3 4 5
8