Questions tagged [feature-extraction]

Variables (used for prediction or explication) used in regression or regression-like models (like clustering, discrimination). Use this tag for questions about constructing such variables or selecting the best among them.

Filter by
Sorted by
Tagged with
2 votes
0 answers
51 views

Feature selection and model performance

Featuretools provides an automated way to generate features from your data, by providing relationships within your data and applying their so-called deep feature synthesis. It generates features like ...
  • 121
1 vote
1 answer
21 views

Handling of readability scores for short texts

I have a classification problem using emails as my dataset. I would like to use scores from various readability formulas as features for the classification. However, most of them are defined for ...
  • 131
0 votes
0 answers
18 views

Features for Sentiment Analysis

I'm trying to implement sentiment analysis (using some modified version of naive bayes). I wanted to know of what features I can use that can help me in classification. As of now I can only think of ...
0 votes
2 answers
13 views

Autoencoder vs Pre-trained network for feature extraction

I wanted to know if anyone has any sort of guidance on what is better for image classification on a lot of classes (about 400) with a small amount of samples per class (around 20), for relatively big ...
1 vote
3 answers
130 views

Using the results of clustering to retrain a neural network

I am following and expanding upon previous work from the winner of the Melanoma Classification from here. The dataset has 9 classes. The competition is only interested in the one class (Melanoma). I ...
  • 153
0 votes
1 answer
20 views

Min Pooling vs Max Pooling

If I train a simple CNN with an MNIST dataset for digit classification. Is it possible to get a similar performance if I replace the max-pooling layers with the min-pooling layers? This problem may be ...
  • 141
0 votes
0 answers
11 views

Which Keras Application to choose for transfer learning?

I am working on using transfer learning in Keras to analyze Chest X-rays. I am going to choose one of the high performing Keras applications. https://keras.io/api/applications/ How should I choose ...
1 vote
1 answer
30 views

How to build a predictive model with multiple features?

I built an R RandomForest Regression model. The source training data is a historical monthly report of all closed tickets, and the data for forecasting/prediction is a report of open tickets. These ...
  • 113
0 votes
0 answers
18 views

Feature Engineering Encoding for multiple category with huge category range

I need to encode a column "Tags" that has a total of 144 different types and at the same time a row can contain multiple tags. What's the best encoding method in this situation? One-hot ...
  • 1
0 votes
0 answers
17 views

Extracting meaningful information from time series data in python

The following graph shows the data (Top: Time domain ; Bottom: Frequency domain). The meaningful data in my context is the one marked by red lines. How can I extract only the meaningful data from the ...
  • 103
0 votes
0 answers
11 views

Which layers are doing image segmentation on AutoEncoders/U-NET?

While I was researching for transfer learning, I saw that people are replacing encoders with VGG-16 weights and only training the decoder part of the network. But in some representations (like This) ...
  • 83
0 votes
0 answers
15 views

Is the result of my feature encoding numeric or categorical?

I have the following categorical feature in a data table (recording the day of week when a certain action happened): ...
  • 8,037
0 votes
0 answers
27 views

How can "profiling" be done in a dataset? What are the different techniques?

I am currently doing an analysis in which I need to "profile" each record. For example, let's say I have a dataset of accounts with customer information (name, id, address, money spent, ...
1 vote
1 answer
72 views

High level-Low Level features in U-NET

Why do the first layers of U-Net or CNN generate low-level features? Why not the last layers? What is the logic behind getting low-level features at the beginning of architecture? And yes, high-level ...
  • 83
1 vote
1 answer
74 views

How to select relevant columns from a dataset with many features

I have a dataset with a large number of potential features (>100) and I am interested in finding a relatively small subset of these (maybe on the order of 5, or 20) features which is best suited to ...
1 vote
1 answer
60 views

Automated feature selection - Best practice to avoid data leakage?

This question relates generally to all automated feature selection approaches. In my particular scenario, we have a python package called tsfresh and multiclass classification. What has been done so ...
  • 39
0 votes
1 answer
29 views

What can we learn from visualizing Feature Maps

I have the following classification model (dogs vs cats): ...
0 votes
0 answers
27 views

extract features from parts of one image

I have several parts of one image that have one caption... I need to do image captioning by evaluating every part of the image to which the caption will belong so do I need to extract the features ...
  • 1
0 votes
0 answers
41 views

log mel energies

I want to convert mel spectogram to log mel energies what I used is ...
0 votes
0 answers
5 views

Calculate features on stationary time-series data

I am trying to create a deep learning model that predicts the future price of crypto currencies based on past data. I downloaded the Open, High, Low, Close and Volume (OHLCV) data from yahoo finance ...
0 votes
1 answer
27 views

extract features from low resolution

I have medical images and need to extract features from the layer before the classification layer using VGG for example but the ...
  • 1
0 votes
1 answer
28 views

For feature selection, do we use Chi-squared with Mutual Information together?

Or do we only choose one out of two for categorical data.
  • 1
2 votes
0 answers
32 views

Task of regression on graphs

Which tools are available to extract features from a graph. After that, I would like to perform regressions on those features. Initially, I was thinking about using the adjacency matrix of the graph. ...
0 votes
0 answers
8 views

how to align sliding window to extract features from multi modal timeseries data?

I have two datasets that are collected at different frequencies at the same time. One is recorded at 128Hz and another one is recorded at 512 Hz. I am trying to extract some features using the moving ...
0 votes
0 answers
14 views

Neural Network One-hot Feature concatenation

I'm trying to add features to a model with two one hot encoded features. The features are defined like this. ...
0 votes
0 answers
9 views

Is Self-Supervised Learning a task of Representation Learning?

Maybe a weird question but: Currently, I'm writing a seminar paper about Self Supervised Learning for time series data. For this paper, I have to find methods to prepare unlabelled time series data ...
0 votes
2 answers
20 views

do feature selection and model selection must share the same ratio between development set and test set?

As the title, after I performed a Feature Selection, is it mandatory to respect the same ratio (between development set and test set) in Model Selection?
1 vote
0 answers
12 views

Methods for combining instance observations for classification

I am working on a project where I classify tiny moving particles into a few classes (fibers, hairs, glass shards, bubbles). The particles are only a few pixels large and are observed in a few frames ...
2 votes
2 answers
86 views

Is it wise to always `StandardScaler()` features? [SOLVED]

My current investigations point to the sklearn.preprocessing.StandardScaler() not always being the right choice for certain types of feature extractions for neural ...
3 votes
1 answer
34 views

NLP text representation techniques that preserve word order in sentence?

I see people are talking mostly about bag-of-words, td-idf and word embeddings. But these are at word levels. BoW and tf-idf fail to represent word orders, and word embeddings are not meant to ...
  • 359
0 votes
0 answers
9 views

How to detect features in a 3D point cloud with a simple neural network?

I have a question and hope that you can help me. I am looking for a simple algorithm to detect features from unstructured 3D data. These features can have holes and they are an important part of the ...
  • 101
0 votes
0 answers
25 views

Automatic feature extraction from EMG, autoencoder vs variational autoencoder?

Thanks for checking my question! From the EMG signals of Parkinon's patients, I want to extract rigidity and bradykinesia information. In order to do that, we need feature engineering. But, I would ...
  • 1
1 vote
1 answer
20 views

How should I handle time-duration-based columns in classification?

For example, say I am trying to predict whether I will win my next pickleball game. Some features I have are the number of hits, how much water I’ve drinken, etc, and the duration of the match. I’m ...
0 votes
0 answers
6 views

What kind of features can I obtain from IP:Port data?

I have a dataset that consist of the fields below. IP_Version,id,IP_TTL,IP_Source,TCP_Source_PORT,IP_Dest,TCP_Dest_PORT,data_size,timestamp What kind of features ...
0 votes
0 answers
54 views

Use VGG19 to extract features from single channel YCbCr image

I have a set of images in the YCbCr format. I took only the Y channel and will use them to train a super resolution network. I would like to use a loss function named perceptual loss, which involves ...
  • 1
0 votes
0 answers
11 views

Feature Selection for vector of targets, Y=(y1,...,yn)

If I have an output/target, y, that is a vector of targets, e.g. (y1(t), y2(t), y3(t), y4(t)) and training data X, also a vector, (x1(t),...,xn(t)), and I wish to do e.g. regression, neural nets, and ...
  • 71
1 vote
1 answer
30 views

Anomaly detection for varying dictionary

I want to detect the anomaly in the processes taking up the most CPU percent. I receive the data as a time series of dictionary values like so: ...
  • 11
1 vote
0 answers
16 views

Best way to represent a version feature based on percentiles

We're training a binary classifier in AutoML, and one of the features consist of browser versions. Currently these versions are provided "normalized" to the model, according to the ...
0 votes
1 answer
27 views

What does re.split(r'[_]' , i) does? [closed]

What does re.split(r'[_]' , i) does? I have a function with the above code. Can someone please explain how does the split occurs.
  • 3
1 vote
1 answer
35 views

When an author says Features are the input to Machine Learning Model what does it mean?

I am reading an article about graph neural network and it is mentioned: In this step, we extract all newly update hidden states and create a final feature vector describing the whole graph. This ...
  • 135
0 votes
0 answers
23 views

Regarding pos tagging

I working on a dataset, I did the pos_tagging using nltk. Now I want to know which sequence of grammar is most common in my rows, then I want to define a chunk grammar based on a common grammar ...
0 votes
0 answers
23 views

Approaches for multiclass classification with a reference level to extract variables of importance?

I have a dataset with with multiple classes (< 20) which I want to classify in reference to one of the classes.The final goal is to extract the variables of importance which are useful to ...
0 votes
0 answers
23 views

Using Validation Set in Transfer Learning for Feature Extractor Preprocessor

I have a set of images of products. I am using transfer learning for images feature extraction in this way : I load a model (res-net, vgg) I add 2 dense layers, first one will be my features and the ...
  • 101
0 votes
0 answers
35 views

Class mismatch in categorical features after ordinal encoding and other feature inconsistencies between train and inference

This post is about three cases of feature mismatch, an issue that is quite prevalent and challenging in many production ML use cases. Let's suppose we have a new dataset on a weekly base which we ...
0 votes
0 answers
10 views

sampling fall on which category of data mining

I have a question regrading the steps of data mining. After searched on Google I came to know that Data mining have 7 key steps ...
  • 297
1 vote
0 answers
162 views

Using SHAP values as features in a classification problem

I'm looking for feedback on a methodology I've tried that has yielded strange results. Problem background: supervised multi-class classification problem for which I've used a random forest to create ...
  • 111
0 votes
2 answers
895 views

What is the best way to limit number of features in TF-IDF?

I am using the tf-idf to build representations. It is large dataset and it quickly becomes too much for my RAM if I convert the matrix to a Data-Frame. What is the best way to reduce the number of ...
0 votes
1 answer
11 views

How to cluster/group these data points (using K-Mean or Hirarachal clustering)

I have genes from different species Gene A , Gene B, Gene C, ... Gene Z Some Genes are similar to each other ...
  • 517
0 votes
0 answers
15 views

An efficient way to encode & embed tabular data of a video into a transformer?

So a little bit of a background: I have a folder which contains video files of lets say humans doing a certain action (i.e. walking) where each .2 seconds is documented in a ...
  • 101
0 votes
0 answers
6 views

Is Hough transform an appropriate line detector for my problem?

I try to get automated labels for images with the help of computer vision. Problem The labels are papyrus fibers on the outer edges of a papyrus fragment. After some research (for example [3]), I come ...

1
2 3 4 5
8