Questions tagged [feature-extraction]

Variables (used for prediction or explication) used in regression or regression-like models (like clustering, discrimination). Use this tag for questions about constructing such variables or selecting the best among them.

Filter by
Sorted by
Tagged with
0
votes
2answers
25 views

What is the best way to limit number of features in TF-IDF?

I am using the tf-idf to build representations. It is large dataset and it quickly becomes too much for my RAM if I convert the matrix to a Data-Frame. What is the best way to reduce the number of ...
0
votes
1answer
6 views

How to cluster/group these data points (using K-Mean or Hirarachal clustering)

I have genes from different species Gene A , Gene B, Gene C, ... Gene Z Some Genes are similar to each other ...
0
votes
0answers
12 views

An efficient way to encode & embed tabular data of a video into a transformer?

So a little bit of a background: I have a folder which contains video files of lets say humans doing a certain action (i.e. walking) where each .2 seconds is documented in a ...
0
votes
0answers
4 views

Is Hough transform an appropriate line detector for my problem?

I try to get automated labels for images with the help of computer vision. Problem The labels are papyrus fibers on the outer edges of a papyrus fragment. After some research (for example [3]), I come ...
1
vote
1answer
30 views

Word classification

I have a task to classify the model of a product from its part number using machine learning. Part numbers can be of different lengths and forms and can contain both letters and numbers and also ...
1
vote
0answers
14 views

Feature creation: Problem with correlated features?

I recently started to read about feature creation. I've seen some general guidelines although I am not really sure if they are completely true, for example: 1 - Linear classifiers for binary ...
0
votes
1answer
20 views

Features of the fourier transform for machine learning

i intend to extract features from time-domain measurement data. I feed the features to machine learning algorithms to detect anomalies. In the time-domain, i extract mean, RMS, skew and standard ...
0
votes
0answers
6 views

Clustering features of a class based upon the difference between features of a reference class and the particular class across multiple datasets?

I want to separate (generalized separation) the features of several classes based on the difference between the features (floating point values) of the particular class and a reference class across 7 ...
1
vote
1answer
60 views

Algorithms for casual feature selection for continuous Y

Currently I have been trying to find some good algorithms for feature selection. Using correlation or other non casual type of method will not be the right way to do a feature selection. I'm am ...
-1
votes
0answers
10 views

What are filter values in subsequent convolutional layers?

Suppose we have the filters for face detection as a combination of eye, nose, and mouth filters. Does this mean we only need to learn the filter values for the nose, eye, and mouth filters, since the ...
0
votes
0answers
8 views

Filters in subsequent layers

So I recently started learning about CNNs, and one question struck out to menthe filters used in the second layer are a combination of the filters used in the first layer, right? Lets say I make use ...
0
votes
0answers
24 views

Why do we use multiple convolutional layers, instead of a single layer

My intuition is that, when we have the raw pixels from the image, suppose we want to extract an eye, why don't we just try to use an eye detector to extract the eye feature from the image, why do we ...
2
votes
1answer
17 views

Imputing Data that Isn't Missing

I have two columns, [Date Activated] and [Date Closed]. One is the date an account was activated, and the other column is the date an account is closed. There are three scenarios: Case 1 (1/6 data) ...
1
vote
0answers
10 views

How to get dummy variables from "first name"

I intend to predict the age of customers using some features. There are some categorical features that I need to convert to dummy variables before the modelling stage. Since the datasets are so big (...
3
votes
0answers
78 views

ResNet50 + Transformer

In many papers people extract features from image using ResNet and than pass them through transformer. I want to implement the same. I want to get features and than classify them using transformer. ...
1
vote
1answer
20 views

How should I engineer features for Named Entity Identification task?

I was working on Named Entity Identification (not recognition) task. In this NLP task, given a sentence, model has to predict whether each word (aka token) is named entity or not. The dataset used ...
1
vote
1answer
24 views

How many features should be there in a dataset to apply any feature selection method?

I am working on a time series, regression problem, where I have 10 features and 180 observations. I would like to understand what the minimum number of features should be in a dataset to use feature ...
0
votes
1answer
32 views

What is the difference between features in vgg

I read the architecture of the model but this is the first time I try to use it . The calculations of the features map will be different if I extract the features from the two last layers or from the ...
0
votes
0answers
5 views

Feature selection algorithm for psychometrics, when there is several predicted variables

I'm on a psychometric study. It is a survey. All variables are on a scale of 7. So these are considered as continuous variables. I have this dataset: 600 features 100 predicted variables 100 survey ...
0
votes
0answers
35 views

Extract features using Bounding Box

I have a ground truth bounding box for a 3d object. I would like to extract useful features for the object. My goal is to concatenate these visual object features with language features (from the ...
0
votes
1answer
23 views

How to load any particular folder files from a zip dataset

Twitter is a great source of information. Using The Health-News-Tweets.zip dataset contains tweets by different agencies like BBC Health, CBC Health, etc. I will ...
1
vote
1answer
24 views

Feature extraction with mixed categorical and numerical variables

I've been reading up on feature extraction methods - but the ones I have come across all seem to be numerical. To do feature extraction on a mixed numerical/ categorical dataset are there techniques ...
3
votes
1answer
108 views

What is the point of generating new features (linear or non linear) out of existing features in a dataset?

During feature engineering, we can create new features out of existing ones by using arithmetic operations albeit linear or not. Let's say we have two features x and z. We can then create (engineer) a ...
0
votes
1answer
23 views

Creating a complex featureset for regression modeling

I am currently working a on project that requires me to convert all of the categorical variables to continuous (or binary) variables to build a regression model. The problem is that I have more than ...
2
votes
0answers
18 views

What technique's can be used to identify and count individual animals in a dataset?

Problem: I have an image dataset that contains a lot of different chitals (a species of deer). The images are taken by cameratraps in a National Park. I would like to count the individual animals. For ...
0
votes
0answers
8 views

I am looking for general image-based clustering methods

My task is to cluster some images, I decided to use the VGG model to extract the features and then use K-Means to cluster these features. But my question: When I use a VGG as a feature extractor, I ...
0
votes
0answers
18 views

Using multiple TF-IDF to create a feature

I have around 200k comments and I extracted the top 200 words (without stop words) out of their content. Each comment is linked to a specific date. I would like to ask a very stupid question: Is it ...
0
votes
0answers
20 views

payment data prediction at test time

I have the payment data of the client. I want to predict the prob of customers paying late with target classes being 0-30 days, 30-60 days, 60-90 days, and 90+ days based on this paper. The features I ...
1
vote
1answer
50 views

How to model a 3D graph into a vector so that I can feed it into a classification algorithm?

I have a 3D graph like below: Ref: google images It has 2 angles as X and Y and the Z axis is amplitude value (Each 3D graph is representing a pixel). I want to model this into some useful data ...
0
votes
0answers
17 views

single layer autoencoder performing a lot worse than pca

I am trying to use a single layer autoencoder with linear activation function to perform dimensionality reduction on a dataset before clustering. The data consists of 5000 samples with 2000 features ...
0
votes
1answer
107 views

How to use efficient net as feature extractor for meta/Few shot learning in PyTorch

I am working on few shot learning and I wanted to use efficient-net as backbone feature extractor. Most of the model use Resnet as feature extractor. For example I can use below line of code and it ...
1
vote
0answers
20 views

What should be the input shape for convLSTM if ResNet-50 is applied before?

I have a video dataset, extracted all its frames, and applied ResNet-50 to extract features from all frames. ResNet-50 provides feature map of (2534, 7, 7, 2048), 2534 are the number of frames. Now I ...
1
vote
0answers
32 views

Permutation importance of features [closed]

This agnostic-model is not well addressed in research papers. I read articles where it was used to test the accuracy of the models, trying to understand the importance of individual features and their ...
3
votes
3answers
120 views

Deciding which samples the model will (probably) classify incorrectly

Problem: Given a neural network for image classification, the objective is to develop an algorithm which decides which images are 'problematic' and the model is probably going to classify them ...
0
votes
0answers
26 views

How to extract important text markers from samples to identify patterns?

Problem I have collected a decently large set of movie trailer titles from various Youtube trailer channels. I'd like to extract or infer the movie title and release year from this set in some way to ...
1
vote
2answers
33 views

Dimensionality reduction for feature extraction when missing some feature values

I have two questions: 1-Which method is appropriate for dimensionality reduction for feature extraction when missing some feature values? 2-Which textbook is the best source for the answer in (1)?
0
votes
0answers
26 views

TensorFlow - Image Vectorization - Add New Files

I am using tensorflow for feature extraction from images. Images are divided into 5 categories, each having various designs (which are labelled and trained to the system). Model is trained to find ...
1
vote
1answer
87 views

How do I get my Neural network to ignore certain values?

I was wondering if there was a way that I can get my CNN encoder-decoder neural network to completely ignore certain values in my data (2d images). There are some pixel values of 0 that never change ...
0
votes
1answer
20 views

Using extracted feature vector to perform zero shot detection

I've developed a deep learning model trained from scratch on fruits and vegetables. However, as the data is limited, I can only cover a few different types of fruits and vegetables with the model ...
0
votes
0answers
12 views

Doing feature extraction from name data?

I'm working with a genre prediction application right now, and I was curious about handling name data. I was planning to try to use that in the prediction(as a normal human can usually estimate ...
0
votes
0answers
96 views

struggling with sklearn Pipeline, FeatureUnion for NLP

I'm working on Kaggle dataset trying to classify in the Tweet is a disaster or not. I have "text" feature that I will transform using TF-IDF but I also want to use "keyword" ...
0
votes
1answer
34 views

Is it a good idea to combine fine tuning and feature extraction techniques?

I have a normal/tumor medical images dataset and, for the same patients, also the relative genomics, and my goal is to predict if a patient has a tumor by combining all the information. To achieve ...
0
votes
1answer
37 views

Would a neural network trained on extracted features have the same accuracy as a full network with frozen layers?

Let's say that I train two neural networks on the exact same dataset. The first network is a VGG19 model with frozen convolutional layers so only the top dense ...
1
vote
0answers
21 views

Regression with a feature which has its own depth

I'm relatively new to ML/Statistical Analysis, and I'm facing a dataset structured like this ...
1
vote
0answers
33 views

How can I transform a sequence into features

When Machine Learning libraries don't support categorical features those features can be one-hot encoded into a series of binary feature columns. I have a feature that represents a sequence or ...
0
votes
1answer
16 views

Using partially defined features in an unified deep learning model

Suppose we have two types of feature A and B. A is defined for all kinds of samples while B is only defined for some of the samples. Here, B is partially defined does not mean B is missing value (such ...
0
votes
1answer
41 views

How to decide which method to use TFIDF, or BOW

In a huge dataset for NLP it is taking very long time to classify my dataset therefore, trying each feature extraction method separetly is time consuming and not effecient. Is there a way that can ...
0
votes
0answers
67 views

Is it wrong to use residuals from one model as data in another model?

Why is it wrong to use the residuals from one model to fit another? Is it wrong? Is the difference down to how the residuals are used? For example, the residuals are useful features for extending a ...
0
votes
0answers
15 views

Feature vector representation

I have a clarification. I have to create a classification model for certain set of documents. We are supposed to flag it anamoly or not based on certain terms in the document. My question is the terms ...
1
vote
1answer
17 views

How many words should be taken as features in a ML problem?

I would like to ask you how many words should be taken as features in a ML program. For example, if I have 30000 distinct words to make a vocabulary, what would a good number be? I am currently ...

1
2 3 4 5
8