Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [feature-extraction]

Variables (used for prediction or explication) used in regression or regression-like models (like clustering, discrimination). Use this tag for questions about constructing such variables or selecting the best among them.

0
votes
1answer
19 views

How exactly do I extract the important features from strings for machine learning?

Forgive me for my ignorance. Linked below is an image of my dataset with 1000 tuples. https://i.stack.imgur.com/WHIlx.png I have the following questions (1) How exactly do I go about extracting ...
1
vote
2answers
23 views

How exactly do I go about extracting features from timestamps for machine learning? [on hold]

My dataset has a timestamp column with the following format: 06/24/18 0:56 How exactly do I convert this information into features that can be used for classification algorithms like logistic ...
0
votes
1answer
20 views

General question on EDA, correlations, classification, ML

I am looking for a general best practices regarding classification and correlations. I created a new predictor feature call it B, based on a certain threshold in a feature A. Now I started to do EDA ...
1
vote
1answer
37 views

To remove Chinese characters as features -

I have created document-term matrix using TfIdfVectorizer, but just noticed the feature contains Chinese characters. Is it possible to remove them using Python's regex? I believe these characters ...
0
votes
1answer
11 views

Inverse Binary Feature

I am feeding a binary value into my NN which represents whether the given example is a public holiday or not. Is there a difference between assigning a 0 to public holidays and 1 to all other days or ...
0
votes
1answer
29 views

Python Time series: extracting features on a rolling window basis

I have a long univariate time series, and before performing some machine learning models with it, I want to extract as many features as I can from the time series on a rolling-window basis. As a ...
1
vote
0answers
25 views

Adding Fourier transform features to data

I'm working on some timeseries data which after visualising seems to be periodic(repeating at some interval), So I planned on finding the Fourier transform of the entire and pick the top n amplitudes ...
0
votes
0answers
22 views

Can I use statistics of original data as input features?

I've got a problem regarding defect analysis, where the goal is to improve the manufacturing process of a factory. The factory does one or more batches of production everyday, each one divided by ...
0
votes
1answer
38 views

How do I create a feature vector for the training of an SVM?

I have an understanding problem with implementing an SVM as a classifier for images. The whole thing should be done in python. Now, when I have extracted all the features, e.g. HOG, contours, textures,...
1
vote
1answer
24 views

Dealing with correlated features when calculating permutation importance

I have implemented the permutation importance calculation as found here in my attempt to identify features that contribute little to my model's (Gradient Boosted Tree model) predictive power. The ...
1
vote
0answers
19 views

Feature Extraction [closed]

I've a data set for different sensor values(like voltage, pressure, vibration etc) for a machine. I need to extract features from it to be able to do predictive maintenance. Which features should i ...
2
votes
0answers
27 views

Using datasets to predict the results for other devices

I have a datasets that contain results from a series of physical tests. It has about a dozen features and the outcome of each test is distinguished by 3 different classes. The dataset includes ...
5
votes
3answers
117 views

How does Doc2Vec treat numerical data which is a part of text data?

I have data containing both numbers and raw text related differently like: The power of diesel generator is 15kva. I need a single phase generator. Three phase generator required of 140 kva. Need 70g/...
3
votes
3answers
58 views

What is the ideal database that allows fast cosine distance?

I'm currently trying to store many feature vectors in a database so that, upon request, I can compare an incoming feature vector against many other (if not all) stored in the db. I would need to ...
0
votes
0answers
6 views

Discrete wavelet transform for image texture analysis

I am planning to use the discrete wavelet transform to extract textural features from grayscale images for classification purpose. However, I am not sure which type of wavelet should I choose? most of ...
-1
votes
1answer
61 views

ML: How to think feature selection?

What is the basic philosophy behind feature selection and modelling? How do you actually start? Could you please share your real (practical) inputs? Bit of background: I am actually trying to analyse ...
0
votes
0answers
16 views

How to balance specificity and sensitivity?

I have included accuracy and feature minimization in my fitness function. How should I balance specificity and sensitivity?
1
vote
0answers
29 views

how to extract the Top contributing labels/words in universal-sentence-encoder-large - TransformerModel?

I'm using the universal-sentence-encoder-large (Transformer Model) encoding process for embedding and then using the embedding for Clustering - Basically for unsupervised learning. I want to get the ...
0
votes
0answers
4 views

A noise robust Local binary patterns variant

I’m thinking of using Local Binary Patterns (LBP) to extract features from MRI images of brain tumours to build a module for classification, due to its computational simplicity and good performance, ...
0
votes
1answer
17 views

Size of Output vector from AvgW2V Vectorizer is less than Size of Input data

Hi, I have been seeing this problem for quite some time. Whenever I tried vectorizing input text data though avgw2v vectorization technique. The size of vectorized data is less than the size of the ...
0
votes
0answers
21 views

Dimensionality reduction categories

According to what I found, dimensionality reduction has two types feature selection and feature extraction . In feature extraction, we find PCA, LDA ,LLE , ISOMAP, etc.. In other works i find random ...
3
votes
3answers
65 views

How to understand features impact in a non linear case?

I give a simple example: I have a set of houses with different features (# rooms, perimeter, # neighbours, etc...), almost 15, and a price value for each house. The features are also quite correlated (...
0
votes
0answers
13 views

Question about signal processing for audio: to use Amplitude_to_db or not? Librosa

So this is my first Audio Processing project (Speaker Recognition), therefore I'm trying to understand a few things. First of all, I understand that there are multiple feature extraction techniques, ...
1
vote
1answer
28 views

A suitable feature vector for images

I have a set of images of various products from different websites. I want to cluster the images based on the product shown in the image. How can I generate a suitable feature vector for an image for ...
3
votes
3answers
100 views

Feature engineering from date, mean and standard deviation

I have a multi class classification problem where I should predict the passengers for flights (0-7 classes). The training set consists of the following features: Date of the flight Mean of the weeks ...
0
votes
1answer
56 views

Value of features is zero in Decision tree Classifier

I used CountVectorizer and TfidfVectorizer seperately to vectorize text which is 100K reviews and passed the vector data to a Decision tree Classifier. Upon using _feature_importances__ attribute of ...
-1
votes
0answers
34 views

Target Encoding for test dataset

While training, i have used target encoding, to built some features, but i am wondering, how to encode features for test data-set? One way, i can recall, is to use training dataset, to encode ...
1
vote
1answer
56 views

How to plot data points and not centroids using sklearn k-means?

I have issues finding a way to plot data points colored by cluster with k-means. I have a very long list of strings.I managed to plot the centroids but not the data points; ...
-1
votes
0answers
13 views

HoG feature detector

I need to implement a keypoint matching using HoG descriptor, my first question: is it possible detect keypoint with HoG in opencv? If not, for example if I use SIFT keypoint detector how can I ...
0
votes
1answer
26 views

feature importance between two data distributions

I have two textual datasets collected from different domains (Twitter and Reddit). I extracted a set of features in the same way from these two datasets, one of these features as an example called ...
1
vote
1answer
33 views

How to merge personalized models together

Let's say I'm building an app like Uber and I want to predict the user's most likely destination based on the user's past history, current latitude-longitude, and current date and time. We have ...
1
vote
1answer
65 views

MR images segmentation for feature extraction

I have datasets of brain MR images with tumours, the tumours are already selected manually by a physicist using Image J. I have read about segmentation, but I still couldn't understand how do they ...
0
votes
1answer
166 views

How to handle noisy data?

I have X values such as [0.2, 0.1, 0.3, 0.5, 20, -0.2, -0.1, -0.6, -0.8, -30] Now I know that if the abs value is below a certain threshold, let's say 2, the y-...
2
votes
0answers
44 views

Layman's explanation of when to use which smoother algorithm/technique: FFT, loess, Savitzky-Golay, etc

As an analytics practitioner, I frequently come across noisy data (e.g. IoT data). When building a model or machine learning algorithm, it can be advantageous to smooth this data. Over the years, I ...
2
votes
1answer
33 views

How to calculate $\phi_{i,j}$ in VGG19 network?

In the paper Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network by Christian Ledig et al., the distance between images (used in the loss function) is calculated from ...
0
votes
1answer
16 views

How to do multivariate survival analysis on dataset having only categorical variables

I have a dataset where I have around 50 independent variables used to run survival analysis on the target variables. But out of these 50 variables, 46 variables are categorical variables i.e. having ...
1
vote
1answer
24 views

Grouping domain specific words/phrases with same meaning

I am looking at NLP methods to group together words/phrases which could have the same meaning. For example, in the sentence 'the table is broken' broken could be replaced by the following words/...
2
votes
0answers
52 views

How to tune parameters for Time Series Analysis, when forecasting is only dominated by one feature and error is not getting reduced?

I am trying to predict time series based on 150 features. When I plot correlation of these features, I am getting 20 features with more or less importance but every model I use, it is completely ...
0
votes
2answers
35 views

Extracting Useful features from large convolutional layers

I have been training a convolutional neural network on emotion detection. Now, I would like to extract features for my data to train an LSTM layer. In my case, the top convolutional layers in the ...
1
vote
1answer
21 views

What exactly does the model generation mean in this diagram?

I've been trying to grasp a research paper on image colorization using neural networks here I am stuck at this diagram. What I need help on, is the Model Generation step after Feature extraction. ...
1
vote
1answer
35 views

Mapping between original feature space and an interpretable feature space

I'm reading the following really interesting paper https://arxiv.org/pdf/1602.04938.pdf on local interpretable model explanations on page 3 however particularly section 3.3 Sampling for Local ...
0
votes
1answer
30 views

How to choose PCA or KernelPCA a priori?

I am learning about dimensionality reduction and I understood that one of the most used techniques in ML is PCA. If I understood correctly, I use PCA whenever I want to reduce the number of features ...
0
votes
1answer
20 views

Tool for test/train automation

I need to test different datasets as well as different algorithm implementations. The current workflow looks like: Perform feature extraction from train set Train classifier on this features Feed ...
2
votes
0answers
93 views

What are the rules when extracting SVO triples from preprocessed text?

If you have some already preprocessed text that is tagged, what are the rules to extract SVO triplets if you want a triple like (word, word, word). Can you give the sentence as example and extract all ...
1
vote
0answers
38 views

Unsupervized NER + sequence tagging to extract attributes from product descriptions

I am trying to extract attribute values from product descriptions in an unsupervised way. For example, given the product title "Variety pack fillet mignon and porterhouse steak dog food (12 count)", ...
1
vote
1answer
85 views

Help with reusing glove word embedding pretrained model

When using pretrained GloVe.6B for embedding generation, How can I get only the top most frequently used 100000 words rather than all the 4M words in the file?
1
vote
0answers
13 views

Model an infrequent linear feature in TensorFlow

I am trying to predict energy usage of homes. I have a feature (home square footage) that is highly useful when available, but is often not available (in which case it is 0). I want TensorFlow to ...
1
vote
1answer
84 views

Timestamps in Ridge Regression Scikit Learn

I am trying to transform data for use in regression, most likely the Ridge or Lasso technique implemented in sklearn.linear_model. My training data contains time ...
1
vote
1answer
179 views

How to use build_analyzer in sklearn feature extraction

I'm trying to get list of n-gram tokens for text Ex: 'How to use build_analyzer in sklearn feature extraction ' output :['How', 'use', 'build_analyzer', 'sklearn', 'feature', 'extraction', 'How use'...
2
votes
3answers
35 views

In machine learning how to find feature interdepencies? [closed]

Given a data set of N features, wherein some the features in this set were derived from other features from the same set, I am trying to discover inter dependencies between features (something like ...