Questions tagged [feature-extraction]

Variables (used for prediction or explication) used in regression or regression-like models (like clustering, discrimination). Use this tag for questions about constructing such variables or selecting the best among them.

Filter by
Sorted by
Tagged with
1
vote
1answer
43 views

What exactly does the model generation mean in this diagram?

I've been trying to grasp a research paper on image colorization using neural networks here I am stuck at this diagram. What I need help on, is the Model Generation step after Feature extraction. ...
1
vote
1answer
161 views

Mapping between original feature space and an interpretable feature space

I'm reading the following really interesting paper https://arxiv.org/pdf/1602.04938.pdf on local interpretable model explanations on page 3 however particularly section 3.3 Sampling for Local ...
0
votes
1answer
65 views

How to choose PCA or KernelPCA a priori?

I am learning about dimensionality reduction and I understood that one of the most used techniques in ML is PCA. If I understood correctly, I use PCA whenever I want to reduce the number of features ...
1
vote
1answer
46 views

Tool for test/train automation

I need to test different datasets as well as different algorithm implementations. The current workflow looks like: Perform feature extraction from train set Train classifier on this features Feed ...
3
votes
0answers
199 views

What are the rules when extracting SVO triples from preprocessed text?

If you have some already preprocessed text that is tagged, what are the rules to extract SVO triplets if you want a triple like (word, word, word). Can you give the sentence as example and extract all ...
1
vote
0answers
72 views

Unsupervized NER + sequence tagging to extract attributes from product descriptions

I am trying to extract attribute values from product descriptions in an unsupervised way. For example, given the product title "Variety pack fillet mignon and porterhouse steak dog food (12 count)", ...
1
vote
2answers
151 views

Help with reusing glove word embedding pretrained model

When using pretrained GloVe.6B for embedding generation, How can I get only the top most frequently used 100000 words rather than all the 4M words in the file?
1
vote
0answers
18 views

Model an infrequent linear feature in TensorFlow

I am trying to predict energy usage of homes. I have a feature (home square footage) that is highly useful when available, but is often not available (in which case it is 0). I want TensorFlow to ...
2
votes
1answer
421 views

Timestamps in Ridge Regression Scikit Learn

I am trying to transform data for use in regression, most likely the Ridge or Lasso technique implemented in sklearn.linear_model. My training data contains time ...
2
votes
1answer
908 views

How to use build_analyzer in sklearn feature extraction

I'm trying to get list of n-gram tokens for text Ex: 'How to use build_analyzer in sklearn feature extraction ' output :['How', 'use', 'build_analyzer', 'sklearn', 'feature', 'extraction', 'How use'...
2
votes
3answers
57 views

In machine learning how to find feature interdepencies? [closed]

Given a data set of N features, wherein some the features in this set were derived from other features from the same set, I am trying to discover inter dependencies between features (something like ...
3
votes
3answers
2k views

What is the intuition behind using 2 consecutive convolutional filters in a Convolutional Neural Network?

I understand the purpose of Convolutional filters (or kernels). I visualize them as learnable feature extractors. E.g. Extract vertical edges or horizontal edges, etc. Could somebody kindly explain ...
2
votes
0answers
282 views

Data augmentation / feature extraction on pre-trained convnets

I'm reading 'Deep Learning with Python' by François Chollet, which is an excellent book. He talks about using pre-trained convnets (in his example, VGG16) and then running smaller datasets to tweak ...
1
vote
0answers
83 views

How does Stanford CRF encode NER string features?

Most features created by the NERFeatureFactory are strings e.g. from usePrev, useNext, ...
3
votes
1answer
49 views

Using historical label as a feature in my ML model?

I am working on a predictive model to predict change in the price of an asset (up, down, no change). The labeling is based on the derivative of the price and is exponentially smoothed with an alpha of ...
1
vote
0answers
33 views

Training Gaussian Restricted Boltzmann Machines with Noisy Rectified (nrelu or ssu) linear hidden units

I'm not sure how to implement this architecture. I'm following this thesis (pages 17-19) or this paper but I'm not sure how to train it. I want to use this to extract features from raw audio. I know I ...
1
vote
2answers
201 views

how to represent location-code as a feature in machine learning model?

I am trying to predict the damage to a buildings after earthquake on a dataset which contains "district number" as feature. I think the feature will have a significant importance in predicting the ...
4
votes
1answer
1k views

Can RNN be used for feature extraction?

I was reading this paper and my question is related to it. What I am stuck at is the intuition behind using two CNNs for feature extraction. Can just RNN not be used for feature extraction as well as ...
3
votes
1answer
145 views

Extract features from a survey

I need to use the answers from a questionnaire for training a classifier. I discovered that some questions can have nested sub-questions.. Let's say (just an example) that I want to predict whether a ...
0
votes
2answers
83 views

What to do if my target variable is column of lists? [closed]

How I can transform my target variable(Y)? As it is list, I cann`t use it for fitting model, because I must use integers for fitting.
6
votes
3answers
14k views

Using python and machine learning to extract information from an invoice? Inital dataset? [closed]

DISCLAIMER: I have absolutely no background with machine learning/data science, and am unfamiliar with the general lingo of data science, so please bear with me. I'm trying to make a machine learning ...
1
vote
1answer
95 views

What's the best way to select features independent of the model being used?

I am using tensorflow's DNNRegressor to model a multivariate regression problem. I want to form an optimal feature subset from a mixed bag of categorical and continuous features. What would be the ...
7
votes
4answers
1k views

Feature extraction from a scatter plot

Say I have a scatter plot like this: Since I have many scatter plots like this I want to do feature transformation i.e. squash (x,y) in a single term to be input ...
3
votes
4answers
8k views

Time series feature extraction from raw sensor data for classification?

I have a tabular raw data from sensors with associated label and i want to extract the time series features like mean,max,min and std from the data all the sensor data and form another table or export ...
1
vote
0answers
40 views

Features Extraction

I'm trying to compare two images (the first one is the ID's image and the second one is a selfie taken by phone) so I am wondering if can we extract features from the two images by using CNN and ...
2
votes
0answers
364 views

DBSMOTE on Short Text Classification

I am trying to use DBSMOTE(Density-Based Synthetic Oversampling TEqnique) to on a data set of short text--tweets to be specific. This will be used to train a classifier model in a multiclass ...
1
vote
1answer
178 views

Fitting and transforming text data in training, testing, and validation sets

I'm trying to implement a simple text classifier wherein the data is split into training (70%) and testing (30%) sets, but cross validation (k=10) to be performed on the training set. My main concern ...
0
votes
1answer
55 views

How to represent relation between users as a feature?

I'm developing a model for unsupervised anomaly detection. I have a dataset representing communications between users (each example represents a communication): there are many features (time, duration,...
0
votes
1answer
106 views

BOVW - Combine vocabularies

What I have so far I have a set of images that I am trying to classify. I can also extract different feature descriptors from the images using algorithms such as hu moments, color histogram, and SIFT....
3
votes
1answer
79 views

Does it make sense to “reorder” a categorical feature to make it monotonic?

Sorry for the vagueness of the title; I'll explain what I mean. I'm doing the Kaggle Titanic beginner tutorial. The label you're interested in is the "survived" rate (0 or 1), which you want to ...
1
vote
0answers
501 views

How to extract relative importance of features from a tensorflow DNNRegressor model?

I followed these two posts to understand about restoring a saved model and then extracting variables from it: Extracting weights values from a tensorflow model checkpoint How to examine the feature ...
2
votes
2answers
192 views

Feature Engineering of mixed data type column

I have a data set in which I have to predict the price of a building. Among many features there is a feature called Availability which has two type values like : ...
1
vote
0answers
18 views

Converting MFCC to FFT

I have 128 bin FFT data for an audio file. I want to convert these FFT into MFCC of size 14, 20, 40. Is there some library which converts directly FFT to MFCC. P.S - I dont have the actual audio file....
1
vote
0answers
551 views

How to concatenate feature vectors of different dimensions?

I have been using different deep learning models and extracting features from different layers for the given images. My code goes like this: ...
1
vote
0answers
92 views

Incorporating new features in document similarity task

I have a model pipeline for finding similar text documents given an input query text. The model is very simple; I have a corpus of documents on which I train a TfIDF model. When a query is input, we ...
0
votes
4answers
777 views

Feature extraction from pure text

I have a dataset (~52k rows) with a column containing just pure sentences (upper and lowercase, with punctuation and stop words) in each row. What can I do to represent this data in a meaningful way ...
-1
votes
1answer
264 views

Suitable Autoencoder for Activity Recognition dataset Feature Extraction

I have text data representing sensor outputs. Dataset: ...
0
votes
1answer
234 views

Proper/Possible methods for extracting unstructured data from websites

I'm working in Python, using Scrapy, and NLTK to try to understand how I can extract data from college websites. My scraper can navigate through the university websites and find their tuition fees ...
3
votes
1answer
151 views

What is the risk of removing a feature with low correlation…?

I'm running a linear regression model as baseline for a specific estimation problem. Based on the resulting R-squared, regressor coefficients and their respective p-values, I can conclude that ...
18
votes
5answers
25k views

Feature selection vs Feature extraction. Which to use when?

Feature extraction and feature selection essentially reduce the dimensionality of the data, but feature extraction also makes the data more separable, if I am right. Which technique would be ...
1
vote
1answer
57 views

How to manipulate this column of less than/greater than?

I have a dataset in which one column is as given in the picture. What will be the best way to handle such a column?
1
vote
0answers
22 views

How is Kernel Matrix on a distribution defined?

Consider the following words taken from the lecture notes: The Hilbert-Schmidt Independence Criterion (HSIC) measures the dependence of the two random variables $X$ and $Y$. An empirical estimate of ...
3
votes
0answers
70 views

What are best practices for collaborative feature engineering?

I work in a large company on several data science projects. For each of the projects me and my colleagues construct features that have some predictive value for the specific target in that project. ...
2
votes
1answer
914 views

Under what conditions should an autoencoder be chosen over kernel PCA?

I've recently been looking at autoencoders and kernel PCA for unsupervised feature extraction. Lets consider just for a moment linear PCA. Its my understanding that if a autoencoder (with a single ...
1
vote
1answer
47 views

Features for blink detection in real-time single channel EEG [closed]

I am looking to detect blink events in real-time single channel EEG. Classification of a moving window of samples to determine whether a blink artifact exists requires feature extraction (except when ...
1
vote
1answer
384 views

Clustering combining numeric features and weekday & hour cyclic features

The question is strictly related to What is a good way to transform Cyclic Ordinal attributes? and Ways to deal with longitude/latitude feature They presented a very clear answer about the approach ...
-1
votes
2answers
362 views

How to find categorical features from a vector representation of text?

The context of the question: I have a pandas dataframe where one column has text values and others have categorical values. I trained a word2vec model with ...
4
votes
3answers
12k views

How to get spike values from a value sequence?

I have pile of vectors where the values could be plotted like this: Now I want to extract the "spike values" (over a certain threshold say 15,000). In this case there is fifteen. How could this be ...
5
votes
1answer
2k views

Extract feature vector of a CNN

How can I get the feature vector of my dataset. I have a fine-tuned CNN model with my data. Now I want to feed the features of all my dataset extracted from the last layer of the CNN into a LSTM. So ...
1
vote
1answer
315 views

Convolutional Neural networks

Hi all: I have a very fundamental question on how CNN works. I understand fully the training process as to take a bunch of images, start with random filters, convolve, activate, calculate loss, back ...

1 2 3
4
5
7