Questions tagged [feature-extraction]

Variables (used for prediction or explication) used in regression or regression-like models (like clustering, discrimination). Use this tag for questions about constructing such variables or selecting the best among them.

Filter by
Sorted by
Tagged with
1
vote
0answers
14 views

Feature engineering one step at a time or in bunches?

Currently, I'm working on my very first classification project. If you want to know what dataset I'm working with, think "playing stairway to heaven in your local guitar store", and it will probably ...
0
votes
0answers
14 views

Upsell project based on sales records

in my company we are working on a upset project in which we are trying to solve the following problem: What we propose to our customer that he/she may be interested in based on the fact that he/she ...
1
vote
1answer
14 views

Text vectorizer that capture feature offset in the text?

I'm using sklearn Tfifdfvectorizer to extract feature from text towards text classification. I believe the information I need tends to be in the beginning of the document, so I would like to somehow ...
1
vote
0answers
22 views

Extract features from Decision tree leaf nodes

Recently came across a coursera course on "How to win Kaggle competitions" where they explain how we can engineer a categorical feature from each leaf node of the decision tree. [Video Link][1] I ...
0
votes
0answers
11 views

which data's are more probable to come together

consider the following table: ...
0
votes
0answers
19 views

Titanic Dataset - Feature Engineering - Ticket feature

I am currently building my first machine learning model using the titanic dataset. After the data exploration, I decided to focus my attention on the 'Ticket' feature. One thing I have noticed about ...
1
vote
0answers
12 views

C3D features perform poorlyon HMDB51

I have extracted C3D features for the HMDB51 dataset and used a SVM on top of that to classify videos. The accuracy on test set is around 0.2. I used C3D network pre-trained on sports1M dataset, and ...
0
votes
0answers
19 views

How to do feature Engineering for Online Data?

I have a dataset that contains user's banking transactions. Since this dataset contains categorical data like country and city of the user I used OneHot Encoding to convert these categorical data into ...
2
votes
1answer
33 views

Isn't one-hot encoding a waste of information? [duplicate]

I was just playing around with one-hot representations of features and thought of the following: Say that we're having 4 categories for a given feature (e.g. fruit) {Apple, Orange, Pear, Melon}. In ...
2
votes
0answers
12 views

features to help distinguish between document images

we are trying to build a model to classify different types of documents as the first step in our pipeline (final goal is to read all the text). Currently we use ImageNet to extract the features and ...
2
votes
1answer
24 views

Several independent variables based on the same underlying data

I got a data containing, among others, two feature variables, which are based from the same underlying data (i.e. have mutual information), but they convey different information/message. How to handle ...
1
vote
1answer
33 views

How to extract features from long chemical names?

I have an interesting problem that I am uncertain about how to even get started. I am working on a binary classifier that will take a chemical name, encoded as a string, and predict whether it is a '...
0
votes
0answers
15 views

CNN: What's the relationship between point clouds and features derived from point clouds?

What's the relationship between point clouds and features derived from point clouds? Particularly in CNN prediction. Particularly, I have point clouds about which I can imagine features that are ...
0
votes
0answers
29 views

Different extraction pipeline for train and test

I'm trying to create a production-ready ML model. The problem is as follows: Training data: Database A + Plus python aggregations. Testing data: Database B + Plus python aggregations. Both ...
0
votes
0answers
16 views

Extract specific Information from unstructured Documents

I am trying to build a system where recruiter will upload a doc file with Job Roles , Location , Experience , Title . the problem is every user will upload a different format document. Please Visit ...
0
votes
1answer
262 views

Find the top n features from feature set using absolute values of `feature_log_prob_ ` parameter of `MultinomialNB`

I am working on Donors choose dataset and have converted categorical, numerical and text features into vectors. I want to find the top 20 features from my 5095 features using absolute values of ...
1
vote
1answer
32 views

how to determine percentage below which a feature can be removed from a model

Let feature $feat$ contain one value $A$ that occurs 5% of the time, while 95% of the time it is empty. Instead of arbitrary saying features that have less than 5% should not be included into the ...
0
votes
2answers
108 views

How to get significance level for ranked features?

I am aware of below approaches of feature selection a) Feature Importance methods which are available in tree based models like Random Forest and ...
1
vote
2answers
53 views

What does many low important feature indicate?

I have a dataset where I am focusing on binary classification problem. In total,I have around 60 features in my dataset When I used Xgboost Feature Importance, I ...
0
votes
2answers
30 views

How to extract crucial features to create an image

Imagine, you have a dataset containing pictures of (example only, just to explain the task) cats and dogs. The data set is labeled, so we can train using supervised learning algorithms. My goal is to ...
0
votes
0answers
8 views

How to summarize/describe feature trends with fix number of scalars?

I'm trying to make feature extraction from features with numbers over different timespans (like stock values movement) and I want to encode these trends into a fix number of descriptive scalars. I ...
1
vote
1answer
471 views

How to extract features from the encoded layer of an autoencoder?

I have done some research on autoencoders, and I have come to understand that they can also be used for feature extraction (see this question on this site as an example). Most of the examples out ...
1
vote
0answers
29 views

Extracting Features for Graph transformation

Suppose I have a directed graph G (V,E) whose transformation is defined by a library of patterns. Each vertex is of particular type. The library of patterns contain subgraphs (g1,g2,g3 etc)and it's ...
8
votes
4answers
2k views

Is (manual) feature extraction outdated?

I recently attended a PhD thesis defence in which one committee members claimed that "manual feature extraction is outdated. Nowadays, we have [deep] machine learning models doing that job for us ...
0
votes
0answers
22 views

Information Gain & Gini Index for NLP

I know how Information Gain and Gini Index work in General. I have problem figuring out how to apply these techniques in NLP and text feature extraction. Can someone show me an example of how to ...
2
votes
1answer
56 views

What to do with large number of collinear variables?

I have this time-series dataset that has 63 features, out of which 57 were manually engineered. While checking for collinearity, I get this correlation matrix: As can be seen there are a number of ...
0
votes
0answers
16 views

Pipeline that cached the results

I use pandas to do feature extraction for machine learning. I hope to achieve the following: Consider I have five data processing steps done sequentially, and I execute it once, the results will be ...
0
votes
1answer
35 views

extract document topic vectors from lda model

how can I extract document-topic matrix from LDA model and use it as input features an svm classifier? I am using gensim for implementation
0
votes
0answers
13 views

Accelerometer and Gyroscope features

I am having accelerometer and gyroscope reading along x,y,z axis and want to get motion direction info at each time step. What all feature extraction would be best suited for this type of requirement. ...
0
votes
0answers
8 views

What techniques can I use to find position relationship in group of elements?

I have 14,000 tagged documents. These are custom forms that our employees create and fill out. I need to build a model that will be able to classify the types of each input field of the form in order ...
1
vote
0answers
13 views

What is the use of having shared weights in later layers of a CNN?

In a CNN, all the neurons in a single layer use the same weights and bias. As a result, all the neurons detect the same feature. The early layers of a CNN detect simple features like edges and hence ...
0
votes
1answer
23 views

feature extraction from single word for classification into nouns and names

I would like to write a NN that can classify different kinds of words(e.g. nouns,verbs,names) and am struggling to find information on how to do feature extraction on single words.For example, i would ...
0
votes
2answers
72 views

Feature extraction; similarity and classification of accelerometer data

I have several expert persons performing the same specific action (for example, squat or leap forward) multiple times. Say 5 persons do 100 squats each. They have an accelerometer attached to the same ...
3
votes
1answer
79 views

Feature selection is not that useful?

I've been doing a few DataScience competitions now, and i'm noticing something quite odd and frustrating to me. Why is it frustrating? Because , in theory, when you read about datascience it's all ...
1
vote
0answers
25 views

Applying the fourier transform

I have been trying to apply Fourier transform for two days, but haven't been able to because all the examples I've seen so far are not on actual dataset and use standard values that I don't understand ...
0
votes
1answer
83 views

Intuition behind the PCA algorithm

I am trying to understand PCA intuitively. Here it goes: After finding the eigenvectors and eigenvalues of the covariance matrix of the dataset, the eigenvalues will represent how spread out the ...
1
vote
1answer
101 views

Bag-of-words model : Boolean vs. TF-IDF

When I design a document classifier using traditional feature engineering, I would prefer (to Boolean model) tf-idf model to represent a document into a vector because intuitively Boolean model loses ...
1
vote
0answers
11 views

Bayesian classification of “JSON” data

"Machine Learning over JSON" describes some issues surrounding the classification of JSON documents. Namely, Categorical Features Data is Hierarchical Missingness is Chunky The first two have fairly ...
0
votes
1answer
25 views

Adding time as a feature with xgboost/random forests

I am trying to use xgboost for performing some regression and the features I have are rather simple and limited. I have the time stamp associated with some ...
0
votes
0answers
20 views

How to use skimage.feature.hog to calculate hog of training data where examples are stacked vertically

I have a $n*1024$ dimensional $2D$ array where n is number of examples which contain n images($32*32$) stacked vertically. I would like to calculate the hog of these images ...
2
votes
1answer
124 views

Feature extraction in audio spectrogram

I have audio and its spectrogram of the words in English language. (A spectrogram is a frequency domain representation of a signal) Consider the words: chain, change, chair, chapter. As you can notice,...
0
votes
0answers
25 views

What is the dimension reduction method to large numbers of independent features while only two of them are important?why?

What is the dimension reduction method to model a data with large numbers of independent features (for instance 5k features), while only two of them are important (are effective in cost function)? I ...
1
vote
1answer
31 views

Modeling the Price Movement- What analysis should be used

I am trying to model the price of a hotel as the check-in date arrives. I have a data set which looks like- For e.g- if I am looking at the booking date of Dec 31st, I would want to analyze the ...
0
votes
0answers
18 views

What are the common feature extraction technic to compare 2 sequence of timestamps?

I am building some predictive models for an online shopping site. I have timestamp log of customers' arrival time, time spent on a product's webpage, purchased or not, and a few others. I have tried ...
0
votes
0answers
25 views

What is the meaning of constraint two neural networks against each other?

I have two identical autoencoders with the same number of layers and parameters. I would like to find the similarity between two images from different domains such as an image captured from the camera ...
0
votes
1answer
218 views

Any issue with “overlapping” sliding windows in time-series data analysis?

I am developing some classification/regression models form accelerometry time-series data. So far, I have created datapoints by extracting features from non-overlapping sliding windows of the time-...
3
votes
1answer
69 views

Blind feature engineering

I received a dataset for analysis that had ~100 numeric columns with anonymous column names(X1, X2, X3, etc...) and asked to do a binary classification. My resulting classification algorithm using a ...
0
votes
0answers
82 views

How to time series forecast with multiple time series data sets on the same time series index

How does time series work with multiple time series data sets on the same index? For example, suppose I were a utilities company. Suppose I have the electricity usage of two homes, each indexed for ...
4
votes
1answer
130 views

How and Why to rescale image range between [0,1] and [-1,1]

I am trying to implement model described in Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network in which author says in section 3.2 that We scaled the range of the ...

1
2 3 4 5 6