Questions tagged [feature-extraction]

Variables (used for prediction or explication) used in regression or regression-like models (like clustering, discrimination). Use this tag for questions about constructing such variables or selecting the best among them.

Filter by
Sorted by
Tagged with
0
votes
0answers
17 views

Dimensionality Reduction on Cross Channel EEG data

I'm working on a project predicting seizures events using 10 minute EEG data recordings. This involves identifying the preictal (pre-seizure) phase from the interictal (normal/between seizure) phase, ...
0
votes
0answers
6 views

Factor Analysis with Mixed Data gives many components

I performed Factor Analysis with Mixed Data using PCAmixdata package from R. My dataset consists of 115000 records with 40 features of both categorical and continuous data. I checked the eigenvalues ...
1
vote
1answer
25 views

File path encoding to feature

I am trying to find some sort of encoding algorithm that would allow to transform system file paths eg. "c:/users/file1/subfile2/targetfile" into a feature that I could use in machine ...
1
vote
1answer
35 views

What are some good methods to forecast future revenue on categorical and value based data?

I have monthly snapshots (3 years) of all the contract data. It includes following information: Contract status [Categorical]: Proposed, tracked, submitted, won, lost, etc Contract stages [...
1
vote
1answer
17 views

Is there any problem with dropping only part of the OneHot generated features?

The one hot encoder adds more columns to the data, one for each category in the encoded feature. In the example below, the column City was transformed into 4 other ...
2
votes
1answer
35 views

Finding a range of values for each feature that contribute to positive classification

Consider a classification problem(lets say 2 classes, 'good' and 'bad), where all features are continuous. what I need is a range of values for each feature that contributes to 'good' classification....
0
votes
0answers
7 views

Extracting the features from several Auto Encoders

I am trying to extract the features of sparsed 3D pictures via a Convolutional Auto Encoder(CAE). Dou to the high computational costs I can not train the CAE over all the samples. I prepared couple of ...
1
vote
1answer
27 views

Importance of features

It is common to say in ML feature selection that features that are irrelevant in isolation can be important in combination with other features. Is there a simple example (one or two features) to ...
0
votes
0answers
9 views

Creating a new unique feature from existing features and using it instead in neural networks

Lets say I have 3 features a,b,c and lets say I am sure they vary between 0 to 100. Instead of feeding my neural network 3 features what happens if I create a unique number from this 3 numbers ...
1
vote
2answers
54 views

Extract information using NLP and store it in csv file

I have a text file that stores the pickup, drops, and time. SMS text is a dummy file that is used to train a cab service model. The text is like in this format: ...
0
votes
0answers
9 views

Do we train a feature extractor or we just use a trained network as a feature extractor?

I've just started to learn Tensorflow and Keras. I'm using Tensorflow 2.1.0 and Keras 2.3.7. I'm using this network as a feature extractor. Well, I'm trying to use it: ...
0
votes
0answers
8 views

Ensemble Model to Handle Different Image Attributes

I'm working on a project where I have images annotated across several attributes, say X, Y, Z...
0
votes
0answers
16 views

Categorical Feature with Most of the Data in a Single Category

I am a beginner with machine learning and I ran into something that made me question qualities for a good categorical feature in a regression model. Picture of Feature and Distribution Here. From the ...
4
votes
2answers
149 views

Mean estimation for nested location data

I want to estimate the average income for a location. I have nested data in the following way: A block is inside a neighborhood, which is inside a zipcode, which is inside a district, which is inside ...
0
votes
1answer
21 views

How to group categorical columns into similar types?

(Forgive me if the question is ill put. I am a novice in data science. Please comment or edit so that the question can be improved) I have a dataset where we have to predict the future sale of a shop....
0
votes
0answers
23 views

Feature Selection with non-linear numerical and categorical variables

I have a dataset of 45 non-linear numerical values and 2 categorical values. I am making a feature selection to predict categorical variables one by one or together. I used the correlation ratio and ...
0
votes
1answer
17 views

Context capturing in a Structured PDF?

I'm trying to extract resume (PDF) data. resumes always tend to follow a structure. so if you see some numbers in a cv; according to the context, we could tell whether its a telephone number, a ...
0
votes
1answer
21 views

Should we do Feature selection in parallel with feature engineering?

I'm working with LightGBM on a large data set about 3M row and about 8 columns. When i ...
1
vote
1answer
35 views

Understanding sklearn FeatureHasher

Wanting to understand "the hashing trick" I've written the following test code: ...
1
vote
1answer
50 views

List of keywords as features

I'm new to machine learning, being this the first time I'm involved in a project in the area. I have a dataset of news articles and have extracted the keywords present on the news title such as ...
1
vote
0answers
17 views

Assigning points to fitted planes

I’m working on a project involving fitting planes to 3D point clouds. The actual plane fitting part is working fine, but I’m trying to decide the best way to actually bound the fitted planes by the ...
1
vote
0answers
17 views

Feature engineering one step at a time or in bunches?

Currently, I'm working on my very first classification project. If you want to know what dataset I'm working with, think "playing stairway to heaven in your local guitar store", and it will probably ...
0
votes
0answers
15 views

Upsell project based on sales records

in my company we are working on a upset project in which we are trying to solve the following problem: What we propose to our customer that he/she may be interested in based on the fact that he/she ...
2
votes
1answer
17 views

Text vectorizer that capture feature offset in the text?

I'm using sklearn Tfifdfvectorizer to extract feature from text towards text classification. I believe the information I need tends to be in the beginning of the document, so I would like to somehow ...
1
vote
1answer
36 views

Extract features from Decision tree leaf nodes

Recently came across a coursera course on "How to win Kaggle competitions" where they explain how we can engineer a categorical feature from each leaf node of the decision tree. [Video Link][1] I ...
0
votes
0answers
11 views

which data's are more probable to come together

consider the following table: ...
0
votes
0answers
27 views

Titanic Dataset - Feature Engineering - Ticket feature

I am currently building my first machine learning model using the titanic dataset. After the data exploration, I decided to focus my attention on the 'Ticket' feature. One thing I have noticed about ...
1
vote
0answers
13 views

C3D features perform poorlyon HMDB51

I have extracted C3D features for the HMDB51 dataset and used a SVM on top of that to classify videos. The accuracy on test set is around 0.2. I used C3D network pre-trained on sports1M dataset, and ...
0
votes
0answers
19 views

How to do feature Engineering for Online Data?

I have a dataset that contains user's banking transactions. Since this dataset contains categorical data like country and city of the user I used OneHot Encoding to convert these categorical data into ...
2
votes
1answer
38 views

Isn't one-hot encoding a waste of information? [duplicate]

I was just playing around with one-hot representations of features and thought of the following: Say that we're having 4 categories for a given feature (e.g. fruit) {Apple, Orange, Pear, Melon}. In ...
2
votes
0answers
12 views

features to help distinguish between document images

we are trying to build a model to classify different types of documents as the first step in our pipeline (final goal is to read all the text). Currently we use ImageNet to extract the features and ...
2
votes
1answer
26 views

Several independent variables based on the same underlying data

I got a data containing, among others, two feature variables, which are based from the same underlying data (i.e. have mutual information), but they convey different information/message. How to handle ...
1
vote
1answer
36 views

How to extract features from long chemical names?

I have an interesting problem that I am uncertain about how to even get started. I am working on a binary classifier that will take a chemical name, encoded as a string, and predict whether it is a '...
0
votes
0answers
15 views

CNN: What's the relationship between point clouds and features derived from point clouds?

What's the relationship between point clouds and features derived from point clouds? Particularly in CNN prediction. Particularly, I have point clouds about which I can imagine features that are ...
0
votes
0answers
29 views

Different extraction pipeline for train and test

I'm trying to create a production-ready ML model. The problem is as follows: Training data: Database A + Plus python aggregations. Testing data: Database B + Plus python aggregations. Both ...
0
votes
0answers
18 views

Extract specific Information from unstructured Documents

I am trying to build a system where recruiter will upload a doc file with Job Roles , Location , Experience , Title . the problem is every user will upload a different format document. Please Visit ...
1
vote
1answer
34 views

how to determine percentage below which a feature can be removed from a model

Let feature $feat$ contain one value $A$ that occurs 5% of the time, while 95% of the time it is empty. Instead of arbitrary saying features that have less than 5% should not be included into the ...
0
votes
2answers
190 views

How to get significance level for ranked features?

I am aware of below approaches of feature selection a) Feature Importance methods which are available in tree based models like Random Forest and ...
1
vote
2answers
57 views

What does many low important feature indicate?

I have a dataset where I am focusing on binary classification problem. In total,I have around 60 features in my dataset When I used Xgboost Feature Importance, I ...
0
votes
2answers
30 views

How to extract crucial features to create an image

Imagine, you have a dataset containing pictures of (example only, just to explain the task) cats and dogs. The data set is labeled, so we can train using supervised learning algorithms. My goal is to ...
0
votes
0answers
8 views

How to summarize/describe feature trends with fix number of scalars?

I'm trying to make feature extraction from features with numbers over different timespans (like stock values movement) and I want to encode these trends into a fix number of descriptive scalars. I ...
3
votes
1answer
1k views

How to extract features from the encoded layer of an autoencoder?

I have done some research on autoencoders, and I have come to understand that they can also be used for feature extraction (see this question on this site as an example). Most of the examples out ...
1
vote
0answers
35 views

Extracting Features for Graph transformation

Suppose I have a directed graph G (V,E) whose transformation is defined by a library of patterns. Each vertex is of particular type. The library of patterns contain subgraphs (g1,g2,g3 etc)and it's ...
8
votes
4answers
2k views

Is (manual) feature extraction outdated?

I recently attended a PhD thesis defence in which one committee members claimed that "manual feature extraction is outdated. Nowadays, we have [deep] machine learning models doing that job for us ...
0
votes
1answer
48 views

Information Gain & Gini Index for NLP

I know how Information Gain and Gini Index work in General. I have problem figuring out how to apply these techniques in NLP and text feature extraction. Can someone show me an example of how to ...
2
votes
1answer
79 views

What to do with large number of collinear variables?

I have this time-series dataset that has 63 features, out of which 57 were manually engineered. While checking for collinearity, I get this correlation matrix: As can be seen there are a number of ...
1
vote
0answers
36 views

Pipeline that cached the results

I use pandas to do feature extraction for machine learning. I hope to achieve the following: Consider I have five data processing steps done sequentially, and I execute it once, the results will be ...
0
votes
1answer
44 views

extract document topic vectors from lda model

how can I extract document-topic matrix from LDA model and use it as input features an svm classifier? I am using gensim for implementation

1
2 3 4 5 6