Questions tagged [feature-construction]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1
vote
1answer
27 views

NLP Feature creation from phrase matching

I'm building a model to classify email content, to decide whether the email should lead to a JIRA ticket being "Raised" or "Not Raised". The problem I am having is the data is highly imbalanced with ...
1
vote
1answer
48 views

Depending samples in ad ranking and click rate prediction

I am struggling with the following problem: Suppose we fit a machine learning model to model advertisers click rates. I used a Logistic Regression approach using a one-hot/dummy encoding. We have ...
0
votes
0answers
10 views
0
votes
0answers
19 views

One feature - several units

I have a dataframe where one of the features is the Mileage expressed in some cases in $\frac{km}{l}$, while in others is expressed in $\frac{km}{kg}$, according to the combustion type of the car (so ...
0
votes
0answers
16 views

How to handle “ordered” features?

I have a dataset with weekly sales figures, and trying to building a classification model (predict stock-out). I want to use some feature(s) to tell the machine that: Week 1 comes after week 0 ...
0
votes
2answers
28 views

How to treat the undefined values which make sense?

I'm currently trying to create a few features to improve the performances of a model. One of those features that I would like to create corresponds to the difference in days between a customer's ...
0
votes
1answer
30 views

Potential problems with expanding training set

The problem is a binary classification one. My dataset contains users with activity over multiple days, where they all start with class 0 and can become class 1 after a certain activity (which is not ...
0
votes
0answers
9 views

How to do feature engineering to the stripplot where the target, `tradeMoney`, has obviously lower than 5000 when 'rentType' is 'shared_rent'?

I am dealing with a house prediction problem. When I am doing EDAs I find the such stripplot() as follows: ...
2
votes
2answers
371 views

Combining Latitude/Longitude position into single feature

I have been playing with 2 dimensional machine learning using pandas (Trying to do something like this: https://github.com/freeman-lab/spark-ml-streaming), and I'd like to combine Lat/Long into a ...
2
votes
2answers
38 views

Creating a metric based on some features

I want to create a new metric based on some features but dont know how to start. I basically want to create a "job satisfaction level" metric based on some features. The features could be work hours, ...
0
votes
2answers
186 views

What are features for state-action pairs in RL?

I read this answer: What are features in the context of reinforcement learning? But it only describes features for the state only in the context of cartpole, ie. Cart Position, Cart Velocity, Pole ...
0
votes
1answer
31 views

How to represent unbounded number of items in a DNN

I have an Order defined as { "order_id": 123, "total_cost": 5000, "items": [ { "name": "Buffalo Wings", "cost": 1500 }, { "name": "Fries", "cost": 500 ...
1
vote
1answer
86 views

How to put multiple features into RNN input vector

I am trying to code a recurrent neural network (LSTM) to create music in python and was considering using multiple features instead of just the note pitch as an input into the network. Initially I had ...
3
votes
3answers
121 views

how to evaluate feature quality for decision tree model

Most of the tutorials assume that the features are known before generating the model and give no way to select 'good' feature and to discard 'bad' ones. The naive method is to test the model with new ...
2
votes
1answer
34 views

I want to create an additional feature(column) based on some manipulation of values from existing features

Consider my data-frame to be like this ('x','y','z' are features): I want to create a python function which will take an expression as a string (something like this: 'x+y-2z') and create a new ...
1
vote
0answers
78 views

Label Encode with pre defined classes

I have trained a model (Random Forest) and now I want to use it to predict for a certain data on a particular day. I have a categorical column where there are some values(say a,b,c,d,e) over a period. ...
2
votes
1answer
35 views

Using historical label as a feature in my ML model?

I am working on a predictive model to predict change in the price of an asset (up, down, no change). The labeling is based on the derivative of the price and is exponentially smoothed with an alpha of ...
1
vote
1answer
28 views

How to deal with Optional Input

I'm from the vision world and only worked with pixels from 0-255, ignoring any side effects. My current problem is different, in the way that I cannot rely on the input data. What my problem is: I ...
1
vote
2answers
63 views

how to represent location-code as a feature in machine learning model?

I am trying to predict the damage to a buildings after earthquake on a dataset which contains "district number" as feature. I think the feature will have a significant importance in predicting the ...
3
votes
2answers
2k views

Too much inputs = overfitting?

First question : can I mix different sorts of inputs types for example, height and age (of course my inputs are normalized)? in general, can we mix different types of inputs in a neural network ? ...
10
votes
3answers
1k views

Is this a good practice of feature engineering?

I have a practical question about feature engineering... say I want to predict house prices by using logistic regression and used a bunch of features including zip code. Then by checking the feature ...
1
vote
1answer
30 views

Making bigram features from a particular dataset

I have a folder which has a number of files which have a format like these ...
1
vote
0answers
23 views

Orange Feature Construction widget: Inability to see formulas while file sharing

Our team is sharing Orange "work bench" files across team members. Unfortunately, we find the Feature Construction widget does not display code consistently (nor is there any "x" to indicate an error)....
1
vote
0answers
38 views

Feature Engineering for POS register events: Anomaly Detection

I am working on a dataset with only one variable: POS Journal Events.It has different values such as Items ordered, Order Placed, OrderItems,Discount applied, Promotion applied, order voided etc. I ...
1
vote
1answer
86 views

Feature construction widget on Orange 3.13

I got stucked with this widget... I am an absolute beginner and I am now working with Orange software for my thesis with logs and core data. I need to combine different features to compare them. What ...
0
votes
1answer
46 views

How to represent relation between users as a feature?

I'm developing a model for unsupervised anomaly detection. I have a dataset representing communications between users (each example represents a communication): there are many features (time, duration,...
9
votes
1answer
2k views

How to combine categorical and continuous input features for neural network training

Suppose we have two kinds of input features, categorical and continuous. The categorical data may be represented as one-hot code A, while the continuous data is just a vector B in N-dimension space. ...
0
votes
1answer
17 views

Feature has a pattern in relation to class but does not enhance classifier to predict class

I have a feature x that when I plot it again my class variable y it shows some sort of pattern, i.e. it is obvious that x has a relation with y, but when I add the x to my logistic model it reduce the ...
2
votes
3answers
349 views

Why adding combinations of features would increase performance of linear SVM?

I have a dataset of ~5000 elements represented by vectors composed by ~30 binary values (0 or 1) on which I am performing binary classification with SVM with linear kernel (I use the Scikit learn lib)...
2
votes
0answers
39 views

What are best practices for collaborative feature engineering?

I work in a large company on several data science projects. For each of the projects me and my colleagues construct features that have some predictive value for the specific target in that project. ...
1
vote
1answer
300 views

Convolutional Neural networks

Hi all: I have a very fundamental question on how CNN works. I understand fully the training process as to take a bunch of images, start with random filters, convolve, activate, calculate loss, back ...
2
votes
1answer
390 views

Ordinal Integer variable vs Continuous Integer variable

I am working on titanic dataset. I have one feature Pclass which I understand is an ordinal variable having values 1,2 and 3. I have created a new feature cabin_int from feature Cabin, which is ...
0
votes
1answer
1k views

Need help in improving accuracy of text classification using Naive Bayes in nltk for movie reviews

I am referring http://www.nltk.org/book/ch06.html for generating a movie review classifier. It considers all words (Nouns, adjectives, verbs..) as part of feature set. I am trying to build a ...
0
votes
1answer
42 views

Some questions about feature hashing in the context of document classification

I'm trying to understand feature hashing, specifically in the context of document classification. I'm under the impression that it is useful because: it allows us to easily deal with 'new' words/...
2
votes
1answer
52 views

Dealing with a dataset where a subset of points live in a higher dimensional space

I have a classification problem where I am dealing with economic data in a high dimensional (~400) space which includes dates, addresses, salaries, and a number of other variables. Most of the ...
1
vote
1answer
225 views

Classification: How to manage data sets where one data row depends on another data row

I am trying to classify heading, image and image caption of a webpage. I am preparing data ...
-1
votes
1answer
216 views

Classification engine with multiple Textual independent fields

I have made a classification engine with only one independent field (Comments) and classified them in multiple dependent variables. Now I want to have multiple independent variables in training data (...
5
votes
1answer
3k views

How to transform raw data to fixed-frequency time series?

How to transform raw data to fixed-frequency time series? For example I have the following raw data in DataFrame ...
-1
votes
1answer
67 views

Suitable aggregations (mean, median or something else) to make features?

Trying to solve a classification problem using a large number of features, some are individual numbers from a signal while others are aggregates of values to create a feature (median, mean etc). I ...
0
votes
2answers
611 views

Does the SVM require lots of features most of the time?

So I know about the curse of dimensionality (too many features too less data). Say I have a 3000 sample dataset, would 3 features be too less?
-1
votes
1answer
50 views

How do I create my features

I am working on the prediction of the behaviour of a new well in a map. The data that I have is a map in a form of a grid and its properties (9 of them), and a set of wells with the position of each ...
8
votes
2answers
11k views

List of feature engineering techniques

Is there any resource with a list of feature engineering techniques? A mapping of type of data, model and feature engineering technique would be a gold mine
1
vote
1answer
43 views

impact of old reviews on new reviews

I am working on e-commerce, how to rank smartphones in the same category. I want to calculate a weighted average of sentiment score from reviews posted by buyers. Weights will be based on how many ...
0
votes
1answer
199 views

Time-stamp for linear model

How can we extract information from time-stamp variable for modelling? I have a variable with format mm-dd-yyyy hh:mm:ss I want to predict an outcome variable using time-stamp as input variable. I do ...
3
votes
1answer
64 views

Is it a good idea to train with a feature which value will be fixed in future predictions?

I am facing a regression problem and I have one feature that has some relevant correlation with the output. The value of this feature will be fixed in ALL the predictions I will use this model for. ...
4
votes
2answers
5k views

How to deal with categorical feature of very high cardinality?

I would like to train a binary classifier on feature vectors. One of the features is categorical feature with string, it is the zip codes of a country. Typically, there is thousands of zip codes, and ...
9
votes
2answers
2k views

What to do when testing data has less features than training data?

Let's say we are predicting the sales of a shop and my training data has two sets of features: One about the store sales with the dates (the field "Store" is not unique) One about the store types (...
3
votes
1answer
2k views

Is there any difference between feature extraction and feature learning?

It appears to me that "feature extraction" and "feature learning" are equivalent concepts, however there are 2 separate wikipedia articles dedicated to them that are notably different. In particular, ...
28
votes
6answers
8k views

Are there any tools for feature engineering?

Specifically what I am looking for are tools with some functionality, which is specific to feature engineering. I would like to be able to easily smooth, visualize, fill gaps, etc. Something similar ...
4
votes
2answers
130 views

Approach to creating a user profile in music web application

I am working on a use case, and I'm unsure of the best way to proceed: in order to analyze the behavior of users of a web-based music application, we retain all songs each has played since 2009. We ...