Questions tagged [feature-construction]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1
vote
1answer
39 views

How to model a supervised recommender system with varying data

Suppose there are 2000 movies and a company wants to recommend some movies (for example, at most 5 movies) to each visitor. The objective is to learn how to predict which movie will be selected if a ...
0
votes
0answers
13 views

Converting the continuous numerical features into gaussian distribution and how to deal with NaN values after that?

I have a dataset in which there are few continuous numerical features that distribution over them is non gaussian and this means, skewness is nonzero (positive or negative). I read that it is good to ...
0
votes
2answers
33 views

Problem with a feature (normal distribution + peak around 0)

I have a feature that shows a characteristic of the instances. That characteristic can be present or not. If present it shows an almost normal distribution of values (actually a bit skewed to the ...
1
vote
0answers
31 views

Handling highly correlated features [closed]

I have a data set of transactions and want to build a fraud detection model (classifier). Only 3 variables are given that could be used as input features. The number of transactions during past 3, 6 ...
1
vote
0answers
23 views

Is there a common relationship between data inputs and the number of attainable features?

Is there a known relationship between the amount of information gain that comes from new data added to a dataset? for eg: If I have a plant watering system that tells me: An integer of how wet the ...
1
vote
1answer
28 views

Imputing features with NA values in classification task

I currently have a dataset where each observation is a person's traffic ticket history over districts. For each column, which represents a district: 1 represents that a person has received 1+ traffic ...
3
votes
1answer
57 views

How to handle a feature vector that could be variable length?

I would like to train a machine learning model with several features as input as X[] and with one output as Y. For example Every sample has a Data frame like this: ...
1
vote
1answer
31 views

Importance of features

It is common to say in ML feature selection that features that are irrelevant in isolation can be important in combination with other features. Is there a simple example (one or two features) to ...
0
votes
1answer
21 views

How to group categorical columns into similar types?

(Forgive me if the question is ill put. I am a novice in data science. Please comment or edit so that the question can be improved) I have a dataset where we have to predict the future sale of a shop....
0
votes
1answer
20 views

Is it suitable to change a feature by itself to generate an another feature?

Hi all I would love to hear your answers on this. Lets say I have two variables, voltage and current, in my data set. I could add another feature by squaring current (so as to calculate power). Is ...
0
votes
1answer
24 views

Representing user information

I have a task of representing a users feature matrix , i have features like gender , age etc but I also have a multivalue feature called as "movies watched" which is essentially another table of ...
0
votes
1answer
54 views

Feature importance and deriving rules using tree based classification models

I have a dataset where I have categorical and continuous values with targets 0/1 (binary classification task). Since I need to find patterns and relationships in the occurrence of the event or target, ...
1
vote
0answers
38 views

Extracting Features for Graph transformation

Suppose I have a directed graph G (V,E) whose transformation is defined by a library of patterns. Each vertex is of particular type. The library of patterns contain subgraphs (g1,g2,g3 etc)and it's ...
2
votes
2answers
152 views

Categorical features preprocessing for clustering

Can anyone tell suggest the best practice for clustering data with mixtured features (both with categorical and continuous). I am struggling with a problem; I realized that for all metrics algorithms ...
1
vote
1answer
73 views

NLP Feature creation from phrase matching

I'm building a model to classify email content, to decide whether the email should lead to a JIRA ticket being "Raised" or "Not Raised". The problem I am having is the data is highly imbalanced with ...
1
vote
1answer
67 views

Depending samples in ad ranking and click rate prediction

I am struggling with the following problem: Suppose we fit a machine learning model to model advertisers click rates. I used a Logistic Regression approach using a one-hot/dummy encoding. We have ...
0
votes
0answers
39 views

One feature - several units

I have a dataframe where one of the features is the Mileage expressed in some cases in $\frac{km}{l}$, while in others is expressed in $\frac{km}{kg}$, according to the combustion type of the car (so ...
1
vote
2answers
94 views

How to treat the undefined values which make sense?

I'm currently trying to create a few features to improve the performances of a model. One of those features that I would like to create corresponds to the difference in days between a customer's ...
1
vote
1answer
35 views

Potential problems with expanding training set

The problem is a binary classification one. My dataset contains users with activity over multiple days, where they all start with class 0 and can become class 1 after a certain activity (which is not ...
3
votes
2answers
4k views

Combining Latitude/Longitude position into single feature

I have been playing with 2 dimensional machine learning using pandas (Trying to do something like this: https://github.com/freeman-lab/spark-ml-streaming), and I'd like to combine Lat/Long into a ...
2
votes
2answers
40 views

Creating a metric based on some features

I want to create a new metric based on some features but dont know how to start. I basically want to create a "job satisfaction level" metric based on some features. The features could be work hours, ...
1
vote
2answers
812 views

What are features for state-action pairs in RL?

I read this answer: What are features in the context of reinforcement learning? But it only describes features for the state only in the context of cartpole, ie. Cart Position, Cart Velocity, Pole ...
1
vote
1answer
651 views

How to put multiple features into RNN input vector

I am trying to code a recurrent neural network (LSTM) to create music in python and was considering using multiple features instead of just the note pitch as an input into the network. Initially I had ...
3
votes
3answers
862 views

how to evaluate feature quality for decision tree model

Most of the tutorials assume that the features are known before generating the model and give no way to select 'good' feature and to discard 'bad' ones. The naive method is to test the model with new ...
2
votes
1answer
35 views

I want to create an additional feature(column) based on some manipulation of values from existing features

Consider my data-frame to be like this ('x','y','z' are features): I want to create a python function which will take an expression as a string (something like this: 'x+y-2z') and create a new ...
2
votes
0answers
147 views

Label Encode with pre defined classes

I have trained a model (Random Forest) and now I want to use it to predict for a certain data on a particular day. I have a categorical column where there are some values(say a,b,c,d,e) over a period. ...
3
votes
1answer
49 views

Using historical label as a feature in my ML model?

I am working on a predictive model to predict change in the price of an asset (up, down, no change). The labeling is based on the derivative of the price and is exponentially smoothed with an alpha of ...
1
vote
1answer
99 views

How to deal with Optional Input

I'm from the vision world and only worked with pixels from 0-255, ignoring any side effects. My current problem is different, in the way that I cannot rely on the input data. What my problem is: I ...
1
vote
2answers
218 views

how to represent location-code as a feature in machine learning model?

I am trying to predict the damage to a buildings after earthquake on a dataset which contains "district number" as feature. I think the feature will have a significant importance in predicting the ...
3
votes
2answers
4k views

Too much inputs = overfitting?

First question : can I mix different sorts of inputs types for example, height and age (of course my inputs are normalized)? in general, can we mix different types of inputs in a neural network ? ...
10
votes
4answers
2k views

Is this a good practice of feature engineering?

I have a practical question about feature engineering... say I want to predict house prices by using logistic regression and used a bunch of features including zip code. Then by checking the feature ...
1
vote
1answer
40 views

Making bigram features from a particular dataset

I have a folder which has a number of files which have a format like these ...
1
vote
0answers
32 views

Orange Feature Construction widget: Inability to see formulas while file sharing

Our team is sharing Orange "work bench" files across team members. Unfortunately, we find the Feature Construction widget does not display code consistently (nor is there any "x" to indicate an error)....
1
vote
1answer
185 views

Feature construction widget on Orange 3.13

I got stucked with this widget... I am an absolute beginner and I am now working with Orange software for my thesis with logs and core data. I need to combine different features to compare them. What ...
0
votes
1answer
55 views

How to represent relation between users as a feature?

I'm developing a model for unsupervised anomaly detection. I have a dataset representing communications between users (each example represents a communication): there are many features (time, duration,...
18
votes
3answers
6k views

How to combine categorical and continuous input features for neural network training

Suppose we have two kinds of input features, categorical and continuous. The categorical data may be represented as one-hot code A, while the continuous data is just a vector B in N-dimension space. ...
0
votes
1answer
19 views

Feature has a pattern in relation to class but does not enhance classifier to predict class

I have a feature x that when I plot it again my class variable y it shows some sort of pattern, i.e. it is obvious that x has a relation with y, but when I add the x to my logistic model it reduce the ...
2
votes
3answers
744 views

Why adding combinations of features would increase performance of linear SVM?

I have a dataset of ~5000 elements represented by vectors composed by ~30 binary values (0 or 1) on which I am performing binary classification with SVM with linear kernel (I use the Scikit learn lib)...
3
votes
0answers
74 views

What are best practices for collaborative feature engineering?

I work in a large company on several data science projects. For each of the projects me and my colleagues construct features that have some predictive value for the specific target in that project. ...
1
vote
1answer
316 views

Convolutional Neural networks

Hi all: I have a very fundamental question on how CNN works. I understand fully the training process as to take a bunch of images, start with random filters, convolve, activate, calculate loss, back ...
2
votes
1answer
582 views

Ordinal Integer variable vs Continuous Integer variable

I am working on titanic dataset. I have one feature Pclass which I understand is an ordinal variable having values 1,2 and 3. I have created a new feature cabin_int from feature Cabin, which is ...
1
vote
1answer
2k views

Need help in improving accuracy of text classification using Naive Bayes in nltk for movie reviews

I am referring http://www.nltk.org/book/ch06.html for generating a movie review classifier. It considers all words (Nouns, adjectives, verbs..) as part of feature set. I am trying to build a ...
0
votes
1answer
56 views

Some questions about feature hashing in the context of document classification

I'm trying to understand feature hashing, specifically in the context of document classification. I'm under the impression that it is useful because: it allows us to easily deal with 'new' words/...
2
votes
1answer
59 views

Dealing with a dataset where a subset of points live in a higher dimensional space

I have a classification problem where I am dealing with economic data in a high dimensional (~400) space which includes dates, addresses, salaries, and a number of other variables. Most of the ...
1
vote
1answer
273 views

Classification: How to manage data sets where one data row depends on another data row

I am trying to classify heading, image and image caption of a webpage. I am preparing data ...
0
votes
1answer
524 views

Classification engine with multiple Textual independent fields

I have made a classification engine with only one independent field (Comments) and classified them in multiple dependent variables. Now I want to have multiple independent variables in training data (...
5
votes
1answer
3k views

How to transform raw data to fixed-frequency time series?

How to transform raw data to fixed-frequency time series? For example I have the following raw data in DataFrame ...
-1
votes
1answer
79 views

Suitable aggregations (mean, median or something else) to make features?

Trying to solve a classification problem using a large number of features, some are individual numbers from a signal while others are aggregates of values to create a feature (median, mean etc). I ...
0
votes
2answers
953 views

Does the SVM require lots of features most of the time?

So I know about the curse of dimensionality (too many features too less data). Say I have a 3000 sample dataset, would 3 features be too less?
-1
votes
1answer
52 views

How do I create my features

I am working on the prediction of the behaviour of a new well in a map. The data that I have is a map in a form of a grid and its properties (9 of them), and a set of wells with the position of each ...