Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [feature-construction]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
2 answers
47 views

Time Series forecasting feature creation/engineering

I'm new to time-series-forecasting and was wondering, whether in a single variable forecast e.g.: X -> Y the creation of additional features of X leads to an improvement when training. So if adding ...
user159972's user avatar
0 votes
0 answers
18 views

How can I extract useful data out of a molecule?

In my dataframe, I have a feature with molecules like C1=CC=C(C=C1)C(=O)OC2=CC=CC3=C2C=CC=C3O, and C1=CC=C(C=C1)CCCNC(=O)/C(=C/C2=CC(=C(C=C2)O)O)/C#N. I have no experience with RDKit and deepchem, and ...
Tanmay Sharma's user avatar
0 votes
1 answer
23 views

How to add lagging Features for Forecasting with a random lag range without adding a new column per lag?

The common way of adding lagging features in time series forecasting problems is adding lag columns with pandas.shift(). While it is a fine method but what about when wanting to use a random integer (...
Emad Ezzeldin's user avatar
0 votes
2 answers
54 views

Encode 10k features where each feature is having more than 500 categories

I have around 10k features in my dataset and each feature is having more than 500 categories. what is the best encoding method to convert this categorical features to vector form? "span_dir":...
khushi's user avatar
  • 111
0 votes
0 answers
9 views

Deriving consistency feature for a student using a study app over the days

I want to build a recommendation engine for the revision app. Basic Idea After each module we will ask student questions and based on the correctness of their answers we will decide after how many ...
Keshav Raj's user avatar
0 votes
0 answers
31 views

propper feature encoding

I working with the following data set also here is it's detailed description of "packet_dat" column I can't understand how I can encode packet_dat column into proper feature so my ...
Roma's user avatar
  • 101
0 votes
0 answers
18 views

How do I reconstruct irregularly spaced lat-lon data (in the form of a matrix) using spherical harmonic fitting?

Introductory Links to spherical harmonics give a mildly rigorous introduction to the concept; but very few links describe how to actually use them. I have latitude-longitude data arranged the ...
requiemman's user avatar
0 votes
0 answers
61 views

Effect of removing duplicate and identical entries on dimensionality reduction

I have huge data with thousands of observations and millions of features. I need to do clustering so I use PCA/t-SNE/UMAP for dimensionality reduction followed by K-Means. Currently, I retain only ...
Quiescent's user avatar
  • 101
1 vote
2 answers
45 views

Tips for scraping crypto data in the right way

I am scraping data from crypto site and want to use neural network algorithm for predicting data. the way i save data is like these: and there is bunch of other features like open/high/low/close for ...
mohammad ariyan rad's user avatar
0 votes
0 answers
56 views

How to structure spatio-temporal data for LSTM model input

I have a dataset representing species distribution (binary presence/absence) and I would like to use a LSTM to predict future species distribution. I am having trouble working out how I should shape ...
SG3141's user avatar
  • 27
2 votes
0 answers
81 views

Feature selection and model performance

Featuretools provides an automated way to generate features from your data, by providing relationships within your data and applying their so-called deep feature synthesis. It generates features like ...
holzben's user avatar
  • 121
0 votes
1 answer
191 views

Feature creation ideas for propensity models?

I'm working on a propensity model, predicting whether customers would buy or not. While doing exploratory data analysis, I found that customers have a buying pattern. Most customers repeat the ...
NAS_2339's user avatar
  • 243
0 votes
1 answer
24 views

How to represent a time duration feature for cases where time is still counting

I have a problem where I am trying to classify the outcome of costumer complaint cases. I have several features already such as type of item bought, reason for complaint etc... I am trying to add a ...
MiguelL's user avatar
1 vote
0 answers
22 views

Best way to represent a version feature based on percentiles

We're training a binary classifier in AutoML, and one of the features consist of browser versions. Currently these versions are provided "normalized" to the model, according to the ...
Gabriel Ballesteros's user avatar
1 vote
1 answer
206 views

Algorithms for casual feature selection for continuous Y

Currently I have been trying to find some good algorithms for feature selection. Using correlation or other non casual type of method will not be the right way to do a feature selection. I'm am ...
minattosama's user avatar
3 votes
1 answer
267 views

Finding if an outcome is predictable

Suppose we are asked to predict something given a set of features, how do we know if that target is actually predictable? That is, how do we know if there is actually some relation between the ...
Bharathi A's user avatar
1 vote
2 answers
906 views

How to model a 3D graph into a vector so that I can feed it into a classification algorithm?

I have a 3D graph like below: Ref: google images It has 2 angles as X and Y and the Z axis is amplitude value (Each 3D graph is representing a pixel). I want to model this into some useful data ...
Tarun Maganti's user avatar
1 vote
0 answers
15 views

Tsallis entropy - advice needed regarding obtaining probability distribution

As is always the way I stumbled across Tsallis entropy on SO whilst looking for something completely different. This soon lead me reading all sorts of interesting but terse academic papers. I am ...
Little Code's user avatar
1 vote
0 answers
137 views

How to add more weight to certain features?

I have extracted features from two types of signals. Prior to merging them to create one feature vector, I have computed an importance score of every feature within that type of signal. I would like ...
ShengLi's user avatar
  • 11
1 vote
0 answers
23 views

Regression with a feature which has its own depth

I'm relatively new to ML/Statistical Analysis, and I'm facing a dataset structured like this ...
ffxx68's user avatar
  • 11
2 votes
1 answer
117 views

How do I combine predictions from classifiers for two different problem?

I am working on a classification problem for predicting whether the shipment is going to be late or not. I would say the classifier is mediocre at predicting the positive class at the moment. But the ...
Jas999's user avatar
  • 21
1 vote
0 answers
61 views

Best Feature extraction for at the end retrieving audio

I work on a machine learning algo, which basically learns sequences in an audio .wav and generates the most “logical” sequences. The algorithm learns features, so I generate MFCCs from the audio file. ...
Martin Guilbert Lejeune's user avatar
1 vote
0 answers
78 views

Data Lineage/Traceability in Pipelines

I want to collect information about: 1) from which single data signals a feature is composed in a ML pipeline and 2) what data preprocessing operations are/were executed on a data signal. Does anyone ...
Gustav1985's user avatar
1 vote
1 answer
58 views

How to model a supervised recommender system with varying data

Suppose there are 2000 movies and a company wants to recommend some movies (for example, at most 5 movies) to each visitor. The objective is to learn how to predict which movie will be selected if a ...
Aaron's user avatar
  • 201
1 vote
2 answers
98 views

Problem with a feature (normal distribution + peak around 0)

I have a feature that shows a characteristic of the instances. That characteristic can be present or not. If present it shows an almost normal distribution of values (actually a bit skewed to the ...
Mario Tormo's user avatar
1 vote
0 answers
59 views

Handling highly correlated features [closed]

I have a data set of transactions and want to build a fraud detection model (classifier). Only 3 variables are given that could be used as input features. The number of transactions during past 3, 6 ...
user2348674's user avatar
1 vote
0 answers
24 views

Is there a common relationship between data inputs and the number of attainable features?

Is there a known relationship between the amount of information gain that comes from new data added to a dataset? for eg: If I have a plant watering system that tells me: An integer of how wet the ...
BFG.Digital's user avatar
1 vote
1 answer
43 views

Imputing features with NA values in classification task

I currently have a dataset where each observation is a person's traffic ticket history over districts. For each column, which represents a district: 1 represents that a person has received 1+ traffic ...
Eisen's user avatar
  • 281
3 votes
1 answer
1k views

How to handle a feature vector that could be variable length?

I would like to train a machine learning model with several features as input as X[] and with one output as Y. For example Every sample has a Data frame like this: ...
Crazy9's user avatar
  • 31
1 vote
1 answer
38 views

Importance of features

It is common to say in ML feature selection that features that are irrelevant in isolation can be important in combination with other features. Is there a simple example (one or two features) to ...
user98580's user avatar
0 votes
2 answers
46 views

How to group categorical columns into similar types?

(Forgive me if the question is ill put. I am a novice in data science. Please comment or edit so that the question can be improved) I have a dataset where we have to predict the future sale of a shop....
samarendra chandan bindu Dash's user avatar
0 votes
1 answer
27 views

Is it suitable to change a feature by itself to generate an another feature?

Hi all I would love to hear your answers on this. Lets say I have two variables, voltage and current, in my data set. I could add another feature by squaring current (so as to calculate power). Is ...
Isara De Silva's user avatar
0 votes
1 answer
33 views

Representing user information

I have a task of representing a users feature matrix , i have features like gender , age etc but I also have a multivalue feature called as "movies watched" which is essentially another table of ...
data101's user avatar
  • 191
0 votes
1 answer
67 views

Feature importance and deriving rules using tree based classification models

I have a dataset where I have categorical and continuous values with targets 0/1 (binary classification task). Since I need to find patterns and relationships in the occurrence of the event or target, ...
sayan_sen's user avatar
  • 101
1 vote
0 answers
41 views

Extracting Features for Graph transformation

Suppose I have a directed graph G (V,E) whose transformation is defined by a library of patterns. Each vertex is of particular type. The library of patterns contain subgraphs (g1,g2,g3 etc)and it's ...
cjMec's user avatar
  • 141
2 votes
2 answers
390 views

Categorical features preprocessing for clustering

Can anyone tell suggest the best practice for clustering data with mixtured features (both with categorical and continuous). I am struggling with a problem; I realized that for all metrics algorithms ...
Anton  Moskvin's user avatar
1 vote
1 answer
104 views

NLP Feature creation from phrase matching

I'm building a model to classify email content, to decide whether the email should lead to a JIRA ticket being "Raised" or "Not Raised". The problem I am having is the data is highly imbalanced with ...
Sql_Pete_Belfast's user avatar
1 vote
1 answer
111 views

Depending samples in ad ranking and click rate prediction

I am struggling with the following problem: Suppose we fit a machine learning model to model advertisers click rates. I used a Logistic Regression approach using a one-hot/dummy encoding. We have ...
user1488793's user avatar
0 votes
0 answers
668 views

One feature - several units

I have a dataframe where one of the features is the Mileage expressed in some cases in $\frac{km}{l}$, while in others is expressed in $\frac{km}{kg}$, according to the combustion type of the car (so ...
89f3a1c's user avatar
  • 369
3 votes
2 answers
521 views

How to treat the undefined values which make sense?

I'm currently trying to create a few features to improve the performances of a model. One of those features that I would like to create corresponds to the difference in days between a customer's ...
qwertzuiop's user avatar
1 vote
1 answer
53 views

Potential problems with expanding training set

The problem is a binary classification one. My dataset contains users with activity over multiple days, where they all start with class 0 and can become class 1 after a certain activity (which is not ...
thatguyoverthere's user avatar
4 votes
2 answers
11k views

Combining Latitude/Longitude position into single feature

I have been playing with two dimensional machine learning using pandas (trying to do something like this), and I would like to combine Lat/Long into a single numerical feature -- ideally in a linear ...
mainstringargs's user avatar
2 votes
2 answers
130 views

Creating a metric based on some features

I want to create a new metric based on some features but dont know how to start. I basically want to create a "job satisfaction level" metric based on some features. The features could be work hours, ...
Fatima's user avatar
  • 71
1 vote
2 answers
2k views

What are features for state-action pairs in RL?

I read this answer: What are features in the context of reinforcement learning? But it only describes features for the state only in the context of cartpole, ie. Cart Position, Cart Velocity, Pole ...
mLstudent33's user avatar
1 vote
1 answer
1k views

How to put multiple features into RNN input vector

I am trying to code a recurrent neural network (LSTM) to create music in python and was considering using multiple features instead of just the note pitch as an input into the network. Initially I had ...
treutm's user avatar
  • 37
3 votes
3 answers
2k views

how to evaluate feature quality for decision tree model

Most of the tutorials assume that the features are known before generating the model and give no way to select 'good' feature and to discard 'bad' ones. The naive method is to test the model with new ...
Bertrand's user avatar
  • 197
2 votes
1 answer
41 views

I want to create an additional feature(column) based on some manipulation of values from existing features

Consider my data-frame to be like this ('x','y','z' are features): I want to create a python function which will take an expression as a string (something like this: 'x+y-2z') and create a new ...
Bumble_BEE's user avatar
3 votes
1 answer
327 views

Label Encode with pre defined classes [closed]

I have trained a model (Random Forest) and now I would like to use it to predict certain data on a particular day. I have a categorical column where there are some values (say a,b,c,d,e) over a period....
ravishankar's user avatar
3 votes
1 answer
91 views

Using historical label as a feature in my ML model?

I am working on a predictive model to predict change in the price of an asset (up, down, no change). The labeling is based on the derivative of the price and is exponentially smoothed with an alpha of ...
Dick Thompson's user avatar
1 vote
1 answer
743 views

How to deal with Optional Input

I'm from the vision world and only worked with pixels from 0-255, ignoring any side effects. My current problem is different, in the way that I cannot rely on the input data. What my problem is: I ...
J. Peters's user avatar