Questions tagged [dummy-variables]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
1 answer
11 views

Is there a way to forecast a time series multiple linear regression using externally made dummy variables?

This question concerns question 4h of this textbook exercise. It asks to make future predictions based on a chosen TSLM model which involves an endogenously (if i'm using this right) made dummy ...
user avatar
  • 21
0 votes
0 answers
11 views

Dummy Predictors / Continuous Dependent Variable

I have a dataset with 50+ dummy coded variables that represent the purchases of an individual customer. Columns represent the products and the cell values 0 or 1, whether the product has been ...
user avatar
0 votes
1 answer
9 views

Should I create single feature for each specific word which i find in text or one for all them?

I am doing feature engineering right now for my classification task. In my dataframe I have a column with text messages. I decided to create a binary feature which depends on whether or not in this ...
user avatar
  • 113
0 votes
0 answers
25 views

what are the effect on machine learning regression model if the dataset has two exact same columns

What will be the effect on the Machine learning model if the dataset has two exact same columns(exact 1 correlation). One thing that comes to my mind is that if two columns are exactly the same then ...
user avatar
1 vote
1 answer
69 views

How to get dummy variables from "first name"

I intend to predict the age of customers using some features. There are some categorical features that I need to convert to dummy variables before the modelling stage. Since the datasets are so big (...
user avatar
1 vote
2 answers
971 views

inconsistency between y and x numbers in the Split into train and test sets

I am new to the field to the data science, and need help in the following: I am working on a data set that consists of both categorical and numerical values, first I have concatenate the two files (...
user avatar
1 vote
2 answers
57 views

what would be the correct representation of categorical variables like sex?

I have a doubt about what will be the right way to use or represent categorical variables with only two values like "sex". I have checked it up from different sources, but I was not able to ...
user avatar
  • 211
1 vote
1 answer
30 views

Use dummy variables to create a rank variable. R

I have a series of multiple response (dummy) variables describing causes for a canceled visits. A visit can have multiple reasons for the cancelation. My goal is to create a single mutually exclusive ...
user avatar
0 votes
1 answer
20 views

Dummies Variables and Scaling in Regression Problems

I was wondering if having dummies variable and scaling other variables could joke my model. In particular, I have implemented a Random Forest Regressor by using scikit-learn, but my data model is ...
user avatar
1 vote
1 answer
163 views

What exactly is a dummy trap? Is dropping one dummy feature really a good practice?

So I'm going through a Machine Learning course, and this course explains that to avoid the dummy trap, a common practice is to drop one column. It also explains that since the info on the dropped ...
user avatar
1 vote
1 answer
311 views

Should I include all dummy variables or N-1 dummy variables (keep one as reference) in neural networks

I have a categorical variable with N factor levels (e.g. gender has two levels) in classification problem. I have converted it into dummy variables (male and female). I have to use neural network (...
user avatar
  • 125
0 votes
1 answer
37 views

how do tree based methods deal with missing feature columns?

all, i have trained a model using xgboost. Some of the features are one hot encoded e.g. currency where it is either gbp or usd. it seems that when i output the feature importance gbp and usd were in ...
user avatar
  • 466
1 vote
2 answers
62 views

Problem with converting string to dummy variables

I'm new in data science, I have data which want to work on it, I omitted extra columns and convert it to 4 columns ( Product, Date, Market, Demand ) . in this data Product and Market are string, I ...
user avatar
  • 13
1 vote
1 answer
30 views

How to handle fixed values for variables in pre-processing

I have a dataset which contains few variables whose values do not change. Some of the variables are non-numeric (for example all values for that variable contain the value 5) and few variables are ...
user avatar
  • 521
0 votes
2 answers
403 views

Pandas get_dummies() rows dropping after joining back with X

I'm having an issue that I can't explain and am hoping I am missing something simple. I have a large dataset of shape(45Million+, 51) and am loading it in for some analyses (classifiers, deep learning,...
user avatar
  • 101
1 vote
1 answer
493 views

onehotencoder random forest

In a Random Forest context, do I need to setup dummies/OnehotEncoder in a dataset where features/varibles are numerical but refer to some kind of category? Let's say I have the following variables: ...
user avatar
  • 113
0 votes
1 answer
223 views

Creating dummy variables [closed]

I have a Eurobarometer dataset and want to eventually create a logistic regression model and a linear probability model using a set of dates as the dependent variable. However, the dates in the ...
user avatar
2 votes
1 answer
99 views

Dummy variable only for character value in a column (Neglecting float and integers)

My dataset consists of 3000 rows and 50 columns, out of which one column (ESTIMATE_FAMILY_CONTRIBUTION) contains all numerical value(around 2000 different values like 20,30,32....) but got one value ...
user avatar
1 vote
1 answer
289 views

Random Forest in R with only character variables

I am new to using random forest in R an my goal is to identify the independent variables which have the highest impact on the dependent variables. I am looking at sales data, and sales is my dependent ...
user avatar
  • 53
1 vote
1 answer
52 views

Handling Numerical Categorical Column in ML models in Python

When I was exploring the titanic dataset to estimate the probability of a person of surviving using the Logistic Model, I realized there are two ways of handling numerical categorical variables : Use ...
user avatar
2 votes
1 answer
1k views

How to interpret dummy variable in ML prediction?

I am working on a binary classification problem where I have a mix of continuous and categorical variables. Categorical variables were created by me using ...
user avatar
  • 2,129
1 vote
2 answers
263 views

Dummy encoding the categorical variables using the changed version of OneHotEncoder [duplicate]

This is my code, I was trying to dummy encode the first column of X using OneHotEncoder but it was showing error and the documentation page of OneHotEncoder says that it has been changed and I wasn't ...
user avatar
1 vote
1 answer
142 views

Create new variable in python for each iteration

I have two different dataframes like below. And I need to make a dataframe like the following one. Can you please help on how to that in python? Thanks
user avatar
2 votes
3 answers
1k views

How to handle "year" variable for Machine Learning models

I have a "year" variable but I don't know which is the best way to handle it for a ML model, as it is a numerical variable, giving some sequence. Should I treat it as a categorical variable? Thanks ...
user avatar
  • 21
0 votes
1 answer
45 views

Frequency of occurrence - dummy variables

I am thinking about it not the first time, namely if I have a variable that I want to convert later to the variable dummy (cities in this case), should I delete lines that occur less often than N ...
user avatar
  • 127
0 votes
2 answers
528 views

How to Keep Missing Values in Ordinal Logistic Regression

I’m using mord package in python to do ordinal logit regression (predict response to movie rating 1-5 stars). One of my predictor variables is also ordinal but ...
user avatar
  • 171
1 vote
0 answers
54 views

Same coefficient in multivariate regression with dummy variables

Hello Data Science community, I have a model with 1 quantitative variable (y) and 2 categorical variables. In order to work with the categorical variables I have created dummy variables (binary) for ...
user avatar
1 vote
1 answer
22 views

Handling hierarchical category independent variables

I have data with huge categorical attributes. For example, main_column, sub_column1, sub_column2 are 3 hierarchical attributes. If if take dummy variable on these columns the column count is ...
user avatar
  • 257
2 votes
2 answers
5k views

Prediction after one hot encoding

I have a regression model that I want to make prediction based on values that I will get from an end user. In my dataset, I have one categorical variable region ...
user avatar
  • 33
4 votes
3 answers
5k views

Obtaining consistent one-hot encoding of train / production data

I'm building an app that will require user input. Currently, on the training set, I run the following code, in which data is a pandas dataframe with a combination ...
user avatar
0 votes
1 answer
480 views

convert keywords in one column into several dummy columns

My dataframe has some free text fields named: {'title', 'description', 'location'} I prepared this text column by: concatenating all into a new column, dropping ...
user avatar
6 votes
3 answers
1k views

How to give a higher importance to certain features in a (k-means) clustering model?

I am clustering data with numeric and categorical variables. To process the categorical variables for the cluster model, I create dummy variables. However, I feel like this results in a higher ...
user avatar
  • 71
1 vote
1 answer
236 views

How Dummy Variables Should Be Modeled In A Linear Regression Model?

I've a cross sectional model where I want predict number of users that take specific service, to make it I've many variables but have specifically two nominal: isWorkday(0 or 1) and weeday(1,2,3,...,7)...
user avatar
1 vote
1 answer
152 views

caret dummyVars on unseen data

I created my dummy variables, trained my model and tested it as below: ...
user avatar
  • 521
2 votes
1 answer
142 views

How to deal with a potencially multiple categorical variable

I'm build a model that has, as inputs, some categorical variables. I had already dealt with this sort of data before, and applied different techniques as creation of dummy variables and factor scoring....
user avatar
1 vote
1 answer
578 views

Can you apply PCA to part of your dataset?

I am working with kaggle dataset that has over 130 features composed of 116 categorical and 14 continuous features. I plotted the heatmap for the 14 continuous variables and found that most of them ...
user avatar
6 votes
2 answers
6k views

In which cases shouldn't we drop the first level of categorical variables?

Beginner in machine learning, I'm looking into the one-hot encoding concept. Unlike in statistics when you always want to drop the first level to have k-1 dummies (...
user avatar
3 votes
1 answer
3k views

Using pandas get_dummies() on real world unseen data

I made a ML model, trained and tested it with my data containing categorical variables. To create dummy variables I used pd.get_dummies() before the split. I now ...
user avatar
  • 521
1 vote
1 answer
104 views

Dummy variables for unseen data in R

I got the following problem: When I trained my model I created my dummy variables(before train-test split) in the following way: ...
user avatar
  • 521
3 votes
3 answers
2k views

How to obtain original feature names after using one-hot encoding

This question is on an implementation aspect of scikit-learn's DecisionTreeClassifier(). How do I get the feature names ranked in descending order, from the ...
user avatar
  • 51