Questions tagged [categorical-data]

Categorical data can take on a limited (usually fixed) number of possible values called categories. Categorical values "label", they do not "measure". Nominal and dichotomous/binary scale types are categorical. Some people consider ordinal scale categorical too.

Filter by
Sorted by
Tagged with
1
vote
2answers
180 views

Scikit alternative for categorical data modeling?

So, sklearn doesn't support categorical data in its models. Is there a known alternative for categorical data modeling (such as random forests, etc.) for Python?
1
vote
3answers
151 views

Method for predicting price based on Geographical market, Product, and Company

I have a dataset which tracks the prices of 21 products, charged by 24 companies, in 150 different cities across the globe. However, the data set has missing values--that is, I might have Company X's ...
1
vote
2answers
33 views

Linear Regression with Category variables

I'm currently learning and exploring machine learning and understand the basics of linear regression based on two numerical variables, but now I wish to go a little further and need some guidance ...
1
vote
1answer
64 views

Calculating correlation for categorical variables

I am struggling to find out a suitable way to calculate correlation coefficient for categorical variables. Pearson's coefficient is not supported for categorical ...
1
vote
1answer
164 views

splitting mechanism with one hot encoded variables (tree based/boosting)

I am using xgboost and have a categorical unordered feature with 25 levels. So when i apply one hot encoding i have 25 columns. This introduces alot of sparsity. Even more unusual, my feature ...
1
vote
2answers
219 views

Appropriate loss function for multi-hot output vectors

I have some data in which model inputs and outputs (which are the same size) belong to multiple classes concurrently. A single input or output is a vector of zeros somewhere between one and four ...
1
vote
1answer
257 views

sklearn serialize label encoder for multiple categorical columns

I have a model with several categorical features that need to be converted to numeric format. I am using a combination of LabelEncoder and OneHotEncoder to achieve this. Once in production, I need to ...
1
vote
1answer
41 views

Can we optimize regression problems that have categorical variables by encoding them if on the other hand we are inserting multicollinearity? [duplicate]

Can we optimize regression problems that have categorical variables by encoding them if, on the other hand, we are inserting multicollinearity?
1
vote
1answer
62 views

Modeling the Price Movement- What analysis should be used

I am trying to model the price of a hotel as the check-in date arrives. I have a data set which looks like- For e.g- if I am looking at the booking date of Dec 31st, I would want to analyze the ...
1
vote
1answer
466 views

How to handle large number of categories in a dataset?

I have one dataset of "Books" which contains 8 columns initially and out of which 3 of them contains text values which can be categorized. The 3 columns contains "Language-code", "Author Name" and "...
1
vote
1answer
364 views

Loss is bad, but accuracy increases?

I have a multicategorial classification problem for images. There are 5 (imbalanced) classes for which i use different class weights. In general there are only a few training images per class: ~56-238 ...
1
vote
2answers
206 views

Is there a quick way to check for multicollinearity between categorical variables in R?

I have a large amount of categorical and dummy variables (36) and I would like to remove a number of them based on their multicollinearity (or just collinearity). Instead of using Chi Square tests ...
1
vote
1answer
907 views

What does “Find pattern” mean in data science? [closed]

I am doing certification and i have a project to complete. In project, they have said "Find Patterns". What does it mean? what steps should i carry out?
1
vote
2answers
704 views

Handling features with multiple values for clustering

Suppose I have a movies dataframe in pandas. One of the features is Genre. It has a list of genre names. for example: ...
1
vote
1answer
29 views

Handling nominal category features in decision tree

I have been reading some stackoverflow questions on how to handle nominal features for decision tree (sklearn implementation). One of the answer states that : Using a OneHotEncoder is the only current ...
1
vote
1answer
47 views

one hot encoding target variable in tree and non tree (knn) methods

I am learning about label encoders, one hot encoding etc applied to datasets for classification via KNN and XGBoost type trees. However, I am a bit confused as to whether the target variable should be ...
1
vote
2answers
31 views

Different number of features in train vs test when using Label Encoding

This is not a duplicate of Different number of features in train vs test There are some categorical columns in my data, and the cardinality for each of them is large, so I chose to use LabelEncoding ...
1
vote
1answer
60 views

Anomaly detection on sparse categorical data

I have a big dataset with a column "clientid" and a categorical column "choice". I want to find out what are the clients that have strange combinations of choices (less frequent ...
1
vote
1answer
41 views

How to handle One Hot Encoded columns with changing categories in supervised ML Problem?

Scenario: I have the following game data about participants, game and the winner in the following format: ...
1
vote
1answer
238 views

unsupervised anomaly detection on sparse data

Given that I have a very sparse data matrix with continuous features, like this dataframe for example ...
1
vote
2answers
53 views

Is there a RandomForest implementation that handles categorical data without encoding in python?

I am working on a binary classification project with both continuous and categorical features. I know that the R implementation of RandomForest can handle categorical data passed in as factor type ...
1
vote
1answer
82 views

Does converting continuous variable to discrete(categorical) variable increases accuracy of a tree based model?

I've read other questions regarding if a continuous feature should be converted to categorical or not. But I'm interested in case of tree based classifiers such as Decision Tree, Random Forest, ...
1
vote
1answer
96 views

WHY or WHEN to convert numeric data to a categorical data?

This is an open ended WHY TO or WHEN TO question rather than a question on HOW TO encode numeric to categorical data. I am currently working on Telco Customer Churn dataset from kaggle. This is ...
1
vote
1answer
39 views

Handling Numerical Categorical Column in ML models in Python

When I was exploring the titanic dataset to estimate the probability of a person of surviving using the Logistic Model, I realized there are two ways of handling numerical categorical variables : Use ...
1
vote
2answers
2k views

what should i do if my target variable is categorical when using decision tree? (many categorical variables)

all, i'm trying to classify a set of features to belong to a particular company (my dependent variable). my independent variables are a mixture of continuous and categorical features. my data-set ...
1
vote
3answers
331 views

Categorical variables with multiple entries transformed to entity embedding

I have structured data with lots (tens of thousads) of categories organized into columns. The goal is to enter the data into gradient boosting machine algorithm for a specific prediction. Some ...
1
vote
1answer
21 views

Turning Histogram values into Numerical format ( Excel-xslx, Pandas-DataFrame, etc.)

I am trying to do a correlation study about personality traits as described in Hofstede's :https://www.hofstede-insights.com/product/compare-countries/ . I would like to have the values described in ...
1
vote
1answer
115 views

Linear regression model with (categorical) predictor variables

I used LM model with (categorical) predictor variables on my data in r like this (I have count variable as dependent/target variable): ...
1
vote
1answer
791 views

Regression - Unbalanced Categorical Features

I have a data set that has some unbalanced categorical features. I would like to build a regression model to predict a label using machine learning (ML). How do I handle data imbalances in ...
1
vote
1answer
178 views

Algorithm for purely categorical data

Looking for an algorithm to deal with purely categorical data. It was suggested to me to look into the K-medoids algorithm. Anyone know if there is a K-medoids algorithm R library(package)?
1
vote
1answer
1k views

Prediction with unseen values in categorical variables

I have created an Artificial Neural Network with 4 features. I am at the point where I want to test the model with a live sample of a malicious file path/exe using: ...
1
vote
1answer
274 views

Dropping less frequently used categorical data?

I'm new to the datascience field and working on an assignment. I have a dataset with 150K rows with a categorical and numerical data, the target is a boolean. A categorical column consist of quite ...
1
vote
1answer
643 views

What is the input space of a neural network (or other supervised learning algorithms)?

While training the neural network (or any other supervised learning algorithms), we supply input variables and corresponding outputs. The input variables can be continuous or discrete (binary in many ...
1
vote
1answer
138 views

Logistic regression if 3 categories in outcome variable

Logistic regression is generally performed if there are 2 categories in outcome variables. I just tried it for iris dataset with species as y variable which has 3 categories. I used following code: <...
1
vote
1answer
2k views

customer segmentation with categorical variables

I was adviced to write in this group regarding my question about modeling categorical database. I have a customer dataset, which is a survey result. I have 1595 obs. and about 200 columns(200 because ...
1
vote
1answer
131 views

How can I make a prediction in a regression model if a category has not been observed already?

I'm researching a regression model to predict a target value that has four features, all of which are categorical. The categories are not fixed, e.g. one is a customer identifier. How could my model ...
1
vote
1answer
133 views

Applying machine learning algorithms to subset of attributes in dataframe

I have this huge mixed data set consisting of both numerical and categorical attributes which upon OneHotEncoding results into a data set with very high dimensionality. Is it wise to apply machine ...
1
vote
2answers
4k views

Faced problem while applying OneHotEncoder

For classification, I was trying to convert categorical data into numeric by applying OneHotEncoder. But it shows error could not convert string to float Here is ...
1
vote
2answers
3k views

Categorical, nominal or continuous variable?

I'm trying to understand what level of measurement is best for describing the 'number of rooms in a flat' feature. First of all, I think it's not a continuous feature, because rational values like 1....
1
vote
1answer
381 views

Best MIMO prediction algorithm for categorical variables

I have researched machine learning for quite a while and would like to test out my knowledge. So I am trying to use it for lottery number prediction. The goal is not to have 100% correct prediction (...
1
vote
1answer
180 views

Python: how to handle categorial values in dataset to build models

I have a training dataframe dfTrain and the output of dfTrain.head() is shown below: ...
1
vote
1answer
612 views

predict the probability of buying a product

I have a huge dataset with 3 variables Company_ID, Area_code, Product_ID each one of them is a categorical variable of levels ...
1
vote
1answer
4k views

Scikit Learn OneHotEncoded Features causing error in classifier

I’m trying to prepare data for input to a Decision Tree and Multinomial Naïve Bayes Classifier. This is what my data looks like (pandas dataframe): ...
1
vote
1answer
750 views

How to visualize (make plot) of regression output against categorical input variable? [closed]

I am doing linear regression with multiple variables. In my data I have n = 143 features and m = 13000 training examples. Some of my features are continuous (ordinal) variables (area, year, number of ...
1
vote
3answers
83 views

logistic regression or density estimation for binary dependent variable and binary (or categorical) features

I have a binary dependent variable $t$ and categorical features. We can even simplify to binary features since I can one-hot encode the categorical variables. In practice the one-hot encoding induces ...
1
vote
0answers
23 views

When doing feature selection, are features like “year”, “month” considered as ordinal features or should I convert them to strings?

I am working on a hotel reservation dataset that has both categorical and numerical (continuous and discrete) features (26 columns, 30244 rows). Target is also categorical and it says if the user &...
1
vote
0answers
10 views

spark ml StringIndexer vs OneHotEncoder, when to use which?

Confused as to when to use StringIndexer vs StringIndexer+OneHotEncoder. The OneHotEncoder docs say For string type input data, it is common to encode categorical features using StringIndexer first. ...
1
vote
0answers
24 views

An Unsupervised learning method suitable for large categorical data sets

I want to detect anomalies in the bank data set in an unsupervised learning method. However, in the bank data set, all columns except time and amount were categorical data, and about half of them had ...
1
vote
2answers
23 views

How do I assign specific values to categorical variables

I have a Pandas data frame with columns within a survey with the following categorical values - "Increased, Decreased, Neutral". My question is how can I assign specific numerical values to ...
1
vote
1answer
32 views

How to build multiple variable regression having a mix of numerical & categorical features?

There is a need to estimate Annual Average Daily Traffic Volume (AADT). We have bunch of data about vehicles' speeds during several years. It is noticed that AADT depends on the average number of such ...

1 2 3
4
5
7