Questions tagged [categorical-data]

Categorical data can take on a limited (usually fixed) number of possible values called categories. Categorical values "label", they do not "measure". Nominal and dichotomous/binary scale types are categorical. Some people consider ordinal scale categorical too.

Filter by
Sorted by
Tagged with
0
votes
1answer
18 views

How to Present All Categories in All Samples

I have a data contains many categorical columns. When I sampled this data randomly a few times and applied one-hot encoding to categorical columns I noticed that it ended up with datasets with ...
1
vote
1answer
35 views

How to handle One Hot Encoded columns with changing categories in supervised ML Problem?

Scenario: I have the following game data about participants, game and the winner in the following format: ...
0
votes
1answer
24 views

How would I approach training a model and encoding this categorical data

So I have the following data: I have one series where each word has a value that describes the average review score that would get. For example, if the word "excellent" showed up in reviews ...
1
vote
1answer
17 views

How to deal with variable number of permutation invariant features?

I want to learn from data where each record has a variable number of features that have no inherent order to them. Take as an example the task to predict whether a repair is worth it of some item. ...
1
vote
0answers
24 views

DBSCAN on textual and numerical columns

I have a dataset which has two columns: title price sentence1 12 sentence2 13 I have used doc2vec to convert the ...
0
votes
1answer
22 views

unsupervised anomaly detection on sparse data

Given that I have a very sparse data matrix with continuous features, like this dataframe for example ...
1
vote
0answers
24 views

How does the R implementation of RandomForest split nodes on categorical data?

The R implementation of RandomForest can take in categorical features as factors and train and predict on these features without encoding. Normally, I use the python implementation from scikit-learn ...
0
votes
2answers
21 views

Is there a RandomForest implementation that handles categorical data without encoding in python?

I am working on a binary classification project with both continuous and categorical features. I know that the R implementation of RandomForest can handle categorical data passed in as factor type ...
2
votes
1answer
17 views

How to Set the Same Categorical Codes to Train and Test data? Python-Pandas

NOTE: If someone else it's wondering about this topic, I understand you're getting deeper in the Data Analysis world, so I did this question before to learn that: You encode categorical values as ...
0
votes
1answer
16 views

How to encode high cardinality categorical data?

I have a dataset of 1600 rows and 28 columns. Only one column is partially complete with 1300 records. The rest is NaN. I did a value count of this columns and it has 84 different categories that are ...
0
votes
0answers
13 views

How can the forecast accuracy of three models for a categorical time series be compared?

This is a general question. Can anybody please point me in the direction of how I can compare the forecasts of a 3 level categorical time series by three competing models? I would like to compare the ...
1
vote
0answers
17 views

Endogenous and exogenous ordinal variables in R studio Lavaan

I am new to using R studio, so apologies for the basic question. I have run a number of Confirmatory Factor Analyses using the Lavaan package. Each questionnaire item is on a 4 point Likert scale 0 = ...
0
votes
0answers
16 views

Collapse categorical variable to reduce levels using a decision tree

I am using zip codes as an independent variable as part of a binary classification problem. Naturally, this feature has many different levels (around 2,000), so I was wondering if there is a ...
2
votes
1answer
33 views

Dealing with categorical variables in regression problems which method to use?

Usually if I have regression problem and my initial dataset contains categorical variables like : column 1: Math Science Science English I would convert this ...
0
votes
2answers
30 views

How would I classify this variable?

I am learning about the difference between categorical, ordinal and numerical variables. From what I understand: Categorical variables have 2+ categories without any intrinsic order. Ordinal ...
0
votes
1answer
23 views

how to handle different categorical values in train and test dataset?

I have a dataset in which if i do train_df["era"].value_counts() then it will show 120 different type of categorical values and then if i do ...
1
vote
1answer
59 views

splitting mechanism with one hot encoded variables (tree based/boosting)

I am using xgboost and have a categorical unordered feature with 25 levels. So when i apply one hot encoding i have 25 columns. This introduces alot of sparsity. Even more unusual, my feature ...
0
votes
1answer
19 views

Determining which categorical data is beneficial in predictive modelling

I am working on a model which will allow me to predict how long it will take for a "job" to be completed, based on historical data. Each job has a handful of categorical characteristics (all ...
0
votes
1answer
20 views

Model for predicting duration based on categorical data

I am working on a model which will allow me to predict how long it will take for a "job" to be completed, based on historical data. Each job has a handful of categorical characteristics (all ...
1
vote
0answers
96 views

Partial Dependence Plot and categorical variables

While reading about machine learning explainability and Partial Dependence Plot (PDP) in this book, the following appeared when dealing with categorical variables: For each of the categories, we get ...
3
votes
1answer
18 views

Is there an encoder which can automatically detect the intrinsic order of an ordinal variable and assign values accordingly?

Given data with an ordinal variable, says "house quality" with values ex (excellent), gd (good), ...
1
vote
2answers
55 views

How to find correlation between categorical data and continuous data

I'm working on imputing null values in the Titanic dataset. The 'Embarked' column has some. I do NOT want to just set them all to the most common value, ...
1
vote
2answers
146 views

Why is count encoding effective in improving accuracy? [duplicate]

Can someone please explain why/how Count encoding of categorical features improve accuracy in classification when compared to simply label encoding them ? I found one explanation in kaggle " ...
1
vote
2answers
76 views

What are some good methods to forecast future revenue on categorical and value based data?

I have monthly snapshots (3 years) of all the contract data. It includes following information: Contract status [Categorical]: Proposed, tracked, submitted, won, lost, etc Contract stages [...
0
votes
1answer
62 views

Is this attribute numeric or categorical (ordinal)? Help!

So I have this dataset I need to perform several techniques on as part of a data mining/machine learning project of some sort in PYTHON. There are a couple of features however, that have me very ...
2
votes
1answer
29 views

is there an adequate number of levels of categorical variables?

I have a project that I'm working on. The dataset contains many categorical variables and some of them have too many levels (+100). My question is : is there any advice to know the "adequate"...
2
votes
0answers
42 views

Regression dataset with categorical features

I have thought of a regression technique that I want to try on several datasets. I would like these datasets to have the following properties: Be a tabular dataset (no images). Have at least 20k rows,...
2
votes
1answer
99 views

Clustering mixed data types - numeric, categorical, arrays, and text

I have a dataset with 4 types of data columns: ...
1
vote
2answers
62 views

Appropriate loss function for multi-hot output vectors

I have some data in which model inputs and outputs (which are the same size) belong to multiple classes concurrently. A single input or output is a vector of zeros somewhere between one and four ...
0
votes
0answers
15 views

How to decide whether to use categorical embeddings in a neural network?

I have a binary classification task with a whole slew of binary categorical features, one multiclass categorical and a few continuous features. I initially treated the categorical features using one-...
1
vote
1answer
35 views

Does converting continuous variable to discrete(categorical) variable increases accuracy of a tree based model?

I've read other questions regarding if a continuous feature should be converted to categorical or not. But I'm interested in case of tree based classifiers such as Decision Tree, Random Forest, ...
0
votes
2answers
22 views

Categorical and non-categorical data in the same column

I have a unique dataset that has many columns and most columns contain both categorical and non-categorical data. For example, let's say that one column is attribute_1 and for observations that have ...
1
vote
1answer
29 views

WHY or WHEN to convert numeric data to a categorical data?

This is an open ended WHY TO or WHEN TO question rather than a question on HOW TO encode numeric to categorical data. I am currently working on Telco Customer Churn dataset from kaggle. This is ...
0
votes
0answers
13 views

Model performance metrics

I have a dataset with multiple numeric input values and a categorical output. How can I measure model performance with different algorithms. As the results are categorical, we can not obtain r squared ...
0
votes
0answers
16 views

Correlation between features in python

I have a dataset which has categorical variables as features. They are nominal in nature. One of the variable has 312 categories. I want to check how correlated the variables are, to check ...
0
votes
0answers
16 views

Sparse data and Neural Networks

I am trying to learn a model to predict the binary outcome of a computer game. The input data consists of the character picks by each of the ten players (two teams of 5, 150 possibilities each, with ...
0
votes
0answers
21 views

Autoencoder to encode features/categories of data

My question is regarding the use of autoencoders (in PyTorch). I have a tabular dataset with a categorical feature that has 10 different categories. Names of these categories are quite different - ...
0
votes
1answer
20 views

How to pre-process the name String of a customer?

I implement logistic regression to predict if a customer is a business or a non-business customer with the help of TensorFlow in Python. I have several feature candidates like name, street, zip, ...
0
votes
1answer
101 views

sklearn serialize label encoder for multiple categorical columns

I have a model with several categorical features that need to be converted to numeric format. I am using a combination of LabelEncoder and OneHotEncoder to achieve this. Once in production, I need to ...
0
votes
0answers
135 views

Silhouette Coefficient Implementation in KModes Clustering

I have been trying to calculate the Silhouette coeffecient for the clusters I have created using KModes clustering (since all of my data fields are categorical). I am using matching dissimilarity as ...
0
votes
0answers
137 views

Additional Explanation Required for KbinsDiscretizer

I am a newbie learning data pre-processing. I have few questions on encoding of categorical data. Q(1) Are ColumnTransformers compulsory to apply any of the various encoding methods on 2d data? Here'...
1
vote
2answers
42 views

Clustering on categorical attributes

I have a dataset with only 2 categorical attributes out of 9. How can I get a clustering analysis on it? I am using R. Do you have any advices about instructions, how to do it, topics, ...? here's my ...
0
votes
1answer
22 views

Data analysis PCA

I have a question about the functioning of PCA. I have a dataset with only 2 categorical attributes out of 9. Is it good to calculate pca between those two? Does it help me understanding anything ...
0
votes
0answers
14 views

PCA and clustering, regression tree with categorical attributes using R

I am trying to analyze a dataset which has 7 categorical attributes out of 9. Can you please help me? I don't know how to find right instructions to do it. I only learnt how to do it with numeric-only ...
2
votes
0answers
323 views

Dealing with categorical variables in Isolation Forest

Isolation Forest is widely used when dealing with outlier/anomaly detection when we have no labels. The theory behind is that making random split at random points and counting how many splits you do ...
0
votes
1answer
29 views

Categorical data - how to handle [closed]

Few questions on categorical data. Need suggestions / pointers: How can we check for correlation between categorical features and target or between the features themselves? How about correlation ...
2
votes
0answers
28 views

ML recommendation system with items organized in a tree

I would like to develop a recommendation system (probably hybrid, user-based and feature-based) for items which are organized in a tree (there are categories, divided in sub-categories, divided in sub-...
0
votes
0answers
120 views

Positional Encoding of Categorical Features in a Time Series Transformer

I am training a Transformer for Multivariate Time Series prediction. I am working with Categorical features and I am thinking of using Positional Encoding to encode them instead of Embedding. Has ...
0
votes
0answers
15 views

looking for a correlation function for categorical + numerical datasets in matlab

I am looking for a function in Matlab that calculates the correlation coefficient of a data set that includes categorical + numerical data. I am quite surprised this isn't as straightforward as I ...
1
vote
1answer
38 views

LSTM Time-series classification - derived feature

I have a time-series dataset and I want to derive a new feature based on a date column which I believe might improve my predictive model. The feature is if it's weekday or weekend. I am not sure how ...

1
2 3 4 5 6