Skip to main content
OverflowAI is here! AI power for your Stack Overflow for Teams knowledge community. Learn more

Questions tagged [categorical-data]

Categorical data can take on a limited (usually fixed) number of possible values called categories. Categorical values "label", they do not "measure". Nominal and dichotomous/binary scale types are categorical. Some people consider ordinal scale categorical too.

Filter by
Sorted by
Tagged with
1 vote
1 answer
26 views

Chi-square test results interpretation

I am comparing with Chi Square the distributions of two categorical variables. Both have the same number of classes. After counting each class per variable, I obtain very similar counts but the p-...
crbl's user avatar
  • 111
3 votes
1 answer
802 views

How does a Neural Net handle an unseen class for a Categorical Feature?

Let's say I train a Neural Net, and I have a Categorical Feature X. During training, there are only 3 classes seen in feature X; A, B, C. Now, let's say I want to make predictions from this trained ...
the man's user avatar
  • 139
0 votes
0 answers
16 views

Model Architecture for Time-Series Forecasting with Categorical and Multivariate Data

Context: I was looking at using an LSTM model to forecast the amount of gold gained for each of 10 heroes in a game of Dota 2, a MOBA game, as a base model in some type of model architecture. The game ...
DCRA's user avatar
  • 1
0 votes
1 answer
24 views

Dealing with only categorical features dataset

I'm trying to do multi-class classification on a labeled dataset with purely categorical features. There are around 30 features in total. 3 of the features in particular have around 100 unique values (...
Shaurya Uniyal's user avatar
0 votes
0 answers
3 views

Options for representing the following "conditional rules" in some data, and if categorical data science would be helpful?

I am new to data science, so I am hoping the following question is a reasonably elementary exercise for someone more experienced. Let us say I have $n$ categories of data. Each category is a ...
Julius Hamilton's user avatar
0 votes
0 answers
28 views

Visualise intersections of group membership (several low-cardinality variables)

I need to visualise joint and marginal frequencies of several low-cardinality categorical variables. Equivalently, I want to visualise sizes of groups and their intersections, where membership in some ...
paperskilltrees's user avatar
3 votes
0 answers
38 views

How to predict multi-variate time-series from different samples [closed]

I'm having issues seeing the best way to predict a time-series when training on a dataset with different samples. I have a dataset that shows the weight of 10 rabbits from their first day to their ...
scootjow's user avatar
1 vote
1 answer
33 views

How to deal with categorical disalignment in test and train in binary classification problems

I have a train and test datasets (600k observations) that have different categories for the same categorical variable. For example train has the categorical variable Letters having unique categories ...
kyara's user avatar
  • 13
0 votes
0 answers
15 views

Feature selection on datasets with both categorical and numerical features

I'm proposing a novel methodology for feature selection in the context of tabular datasets that contain both numerical and categorical features. In order to prove the efficacy of my methodology, I ...
Francesco De Santis's user avatar
4 votes
1 answer
823 views

Decision Tree only splits to the left

I can’t really understand, why my decision tree only splits to the left. I originally have 2 categorical features (further named feature 0 and 1), which I concat to one feature since feature 1 is ...
Taitex's user avatar
  • 41
0 votes
0 answers
12 views

classification using simple relationships between time series data

I am looking to predict which courses are taught by which university professors at my school. More specifically, for each semester and professor I want to know the probability breakdown of which ...
retep's user avatar
  • 101
0 votes
0 answers
55 views

Combining Textual, Categorical and Numerical data for Semantic Search using SentenceTransformers model

I'm building a food semantic search model and I want to use a pre-trained SentenceTransformers model with cosine similarity. I'm using Epicurious dataset for the corpus which consists of textual (&...
Alex's user avatar
  • 1
0 votes
0 answers
49 views

Training Biased/Uneven Categorical Data with CatBoost, Unbalanced/Unseen Categories Handling

Summary: I am training a discount eligibility model where the dataset represents historical data for products where people availed discounts based on simple features like product category, discount ...
glory9211's user avatar
  • 101
0 votes
2 answers
29 views

How to encode Income Type Ordinal Data into numbers?

I am doing a mini project on Credit card Approval Prediction. The Dataset I have taken is from Kaggle: https://www.kaggle.com/datasets/rikdifos/credit-card-approval-prediction Problem: I want to ...
Prajwal Dhage's user avatar
0 votes
0 answers
7 views

What are some standard methods for studying co-ocurrence patterns?

We have a series of academic articles annotated with several tags eg "environmental issues", "legal issues", etc. I wish to understand whether some of these topics are frequently ...
Jsevillamol's user avatar
0 votes
0 answers
28 views

Encode multiple label categorical variables with consideration of the frequency and standardization

I'm currently working on a dataset containing, among other features, multi-labeled categorical data per person for the last 3 years. I'm not sure how to handle this kind of categorical data where ...
codade's user avatar
  • 1
0 votes
0 answers
18 views

Unsupervised rule extraction of categorical data

I have a dataset of network traffic with three features that I would like to extract rules from in order to apply firewall/flow control rules i.e. the permitted flows. I am able to classify a ...
user153170's user avatar
1 vote
1 answer
382 views

Mixing categorical data and time-series data for regression purpose

I came across a problem and I have been looking on internet how to solve it without finding a solution that fits my need. I am trying to predict investors behavior. To be precise, I would like to ...
Hugoz13's user avatar
  • 13
0 votes
1 answer
77 views

Converting categorical to the percentage

How do I convert the categorical value to the percentage?| I have this asset wellness data: Poor: 3 Warning: 27 Good: 120 How do I convert it to the percentage ...
Muhammad Ikhwan Perwira's user avatar
0 votes
1 answer
83 views

When using Chi-Squrare test in feature selection makes sense?

What are the prerequisites that need to be fulfilled before conducting a chi-square test (Bivariate analysis)? For instance, before having a correlation matrix, we should first ensure linearity. What ...
user's user avatar
  • 1
0 votes
1 answer
63 views

Feature Selection - determining the significance of imbalanced categorical data column

I have a dataset with a categorical column that contains three categories. One of the categories represents 98% of the data, while the remaining 2% are distributed between the other two categories, ...
user151171's user avatar
0 votes
0 answers
111 views

Interpretation of best subset selection regression model for factor variables with more than 2 levels

I applied the best subset selection regression model in R from leaps package to my dat dataframe. ...
rez's user avatar
  • 63
0 votes
1 answer
29 views

Help me identify the type of plot and the relationship between the dependent variables

Question: I am not sure how to describe the sample graph attached. Can you please help me identify the type of plot and how to statistically measure the relationship between the dependent variable (Y-...
Leo82's user avatar
  • 1
0 votes
0 answers
31 views

Clustering with features of categorical type

I have a dataset of 17 features and class label for each datapoint. The description of dataset is as follows: 2 features contain values 0, 1, 2, 3 15 features contain values 0, 1, 2 class label ...
foobar's user avatar
  • 1
1 vote
2 answers
1k views

Can decision trees handle Nominal Categorical variables?

I have read that decision trees can handle categorical columns without encoding them. However, as decision trees make splits on the data, how does it handle Nominal Categorical variables? Surely a ...
Connor's user avatar
  • 651
0 votes
1 answer
26 views

Best practice for variables that only have answer if yes in previous column

I currently have a dataset that consists of survey data that has several columns that have answers dependent on the previous question. For example, I may have a question that says "Did you take ...
RLB's user avatar
  • 23
0 votes
1 answer
412 views

How to efficiently reduce dimensions of one-hot encoded categorical values?

I'm currently working on a project where I'm using an LSTM to learn and predict sequences of categorical data. My dataset consists of variable-length sequences of items $s_i = [x_{i_0}, x_{i_1}, ..., ...
acxcv's user avatar
  • 13
0 votes
1 answer
28 views

Data Leakage possibility related to replacing rare values on a categorical column

I am exploring the possibility of data leakage for categorical columns, replacing rare values with category 'Other' Let's say I have a DF with 40 categorical columns. I will check each of them, find ...
Ali Kılınç's user avatar
0 votes
1 answer
28 views

Categorical Dataset Machine Learning

I have a dataset that is completely binary and labeled. I would like to be able to use machine learning for one of the columns. I have read that unsupervised models, such as K-Means, do not work with ...
ml_beginner's user avatar
1 vote
1 answer
76 views

Customer Segmentation with mixed data [closed]

I want to perform clustering. I am reading about this topic but I am totally confused. My dataset has 490 observations and it consists of numerical data (3 columns: Recency, Frequency, Monetary), ...
ArgyGr's user avatar
  • 11
-1 votes
1 answer
146 views

Manual computation of the predictions in a convolutional neural network

I am trying to manually compute the predictions of the Keras library for a convolutional neural network. However, I am struggling a lot to match my final result with the ones provided by Keras. I do ...
mdslt's user avatar
  • 59
1 vote
1 answer
65 views

How to predict probability of an event when we have a month to month data?

I'm trying to find references about how to proceed to get the probability of an event happening when we have "temporal data" in our table My data is basically: hex_id: id of the object date:...
Leonardo Ferreira's user avatar
1 vote
1 answer
275 views

Encode categorical data for unsupervised learning

What is the best encoder for categorical data in unsupervised learning? I am using unsupervised learning on mixed data (such as K-means). Before running my unsupervised algorithm, I am using dimension ...
Julien PETOT's user avatar
0 votes
1 answer
22 views

Is there a procedure for determining if a classification problem is ill-defined?

Consider a group of objects denoted $O = \{o_0, o_1, \cdots\}$ where each object is associated with a feature vector $F = \{f_0, f_1, \cdots\, f_{N-1}\}$. For this case, assume the features are ...
Ralff's user avatar
  • 123
2 votes
1 answer
2k views

How to use categorical data in forecasting with Prophet?

I'm trying to create a model to predict the number of players on a video game at a certain time and was trying to figure out how to integrate categorical data into my forecasting problem. So far, my ...
CrazyRageMonkey's user avatar
0 votes
2 answers
1k views

Does Scikit-Learn's OneHotEncoder make all Columns Categorical?

I've been using Scikit-Learn's OneHotEncoder to turn categorical data into binary columns, however, it seems that fitting ...
Connor's user avatar
  • 651
1 vote
1 answer
74 views

N-ary decision tree with categorical features

I want to build an n-ary decision tree with categorical features. I am using ordinary ID3 algorithm to build a tree. Lets take the next dataset as a training dataset for building a decision tree: ...
dzi's user avatar
  • 111
0 votes
0 answers
35 views

What Would Be a Good Measure of Feature Importance in Regression?

Doing simple supervised regression where the label is a floating point number (guaranteed positive) and the features are a mix of continuous floating point values and some categorical features. What ...
Della's user avatar
  • 335
0 votes
0 answers
30 views

Dealing with categorical columns with unbalanced value count

I'm doing some data processing and wondering what is the best practice for dealing with categorical columns that has a value counts plot looking like the below (these are one-hot-encoded at a later ...
fendrbud's user avatar
0 votes
1 answer
28 views

How do I use one hot encoding with 240 nominal variables and each with equal occurrence?

The method I saw that's generally used to deal with large # of nominal variables is to keep the most frequent variables and introduce a new "other" category. But that's not possible with my ...
learner's user avatar
1 vote
1 answer
62 views

How do well informed labels for ordinal encoding improve model performance?

From Kaggle's intermediate machine learning tutorial, it was stated that for each column, we randomly assign each unique value to a different integer. This is a common approach that is simpler than ...
Terrarium's user avatar
  • 113
1 vote
1 answer
630 views

How to predict on data that is label encoded as end user will input a categorical data?

My dataset contains about 29 features with 3 class labels as result. Among these 29 features around 24 features are categorical i cannot transform each category into numbers as there are many more ...
Muhammad Minhas's user avatar
2 votes
1 answer
432 views

Encoding for Linear Regression

I have a CSV file with salary information and other columns. I am trying to transform some of these columns into proper values, for a ...
Alix Blaine's user avatar
1 vote
1 answer
60 views

working principle of Support Vector Machine

I have a dataset consisting of numerical features and categorical features. I want train the training set using SVM. SVM is a quadratic optimization algorithm. I would like to know the how SVM works ...
Encipher's user avatar
  • 361
0 votes
0 answers
40 views

How to cluster data where each sample has a different amount of different features?

I decompiled one million binaries and stored a representation of every function in a database. Each binary has a list of functions. Each function has a number of callees (how often its called by ...
SaulGoodman's user avatar
0 votes
1 answer
64 views

SkLearn Categorical Naive Bayes Vs Mathematical theory of Naive Bayes

The Naive Bayes classification based on the following formula $P(C_i|X) = {P(X|C_i)P(C_i) \over P(X)} ... i)$ $P(X|C_i)$ is the posterior probability of $X$ conditioned on $C_i$, $P(X)$ prior ...
Encipher's user avatar
  • 361
1 vote
1 answer
144 views

If I use Weight of Evidence to transform categorical variables, do I still need to inform their indexes to Catboost

I'm using Weight of Evidence (WOE) to encode my categorical features. Do I still need to inform Catboost that they are categorical features by using cat_features parameter?
Jorge Amaral's user avatar
1 vote
0 answers
50 views

Intuition behind catboost encoding techniques

Can anyone please help me in understanding the effect of various bucketing techniques used in CatBoost Algorithm for categorical features? Like there is border, buckets, binarized target mean, counter ...
Mimansa Maheshwari's user avatar
0 votes
1 answer
62 views

Do you use categorical data types?

Personally I've never used the categorical data type in pandas, and leave eveything as objects. I've seen it has the capability to be saved as parquet files, saves data etc... What are the pros and ...
GooJ's user avatar
  • 435
0 votes
1 answer
143 views

Is it possible to implement logistic regression (or any other ML method) to impute null values in a categorical feature with multiple values?

I'm doing a Data Science project, and I'm on the stage of cleaning categorical features. I've been researching, and it seems that imputing the mean or median can change the distribution. Therefore, a ...
Álvaro V.'s user avatar

1
2 3 4 5
8