Questions tagged [categorical-data]
Categorical data can take on a limited (usually fixed) number of possible values called categories. Categorical values "label", they do not "measure". Nominal and dichotomous/binary scale types are categorical. Some people consider ordinal scale categorical too.
391
questions
0
votes
1
answer
12
views
Best practice for variables that only have answer if yes in previous column
I currently have a dataset that consists of survey data that has several columns that have answers dependent on the previous question. For example, I may have a question that says "Did you take ...
- 23
0
votes
0
answers
12
views
What is the best way to whittle down rendundant categorical data?
I'm trying to build a linear regression model in Tensorflow (and preprocessing with pandas) that will help me categorize bank transactions. I'm trying to whittle down the vendor parameter, because the ...
0
votes
1
answer
28
views
How to efficiently reduce dimensions of one-hot encoded categorical values?
I'm currently working on a project where I'm using an LSTM to learn and predict sequences of categorical data.
My dataset consists of variable-length sequences of items $s_i = [x_{i_0}, x_{i_1}, ..., ...
- 13
0
votes
0
answers
18
views
Finding a relation between three variables
I am new to data mining and have learnt about association rules mining, classification analysis, cluster analysis and outlier analysis. So, to find relationship between three variables, regression ...
- 1
0
votes
1
answer
21
views
Data Leakage possibility related to replacing rare values on a categorical column
I am exploring the possibility of data leakage for categorical columns, replacing rare values with category 'Other'
Let's say I have a DF with 40 categorical columns. I will check each of them, find ...
- 29
0
votes
0
answers
14
views
0
votes
0
answers
13
views
How to make a predictive model using a timeseries data consisted of binary information?
I have a set of data that is showing the state of an object as a function of time. I would like to know what and how I should be utilizing machine learning to predict the state of the object at some ...
- 101
0
votes
0
answers
10
views
Encoding of categorical variables to reduce the effect of erroneous labels
I have a structured dataset containing (nominal) categorical variables encoded as labels, let's say a feature includes labels from 1 to 20. Some of the labels in that feature could just be errors, ...
- 11
0
votes
0
answers
24
views
How to do forecasting with categorical timeseries?
I have a dataset that is in the form of categorical timeseries: (specifically, we either know or don't know the values of 6 degrees of freedom of an object at any given time). If we know it, it's ...
- 101
0
votes
1
answer
23
views
Categorical Dataset Machine Learning
I have a dataset that is completely binary and labeled. I would like to be able to use machine learning for one of the columns. I have read that unsupervised models, such as K-Means, do not work with ...
1
vote
1
answer
27
views
Customer Segmentation with mixed data [closed]
I want to perform clustering. I am reading about this topic but I am totally confused. My dataset has 490 observations and it consists of numerical data (3 columns: Recency, Frequency, Monetary), ...
- 11
-1
votes
1
answer
83
views
Manual computation of the predictions in a convolutional neural network
I am trying to manually compute the predictions of the Keras library for a convolutional neural network. However, I am struggling a lot to match my final result with the ones provided by Keras. I do ...
- 59
1
vote
1
answer
17
views
How to predict probability of an event when we have a month to month data?
I'm trying to find references about how to proceed to get the probability of an event happening when we have "temporal data" in our table
My data is basically:
hex_id: id of the object
date:...
0
votes
0
answers
24
views
How to detect Novelty from different ranges of target variable?
I've a dataset of multiple categorical columns along with a target column that is continuous. Assume combination of categorical columns has a different range of values of target. Ex
Col1 - col2 - col3 ...
0
votes
0
answers
59
views
How to do DBSCAN clustering with mixed variables (numerical features and binary/ordinal variables)?
I have a question written at the end of the post which refers to the "Distances" paragraph. The other first two paragraphs give additional info.
Context
I'm working on a project where I have ...
1
vote
1
answer
40
views
Encode categorical data for unsupervised learning
What is the best encoder for categorical data in unsupervised learning?
I am using unsupervised learning on mixed data (such as K-means).
Before running my unsupervised algorithm, I am using dimension ...
- 11
0
votes
0
answers
21
views
Clustering mixed type variables with Orange
I wonder if with Orange it is possible to cluster mixed type data, so a dataset with numeric as well as discrete (categorical) data (ordered / unordered). Can you show an example of how that could be ...
0
votes
1
answer
18
views
Is there a procedure for determining if a classification problem is ill-defined?
Consider a group of objects denoted $O = \{o_0, o_1, \cdots\}$ where each object is associated with a feature vector $F = \{f_0, f_1, \cdots\, f_{N-1}\}$. For this case, assume the features are ...
- 113
2
votes
1
answer
238
views
How to use categorical data in forecasting with Prophet?
I'm trying to create a model to predict the number of players on a video game at a certain time and was trying to figure out how to integrate categorical data into my forecasting problem. So far, my ...
0
votes
1
answer
299
views
Does Scikit-Learn's OneHotEncoder make all Columns Categorical?
I've been using Scikit-Learn's OneHotEncoder to turn categorical data into binary columns, however, it seems that fitting ...
- 523
1
vote
1
answer
37
views
N-ary decision tree with categorical features
I want to build an n-ary decision tree with categorical features.
I am using ordinary ID3 algorithm to build a tree.
Lets take the next dataset as a training dataset for building a decision tree:
...
- 111
0
votes
0
answers
28
views
What Would Be a Good Measure of Feature Importance in Regression?
Doing simple supervised regression where the label is a floating point number (guaranteed positive) and the features are a mix of continuous floating point values and some categorical features.
What ...
- 305
0
votes
0
answers
12
views
Dealing with categorical columns with unbalanced value count
I'm doing some data processing and wondering what is the best practice for dealing with categorical columns that has a value counts plot looking like the below (these are one-hot-encoded at a later ...
- 23
0
votes
1
answer
19
views
How do I use one hot encoding with 240 nominal variables and each with equal occurrence?
The method I saw that's generally used to deal with large # of nominal variables is to keep the most frequent variables and introduce a new "other" category. But that's not possible with my ...
- 1
1
vote
1
answer
20
views
How do well informed labels for ordinal encoding improve model performance?
From Kaggle's intermediate machine learning tutorial, it was stated that
for each column, we randomly assign each unique value to a different integer. This is a common approach that is simpler than ...
- 13
0
votes
0
answers
12
views
How to classify and organize very complex data for easy future reference?
I've studied binomial nomenclature in college, but forgot all about it!
I use Joplin to organize my life and business, but it's getting out of hand complicated and I can't find stuff.
Struggling on ...
- 101
1
vote
1
answer
151
views
How to predict on data that is label encoded as end user will input a categorical data?
My dataset contains about 29 features with 3 class labels as result. Among these 29 features around 24 features are categorical i cannot transform each category into numbers as there are many more ...
1
vote
1
answer
73
views
Encoding for Linear Regression
I have a CSV file with salary information and other columns.
I am trying to transform some of these columns into proper values, for a ...
- 115
1
vote
1
answer
43
views
working principle of Support Vector Machine
I have a dataset consisting of numerical features and categorical features. I want train the training set using SVM. SVM is a quadratic optimization algorithm. I would like to know the how SVM works ...
- 359
0
votes
0
answers
24
views
How to cluster data where each sample has a different amount of different features?
I decompiled one million binaries and stored a representation of every function in a database.
Each binary has a list of functions.
Each function has a number of callees (how often its called by ...
0
votes
1
answer
43
views
SkLearn Categorical Naive Bayes Vs Mathematical theory of Naive Bayes
The Naive Bayes classification based on the following formula
$P(C_i|X) = {P(X|C_i)P(C_i) \over P(X)} ... i)$
$P(X|C_i)$ is the posterior probability of $X$ conditioned on $C_i$, $P(X)$ prior ...
- 359
1
vote
1
answer
27
views
If I use Weight of Evidence to transform categorical variables, do I still need to inform their indexes to Catboost
I'm using Weight of Evidence (WOE) to encode my categorical features. Do I still need to inform Catboost that they are categorical features by using cat_features parameter?
- 131
1
vote
0
answers
30
views
Intuition behind catboost encoding techniques
Can anyone please help me in understanding the effect of various bucketing techniques used in CatBoost Algorithm for categorical features? Like there is border, buckets, binarized target mean, counter ...
0
votes
0
answers
17
views
OHE vs dummy variables
Could someone explain to me the difference between creating categorical “embeddings” with StringIndexer and OneHotEncoder, vs just creating dummy variables for each category? Aside from it being more ...
- 1
0
votes
1
answer
39
views
Do you use categorical data types?
Personally I've never used the categorical data type in pandas, and leave eveything as objects.
I've seen it has the capability to be saved as parquet files, saves data etc...
What are the pros and ...
- 435
0
votes
1
answer
56
views
Is it possible to implement logistic regression (or any other ML method) to impute null values in a categorical feature with multiple values?
I'm doing a Data Science project, and I'm on the stage of cleaning categorical features. I've been researching, and it seems that imputing the mean or median can change the distribution. Therefore, a ...
- 25
0
votes
0
answers
12
views
Behavioral Segmentation Clustering Mixed Large Data - with or without categorical variables
This question may be closed due to being too broad but I feel as though this is the best place to ask my question.
At the moment, I am dealing with some customer data and am looking on segmenting them ...
0
votes
1
answer
24
views
Method of accuracy improve for binary classification imbalanced dataset
I have an imbalance data set where the imbalance ratio No: Yes is 8:1. If I run classifiers on the groundtruth dataset I got recall and F2 measure for Naive bayes, Logistic regression, random forest. ...
- 359
0
votes
0
answers
73
views
2
votes
1
answer
79
views
Clustering ordered categorical data
Suppose I have a list of, say, 100 countries, as well as their respective historical sovereign credit ratings as such
...
- 123
1
vote
0
answers
19
views
What methods I could use to analyze the contingency table?
I am data science beginner, and I have a question about methods that I could use to analyze the following data. It is a simple case, I am trying to check the influence of cohabitation before marriage ...
- 11
0
votes
0
answers
19
views
How to handle tags/lists with CatBoost?
I have database like this:
Id, type, category1, category2, tags
1, ‘cosmetics’, 123, 456, [446, 354]
2, ‘electronics’, 234, 213, [55, 978, 12]
…
And I want to predict some value with ...
- 1
0
votes
0
answers
30
views
Constructing pandas DF with model trained categorical variable
I've trained a lightGBM model on a dataset X where X has a categorical (in the pandas sense) variable. This model trains fine and when I predict using it all looks good - I can even change the value ...
1
vote
0
answers
303
views
Library for Phi correlation coefficient in python?
I want to calculate correlation b/w categorical features in my data. I reviewed the literature and found phi coefficient can be used for this purpose. I found one library called phik in python enter ...
- 47
0
votes
0
answers
17
views
improve the project of binary classification with categorical features
Here is a interview task of training a binary classification model. I will not show the detail of training data as the confidential issue. The variable $X$ has ten features and all the features are ...
- 127
0
votes
0
answers
69
views
How to calculate distance for symmetric binary and nomianl variables?
In the existing function dist(), the only method for nominal variable is 'binary', and it's for asymmetric binary.
However, I ...
- 3
0
votes
0
answers
18
views
How do GANs learn category distributions
I'm currently getting more into the topic of GANs and Generating Models. I've understood how the Generator and Discriminator work together in optimization to generate synthetic samples.
Now I'm ...
- 1
1
vote
0
answers
179
views
0
votes
1
answer
98
views
Categorical data preprocessing for training a algorithm
I have a training dataset where values of "Output" col is dependent on three columns (which are categorical [No ordering]).
...
- 119
1
vote
0
answers
16
views
Model a classification problem with multiple categorical varialbes as input features only. Diff Model performance
I'm having an input data with 100k rows, 8 input features, I'm trying to predict y (binary 1/0).
But all the X are categorical variables(strictly nominal variables, not ordinal). Some with 8 levels, ...
- 11