Questions tagged [categorical-data]
Categorical data can take on a limited (usually fixed) number of possible values called categories. Categorical values "label", they do not "measure". Nominal and dichotomous/binary scale types are categorical. Some people consider ordinal scale categorical too.
391
questions
1
vote
1
answer
104
views
Queries regarding feature importance for categorical features
Queries regarding feature importance for categorical features:
Context:
I have almost 185 categorical features and these categorical features have either 2 or 3 or 8 or 1 or sometimes 4 categories, ...
CommunityBot
- 1
1
vote
1
answer
216
views
Is there any way to collect categorical features quickly in Julia DataFrames?
I'm using Julia 0.6.3 with Dataframes.jl
I was wondering if there was any way to get categorial features easily in Julia?
For large datasets it can be impossible to enter everything by hand.
My ...
CommunityBot
- 1
1
vote
1
answer
198
views
Model for predicting duration based on categorical data
I am working on a model which will allow me to predict how long it will take for a "job" to be completed, based on historical data. Each job has a handful of categorical characteristics (all ...
CommunityBot
- 1
0
votes
2
answers
397
views
XGBOOST with target column has categorical data and features also has categorical data
I have a huge dataset with the categorical columns in features and also my target variable is categorical.
All the values are not ordinal so I think it is best to use one hot encoding.
But I have one ...
- 1,576
0
votes
0
answers
61
views
+50
Interpretation of best subset selection regression model for factor variables with more than 2 levels
I applied the best subset selection regression model in R from leaps package to my dat dataframe. ...
- 63
1
vote
1
answer
833
views
faster alternatives to sparse.model.matrix?
I have a large dataset that is entirely categorical. I'm trying to train with it using xgboost, so I must first convert this categorical data to numerical. So far I've been using sparse.model.matrix() ...
CommunityBot
- 1
1
vote
1
answer
82
views
An Unsupervised learning method suitable for large categorical data sets
I want to detect anomalies in the bank data set in an unsupervised learning method. However, in the bank data set, all columns except time and amount were categorical data, and about half of them had ...
CommunityBot
- 1
0
votes
1
answer
133
views
Collapsing categorical data into more than 3 categories
I have a bunch of categorical, part of speech data that I want to collapse into fewer categories. np.where() won't do because I want to have 6 categories at the end: noun, verb, adjective, adverb, ...
CommunityBot
- 1
0
votes
1
answer
23
views
Help me identify the type of plot and the relationship between the dependent variables
Question: I am not sure how to describe the sample graph attached. Can you please help me identify the type of plot and how to statistically measure the relationship between the dependent variable (Y-...
- 1
1
vote
1
answer
64
views
Encode categorical data for unsupervised learning
What is the best encoder for categorical data in unsupervised learning?
I am using unsupervised learning on mixed data (such as K-means).
Before running my unsupervised algorithm, I am using dimension ...
CommunityBot
- 1
1
vote
2
answers
990
views
What are some good methods to forecast future revenue on categorical and value based data?
I have monthly snapshots (3 years) of all the contract data. It includes following information:
Contract status [Categorical]: Proposed, tracked, submitted, won, lost, etc
Contract stages [...
CommunityBot
- 1
1
vote
2
answers
3k
views
Dealing with multiple distinct-value categorical variables
So, I've got a dataset with almost all of its columns are categorical variables. Problem is that most of the categorical variables have so many distinct values.
For instance, one column have more ...
CommunityBot
- 1
15
votes
2
answers
5k
views
Why does frequency encoding work?
Frequency encoding is a widely used technique in Kaggle competitions, and many times proves to be a very reasonable way of dealing with categorical features with high cardinality. I really don't ...
- 1
2
votes
2
answers
820
views
Clustering mixed data types - numeric, categorical, arrays, and text
I have a dataset with 4 types of data columns:
...
CommunityBot
- 1
0
votes
2
answers
256
views
Custom Encoding for Categorical Features - sklearn
Just wanted to check if there are any obvious flaws with a custom encoding idea I have - for categorical features used with RandomForestClassifer or any tree-based classifier.
As all of you would know ...
CommunityBot
- 1
2
votes
2
answers
134
views
How to leverage description data in multi-class classification (dimensionality reduction)
I'm currently working with a dataset of 55k records and seven columns (one target variable), three of which are nominal categorical. The other three are 'description' fields with high cardinality, as ...
CommunityBot
- 1
1
vote
1
answer
202
views
Alternatives for MultiLabelBinarizer
There a lot of information on how to handle categorical variables when preprocessing data for ML classification. However, I cannot find any feedback on how to handle categorical variables, where each ...
CommunityBot
- 1
2
votes
1
answer
4k
views
How to plot a heatmap-like plot for categorical features?
I would greatly appreciate let me know how to plot a heatmap-like plot for categorical features?
In fact, based on this post, the association between categorical ...
CommunityBot
- 1
4
votes
1
answer
2k
views
Strategies to encode categorical variables with many categories
I was going over the Kaggle competitions IEEE,Categorical Feature Encoding Challenge and one of the ways in which categorical variables have been handled is by replacing the variables by the ...
- 3,370
0
votes
0
answers
20
views
Clustering with features of categorical type
I have a dataset of 17 features and class label for each datapoint. The description of dataset is as follows:
2 features contain values 0, 1, 2, 3
15 features contain values 0, 1, 2
class label ...
- 1
0
votes
1
answer
78
views
Encoding concept for categorical data - pick one for all the columns or different for different kinds in the same df
[Beginner here] If dataset contains - both ordinal, nonordinal (few categories) & nonordinal (multiple categories > 30). Is one supposed to pick one to encapsulate of all such situations or ...
CommunityBot
- 1
1
vote
3
answers
148
views
Anomaly detection in nominal big data
I have to apply an anomaly detection algorithm on big data, the values of each column on my dataframe are nominal and vary over 10000 times, the algorithms I've found only accept numeric values, is ...
CommunityBot
- 1
1
vote
2
answers
108
views
Can decision trees handle Nominal Categorical variables?
I have read that decision trees can handle categorical columns without encoding them.
However, as decision trees make splits on the data, how does it handle Nominal Categorical variables?
Surely a ...
1
vote
1
answer
121
views
Dummy variables for unseen data in R
I got the following problem: When I trained my model I created my dummy variables(before train-test split) in the following way:
...
CommunityBot
- 1
2
votes
1
answer
180
views
How to deal with a potencially multiple categorical variable
I'm build a model that has, as inputs, some categorical variables. I had already dealt with this sort of data before, and applied different techniques as creation of dummy variables and factor scoring....
CommunityBot
- 1
2
votes
1
answer
78
views
What to do if a specific label of a category appears only a few times?
Let's say I am trying to predict whether a car will be auctioned or not (not what I'm actually trying to do, but it represents it pretty well) using tabular data. I have the year the car was made, its ...
CommunityBot
- 1
3
votes
1
answer
396
views
Encode missing data and unseen data
Let's assume that I have a classification problem and all my features are categorical data.
I have missing data (and I do not want to do any imputation).
Also, I know that I will have some unseen ...
CommunityBot
- 1
0
votes
1
answer
17
views
Best practice for variables that only have answer if yes in previous column
I currently have a dataset that consists of survey data that has several columns that have answers dependent on the previous question. For example, I may have a question that says "Did you take ...
- 19.8k
1
vote
2
answers
98
views
How to encode high cardinality categorical data?
I have a dataset of 1600 rows and 28 columns. Only one column is partially complete with 1300 records. The rest is NaN. I did a value count of this columns and it has 84 different categories that are ...
CommunityBot
- 1
2
votes
1
answer
123
views
How to retrain a K-Modes model based on daily data?
I have read that retraining a model depends highly on what you are trying to achieve. I am conscious that maybe I need to retrain my model daily and after a certain time I have to train the model ...
CommunityBot
- 1
0
votes
1
answer
43
views
Do you use categorical data types?
Personally I've never used the categorical data type in pandas, and leave eveything as objects.
I've seen it has the capability to be saved as parquet files, saves data etc...
What are the pros and ...
- 289
0
votes
0
answers
14
views
What is the best way to whittle down rendundant categorical data?
I'm trying to build a linear regression model in Tensorflow (and preprocessing with pandas) that will help me categorize bank transactions. I'm trying to whittle down the vendor parameter, because the ...
0
votes
2
answers
71
views
How would I classify this variable?
I am learning about the difference between categorical, ordinal and numerical variables.
From what I understand:
Categorical variables have 2+ categories without any intrinsic order.
Ordinal ...
CommunityBot
- 1
0
votes
1
answer
111
views
Categorical data preprocessing for training a algorithm
I have a training dataset where values of "Output" col is dependent on three columns (which are categorical [No ordering]).
...
CommunityBot
- 1
0
votes
1
answer
55
views
How to efficiently reduce dimensions of one-hot encoded categorical values?
I'm currently working on a project where I'm using an LSTM to learn and predict sequences of categorical data.
My dataset consists of variable-length sequences of items $s_i = [x_{i_0}, x_{i_1}, ..., ...
- 20.7k
1
vote
2
answers
804
views
Aggregating multiple encoded categorical values
I am trying find commonly used techniques when dealing with high cardinality multi-valued categorical variables.
I am currently using a dataset with a feature CATEGORY which has a cardinality of ~20,...
CommunityBot
- 1
0
votes
0
answers
21
views
Finding a relation between three variables
I am new to data mining and have learnt about association rules mining, classification analysis, cluster analysis and outlier analysis. So, to find relationship between three variables, regression ...
- 1
2
votes
1
answer
235
views
Logistic Regression Model for categorical features with multiple values in each category
I am working on an insurance use case to build a logistic regression classifier to predict if a policy will lapse or not. The dataset has more than 20 categorical features for a policy. Each ...
CommunityBot
- 1
6
votes
3
answers
307
views
Anomaly detection using clustering of highly correlated Categorical data
My data has two columns and both are highly correlated e.g. if column1 has value ABC, column2 should be XYZ i.e. ABC-->XYZ. If column2 has anything else it's Anomaly. Likewise, there are thousands ...
CommunityBot
- 1
1
vote
1
answer
255
views
How to predict on data that is label encoded as end user will input a categorical data?
My dataset contains about 29 features with 3 class labels as result. Among these 29 features around 24 features are categorical i cannot transform each category into numbers as there are many more ...
CommunityBot
- 1
0
votes
1
answer
24
views
Data Leakage possibility related to replacing rare values on a categorical column
I am exploring the possibility of data leakage for categorical columns, replacing rare values with category 'Other'
Let's say I have a DF with 40 categorical columns. I will check each of them, find ...
- 649
0
votes
0
answers
25
views
0
votes
0
answers
33
views
How to do forecasting with categorical timeseries?
I have a dataset that is in the form of categorical timeseries: (specifically, we either know or don't know the values of 6 degrees of freedom of an object at any given time). If we know it, it's ...
- 101
0
votes
0
answers
13
views
How to make a predictive model using a timeseries data consisted of binary information?
I have a set of data that is showing the state of an object as a function of time. I would like to know what and how I should be utilizing machine learning to predict the state of the object at some ...
- 101
0
votes
0
answers
11
views
Encoding of categorical variables to reduce the effect of erroneous labels
I have a structured dataset containing (nominal) categorical variables encoded as labels, let's say a feature includes labels from 1 to 20. Some of the labels in that feature could just be errors, ...
- 11
0
votes
1
answer
25
views
Categorical Dataset Machine Learning
I have a dataset that is completely binary and labeled. I would like to be able to use machine learning for one of the columns. I have read that unsupervised models, such as K-Means, do not work with ...
- 649
-1
votes
1
answer
91
views
Manual computation of the predictions in a convolutional neural network
I am trying to manually compute the predictions of the Keras library for a convolutional neural network. However, I am struggling a lot to match my final result with the ones provided by Keras. I do ...
- 59
1
vote
1
answer
49
views
Customer Segmentation with mixed data [closed]
I want to perform clustering. I am reading about this topic but I am totally confused. My dataset has 490 observations and it consists of numerical data (3 columns: Recency, Frequency, Monetary), ...
- 649
1
vote
1
answer
23
views
How to predict probability of an event when we have a month to month data?
I'm trying to find references about how to proceed to get the probability of an event happening when we have "temporal data" in our table
My data is basically:
hex_id: id of the object
date:...
0
votes
0
answers
25
views
How to detect Novelty from different ranges of target variable?
I've a dataset of multiple categorical columns along with a target column that is continuous. Assume combination of categorical columns has a different range of values of target. Ex
Col1 - col2 - col3 ...
