Questions tagged [categorical-data]

Categorical data can take on a limited (usually fixed) number of possible values called categories. Categorical values "label", they do not "measure". Nominal and dichotomous/binary scale types are categorical. Some people consider ordinal scale categorical too.

Filter by
Sorted by
Tagged with
1 vote
1 answer
101 views

Queries regarding feature importance for categorical features

Queries regarding feature importance for categorical features: Context: I have almost 185 categorical features and these categorical features have either 2 or 3 or 8 or 1 or sometimes 4 categories, ...
0 votes
0 answers
6 views

Ordinal Encoding for Differing Categories

As an example, I have a dataset of available games. ...
0 votes
0 answers
15 views

How to do forecasting with categorical timeseries?

I have a dataset that is in the form of categorical timeseries: (specifically, we either know or don't know the values of 6 degrees of freedom of an object at any given time). If we know it, it's ...
0 votes
0 answers
11 views

How to make a predictive model using a timeseries data consisted of binary information?

I have a set of data that is showing the state of an object as a function of time. I would like to know what and how I should be utilizing machine learning to predict the state of the object at some ...
1 vote
1 answer
208 views

Is there any way to collect categorical features quickly in Julia DataFrames?

I'm using Julia 0.6.3 with Dataframes.jl I was wondering if there was any way to get categorial features easily in Julia? For large datasets it can be impossible to enter everything by hand. My ...
0 votes
0 answers
9 views

Encoding of categorical variables to reduce the effect of erroneous labels

I have a structured dataset containing (nominal) categorical variables encoded as labels, let's say a feature includes labels from 1 to 20. Some of the labels in that feature could just be errors, ...
1 vote
1 answer
167 views

Model for predicting duration based on categorical data

I am working on a model which will allow me to predict how long it will take for a "job" to be completed, based on historical data. Each job has a handful of categorical characteristics (all ...
0 votes
1 answer
23 views

Categorical Dataset Machine Learning

I have a dataset that is completely binary and labeled. I would like to be able to use machine learning for one of the columns. I have read that unsupervised models, such as K-Means, do not work with ...
1 vote
1 answer
796 views

faster alternatives to sparse.model.matrix?

I have a large dataset that is entirely categorical. I'm trying to train with it using xgboost, so I must first convert this categorical data to numerical. So far I've been using sparse.model.matrix() ...
-1 votes
1 answer
79 views

Manual computation of the predictions in a convolutional neural network

I am trying to manually compute the predictions of the Keras library for a convolutional neural network. However, I am struggling a lot to match my final result with the ones provided by Keras. I do ...
0 votes
1 answer
36 views

Do you use categorical data types?

Personally I've never used the categorical data type in pandas, and leave eveything as objects. I've seen it has the capability to be saved as parquet files, saves data etc... What are the pros and ...
1 vote
1 answer
23 views

Customer Segmentation with mixed data

I want to perform clustering. I am reading about this topic but I am totally confused. My dataset has 490 observations and it consists of numerical data (3 columns: Recency, Frequency, Monetary), ...
1 vote
1 answer
73 views

An Unsupervised learning method suitable for large categorical data sets

I want to detect anomalies in the bank data set in an unsupervised learning method. However, in the bank data set, all columns except time and amount were categorical data, and about half of them had ...
0 votes
1 answer
103 views

Collapsing categorical data into more than 3 categories

I have a bunch of categorical, part of speech data that I want to collapse into fewer categories. np.where() won't do because I want to have 6 categories at the end: noun, verb, adjective, adverb, ...
0 votes
1 answer
332 views

XGBOOST with target column has categorical data and features also has categorical data

I have a huge dataset with the categorical columns in features and also my target variable is categorical. All the values are not ordinal so I think it is best to use one hot encoding. But I have one ...
1 vote
2 answers
833 views

What are some good methods to forecast future revenue on categorical and value based data?

I have monthly snapshots (3 years) of all the contract data. It includes following information: Contract status [Categorical]: Proposed, tracked, submitted, won, lost, etc Contract stages [...
1 vote
2 answers
3k views

Dealing with multiple distinct-value categorical variables

So, I've got a dataset with almost all of its columns are categorical variables. Problem is that most of the categorical variables have so many distinct values. For instance, one column have more ...
0 votes
1 answer
13 views

How to predict probability of an event when we have a month to month data?

I'm trying to find references about how to proceed to get the probability of an event happening when we have "temporal data" in our table My data is basically: hex_id: id of the object date:...
6 votes
3 answers
301 views

Anomaly detection using clustering of highly correlated Categorical data

My data has two columns and both are highly correlated e.g. if column1 has value ABC, column2 should be XYZ i.e. ABC-->XYZ. If column2 has anything else it's Anomaly. Likewise, there are thousands ...
0 votes
0 answers
24 views

How to detect Novelty from different ranges of target variable?

I've a dataset of multiple categorical columns along with a target column that is continuous. Assume combination of categorical columns has a different range of values of target. Ex Col1 - col2 - col3 ...
0 votes
1 answer
18 views

How do I use one hot encoding with 240 nominal variables and each with equal occurrence?

The method I saw that's generally used to deal with large # of nominal variables is to keep the most frequent variables and introduce a new "other" category. But that's not possible with my ...
0 votes
2 answers
186 views

Custom Encoding for Categorical Features - sklearn

Just wanted to check if there are any obvious flaws with a custom encoding idea I have - for categorical features used with RandomForestClassifer or any tree-based classifier. As all of you would know ...
1 vote
1 answer
175 views

Alternatives for MultiLabelBinarizer

There a lot of information on how to handle categorical variables when preprocessing data for ML classification. However, I cannot find any feedback on how to handle categorical variables, where each ...
2 votes
1 answer
4k views

How to plot a heatmap-like plot for categorical features?

I would greatly appreciate let me know how to plot a heatmap-like plot for categorical features? In fact, based on this post, the association between categorical ...
2 votes
2 answers
129 views

How to leverage description data in multi-class classification (dimensionality reduction)

I'm currently working with a dataset of 55k records and seven columns (one target variable), three of which are nominal categorical. The other three are 'description' fields with high cardinality, as ...
0 votes
0 answers
30 views

How to do DBSCAN clustering with mixed variables (numerical features and binary/ordinal variables)?

I have a question written at the end of the post which refers to the "Distances" paragraph. The other first two paragraphs give additional info. Context I'm working on a project where I have ...
0 votes
1 answer
55 views

Encoding concept for categorical data - pick one for all the columns or different for different kinds in the same df

[Beginner here] If dataset contains - both ordinal, nonordinal (few categories) & nonordinal (multiple categories > 30). Is one supposed to pick one to encapsulate of all such situations or ...
2 votes
1 answer
199 views

DBSCAN on textual and numerical columns

I have a dataset which has two columns: title price sentence1 12 sentence2 13 I have used doc2vec to convert the ...
3 votes
1 answer
314 views

PCA and k-means for categorical variables?

I have a clustering task at hand. The data that I have contains only categorical variables. So, k-modes seemed like the best option. But I am not sure what are the data pre processing steps required ...
2 votes
2 answers
668 views

Clustering mixed data types - numeric, categorical, arrays, and text

I have a dataset with 4 types of data columns: ...
1 vote
3 answers
134 views

Anomaly detection in nominal big data

I have to apply an anomaly detection algorithm on big data, the values of each column on my dataframe are nominal and vary over 10000 times, the algorithms I've found only accept numeric values, is ...
2 votes
1 answer
173 views

How to deal with a potencially multiple categorical variable

I'm build a model that has, as inputs, some categorical variables. I had already dealt with this sort of data before, and applied different techniques as creation of dummy variables and factor scoring....
1 vote
1 answer
28 views

Encode categorical data for unsupervised learning

What is the best encoder for categorical data in unsupervised learning? I am using unsupervised learning on mixed data (such as K-means). Before running my unsupervised algorithm, I am using dimension ...
0 votes
0 answers
14 views

Clustering mixed type variables with Orange

I wonder if with Orange it is possible to cluster mixed type data, so a dataset with numeric as well as discrete (categorical) data (ordered / unordered). Can you show an example of how that could be ...
1 vote
1 answer
114 views

Dummy variables for unseen data in R

I got the following problem: When I trained my model I created my dummy variables(before train-test split) in the following way: ...
0 votes
1 answer
18 views

Is there a procedure for determining if a classification problem is ill-defined?

Consider a group of objects denoted $O = \{o_0, o_1, \cdots\}$ where each object is associated with a feature vector $F = \{f_0, f_1, \cdots\, f_{N-1}\}$. For this case, assume the features are ...
2 votes
1 answer
89 views

How to use categorical data in forecasting with Prophet?

I'm trying to create a model to predict the number of players on a video game at a certain time and was trying to figure out how to integrate categorical data into my forecasting problem. So far, my ...
2 votes
1 answer
74 views

What to do if a specific label of a category appears only a few times?

Let's say I am trying to predict whether a car will be auctioned or not (not what I'm actually trying to do, but it represents it pretty well) using tabular data. I have the year the car was made, its ...
1 vote
2 answers
89 views

How to encode high cardinality categorical data?

I have a dataset of 1600 rows and 28 columns. Only one column is partially complete with 1300 records. The rest is NaN. I did a value count of this columns and it has 84 different categories that are ...
0 votes
1 answer
114 views

Does Scikit-Learn's OneHotEncoder make all Columns Categorical?

I've been using Scikit-Learn's OneHotEncoder to turn categorical data into binary columns, however, it seems that fitting ...
3 votes
1 answer
378 views

Encode missing data and unseen data

Let's assume that I have a classification problem and all my features are categorical data. I have missing data (and I do not want to do any imputation). Also, I know that I will have some unseen ...
1 vote
2 answers
1k views

High train and val results. Bad test and predict results

For my thesis project I've been trying to make a CNN for some challenging data. There's four classes with the following amount of images respectively [410, 410, 269, 206] = 1,295 total. Now I know ...
1 vote
1 answer
33 views

N-ary decision tree with categorical features

I want to build an n-ary decision tree with categorical features. I am using ordinary ID3 algorithm to build a tree. Lets take the next dataset as a training dataset for building a decision tree: ...
0 votes
0 answers
24 views

What Would Be a Good Measure of Feature Importance in Regression?

Doing simple supervised regression where the label is a floating point number (guaranteed positive) and the features are a mix of continuous floating point values and some categorical features. What ...
0 votes
0 answers
11 views

Dealing with categorical columns with unbalanced value count

I'm doing some data processing and wondering what is the best practice for dealing with categorical columns that has a value counts plot looking like the below (these are one-hot-encoded at a later ...
2 votes
1 answer
102 views

How to retrain a K-Modes model based on daily data?

I have read that retraining a model depends highly on what you are trying to achieve. I am conscious that maybe I need to retrain my model daily and after a certain time I have to train the model ...
1 vote
1 answer
19 views

How do well informed labels for ordinal encoding improve model performance?

From Kaggle's intermediate machine learning tutorial, it was stated that for each column, we randomly assign each unique value to a different integer. This is a common approach that is simpler than ...
0 votes
0 answers
11 views

How to classify and organize very complex data for easy future reference?

I've studied binomial nomenclature in college, but forgot all about it! I use Joplin to organize my life and business, but it's getting out of hand complicated and I can't find stuff. Struggling on ...
0 votes
1 answer
1k views

Multiple variable as input and output

I'm trying to predict the possible diagnosis given a consultation reason. I have ID's for all the data. So my data kind of looks like below ...
0 votes
2 answers
63 views

How would I classify this variable?

I am learning about the difference between categorical, ordinal and numerical variables. From what I understand: Categorical variables have 2+ categories without any intrinsic order. Ordinal ...

1
2 3 4 5
8