Questions tagged [categorical-data]

Categorical data can take on a limited (usually fixed) number of possible values called categories. Categorical values "label", they do not "measure". Nominal and dichotomous/binary scale types are categorical. Some people consider ordinal scale categorical too.

Filter by
Sorted by
Tagged with
2
votes
1answer
15 views

Categorical features preprocessing for clustering

Can anyone tell suggest the best practice for clustering data with mixtured features (both with categorical and continuous). I am struggling with a problem; I realized that for all metrics algorithms ...
2
votes
0answers
14 views

Encode missing data and unseen data

Let's assume that I have a classification problem and all my features are categorical data. I have missing data (and I do not want to do any imputation). Also, I know that I will have some unseen ...
1
vote
0answers
7 views

Bayesian classification of “JSON” data

"Machine Learning over JSON" describes some issues surrounding the classification of JSON documents. Namely, Categorical Features Data is Hierarchical Missingness is Chunky The first two have fairly ...
-1
votes
0answers
8 views

Does the Orange PCA plugin deal with Categorical Values?

On my dataset, I have several categorical variables (with more than 2 categories). Does Orange can deal with them ? I found some Python and R solutions here and here but I would like to know if any ...
1
vote
0answers
38 views

Naive Bayes for Categorical Features (Non Binary)

How do i use Naive Bayes Classifier (Using sklearn) for a Dataset considering that my feature set is categorical, ie more than 2 categories per feature are present. I've looked everywhere, some ...
3
votes
2answers
41 views

how to handle values that only appear once in a column?

Counting the values of a column using pandas I got the following result: ...
1
vote
1answer
26 views

What are the best practises to decide whether a variable is categorical?

What are some of the systematic ways to categorise variables into categorical or numeric? I believe using only intuition in such scenarios can many-a-times lead to major irreversible errors. What are ...
1
vote
1answer
29 views

Modeling the Price Movement- What analysis should be used

I am trying to model the price of a hotel as the check-in date arrives. I have a data set which looks like- For e.g- if I am looking at the booking date of Dec 31st, I would want to analyze the ...
0
votes
0answers
10 views

Categorical Multivariate Time Series

I have a small dataset of products of which the price varies along time. Each product is represented by categorical features mostly ( type, matter, use, location ...) and one or two scalar features ( ...
2
votes
0answers
41 views

binning high cardinality categorical features

one approach I have tried when preprocessing high cardinality categorical features (for example, US City) is to do a value count of all the values in the data, then take the top x most frequently ...
0
votes
1answer
83 views

SMOTE-NC does not help to oversample my mixed continuous/categorical dataset

When I use SMOTE-NC to oversample three classes of a 4-class classification problem, the Prec, Recall, and F1 metrics for minority classes are still VERY low (~3%). I have 32 categorical and 30 ...
0
votes
0answers
52 views

Hotel Booking Analytics: Perform an analysis in order to understand the movement of the price as the day approaches the check-in date

I am working on a hotel booking dataset. I have transactional level booking data, where each row corresponds to a booking. Please refer to below snippet of the data: I am trying to find out the ...
1
vote
0answers
10 views

Poisson Model (w/ multiple levels X)

Question Is Poisson model the best method for predicting counts among multiple levels within nominal variable? Details Imagine data of 7000 observations, where output= Obs.Count {numeric,0,1,2..8} ...
0
votes
0answers
22 views

The Difference between One Hot Encoding and LabelEncoder? [duplicate]

I am working on a ML problem to predict house prices and Zip Code is one feature which will be useful. I am also trying to use ...
2
votes
1answer
25 views

How can I count the number of occurrences of a category in dataset as part of an Sklearn Pipeline

Let us say we have a dataset with a feature such as Surname. arr['Surname'] = ['Smith', 'Jones', 'Johnson', 'Smith'] And I want to encode this categorical info ...
0
votes
0answers
9 views

What statistical test would I use to test that both data sources are matching

Suppose I am only working with two datasets, Data Source 1 with columns Customer ID and ...
1
vote
0answers
16 views

Alternatives for MultiLabelBinarizer

There a lot of information on how to handle categorical variables when preprocessing data for ML classification. However, I cannot find any feedback on how to handle categorical variables, where each ...
0
votes
1answer
32 views

How to handle large number of categories in a dataset?

I have one dataset of "Books" which contains 8 columns initially and out of which 3 of them contains text values which can be categorized. The 3 columns contains "Language-code", "Author Name" and "...
1
vote
0answers
13 views

Measuring the similarity between a numeric data matrix and one or more categorical variables?

Given a numeric data matrix $A$ of size $n \times p$, which each row represents an observation along $p$ variables, and a second categorical data matrix $M$ of size $n \times z$, where each row ...
0
votes
0answers
16 views

Class Size Imbalance for LDA or any other Content based analysis

I am running some content analysis studies on my dataset which has two different classes, and each class has a respective list of the document I am analyzing. I compare the LDA topic model inference ...
1
vote
1answer
26 views

Which classification model to use on large, high-dimensional dataset?

I face a classification task: with several features a target features is to be predicted. I'm working with python. My dataset includes 60 features from which I picked 16 which I think could be ...
3
votes
0answers
25 views

Why don't Target/LeaveOneOut Encoders work well for Regression Tasks?

In this review of categorical encoding, it states early on that For regression tasks, Target and LeaveOneOut probably won’t work well and later repeats that Target/LeaveOneOut (Owen Zhang's ...
0
votes
0answers
11 views

Identifying patterns/motifs in categorical time series

There is a data set I currently possess of the following form: $D = [(s_0, t_0), (s_1, t_1), ..., (s_N, t_N)]$, where $s_i \in \mathcal{S}, 0 \leq t_i < T$. Here $\mathcal{S}$ is a finite alphabet ...
2
votes
1answer
38 views

Mapping of categorical features into binary indicator features

I am currently reading an introductory machine learning book by Daumé (ch. 03, p. 30) and when discussing the mapping of categorical features with "n" possible values into "n" binary indicator ...
2
votes
3answers
73 views

Purpose of converting continuous data to categorical data

I was reading through a notebook tutorial working with the Titanic dataset, linked here, and noticed that they highly favored ordinal data to continuous data. For example, they converted both the Age ...
0
votes
2answers
123 views

How can Time Series Analysis be done with Categorical Variables

Most of the time series analysis tutorials/textbooks I've read about, be they for univariate or multivariate time series data, usually deal with continuous numerical variables. I currently have a ...
1
vote
0answers
15 views

Reducing the dimensions of data who's predominant categorical feature, its layer, has depths that overlaps with other samples layer values

I am working with a data set of soil types with multiple layers of varying depths and sizes with multiple features. There are $1-9$ layers each with differing dimensions, for example, a soil type ...
1
vote
1answer
51 views

High train and val results. Bad test and predict results

For my thesis project I've been trying to make a CNN for some challenging data. There's four classes with the following amount of images respectively [410, 410, 269, 206] = 1,295 total. Now I know ...
1
vote
0answers
29 views

Embedding Layer on unseen data

Let's say we have a categorical variables with 5 different categories (levels). I train and get a good model based on this dataset using embedding layer with, say, 3 embedding size and with some ...
2
votes
5answers
98 views

Large no of categorical variables with large no of categories

I'm working on a binary classification problem where the dataset is slightly imbalanced (30% class 0 | 70% class 1). Most of my features are categorical with large number of categories. For example: ...
0
votes
1answer
62 views

Categorical Variable and Target Variable

Though a similar question is answered here , but I wanted to take a different approach. Assuming that I have a binary target variable 1/0 and a categorical variable Gender M/F. From this, I can have a ...
1
vote
1answer
456 views

Feature Selection with one-hot-encoded categorical data

I have a dataset with 400+ columns. Almost 90% of these are categorical data with One-Hot-Encoding (OHE). I'm using the dataset for a classification problem. My professors asked me to perform feature ...
0
votes
0answers
24 views

Unsupervised learning/ clustering for data with multiple categorical variables

Dataset: I have been trying unsupervised clustering algorithms (K-modes & SOM) to cluster the students based on their grades in 3 exams. Should I one-hot encode the data (even though grades are ...
0
votes
0answers
19 views

How to use contrast coding schemes in the case of multiclass target variable? How to encode categorical features if contrast coding fails?

How do you deal with a dataset which only has categorical variables, all of whom have high cardinality? What is the right way to encode high cardinality categorical variables if the target variable ...
4
votes
3answers
70 views

Test dataset with categorical variable value not present in train dataset & transformer

I want to replace values of a categorical variable ( named 'six' ) by the mean of my target variable ( named 'target' ). I am fitting a transformer doing just that on a train dataset df and then ...
1
vote
2answers
365 views

Applying mean encoding before or after splitting into train and test set

I have a dataset of 50000 observations with columns of high cardinality. The best way to encode them is with mean encoding, then to use regularization. I will use CV rather than smoothing. But when I ...
0
votes
0answers
15 views

How to Decide Topics For the Documents using LDA

I am trying to Classify Topics From Documents using LDA. I want to get topics classified as human classify from words, https://medium.com/mlreview/topic-modeling-with-scikit-learn-e80d33668730 https:...
4
votes
1answer
666 views

How to handle columns with categorical data and many unique values

I have a column with categorical data with nunique 3349 values, in a 18000k row dataset, which represent cities of the world. I also have another column with 145 nunique values that I could also use ...
1
vote
0answers
42 views

When should embeddings not be used for categorical data? What are their limitations?

I recently came across the concept of embeddings so the concept is still new to me, but it is my understanding that embeddings convert one-hot encoded input data into a dense vector. Vectors ...
1
vote
4answers
93 views

How do I identify clusters that match on categorical data?

I am seeking some directions for a proper path to research the solve for this problem: My company made all our employees take a "StrengthFinders" test, which results in every employee being assigned ...
2
votes
0answers
45 views

How to deal with a potencially multiple categorical variable

I'm build a model that has, as inputs, some categorical variables. I had already dealt with this sort of data before, and applied different techniques as creation of dummy variables and factor scoring....
1
vote
1answer
238 views

Dealing with multiple distinct-value categorical variables

So, I've got a dataset with almost all of its columns are categorical variables. Problem is that most of the categorical variables have so many distinct values. For instance, one column have more ...
2
votes
1answer
29 views

Positive semidefinite kernel matrix from Gower distance

I have a dataframe with continuous and categorical variables and I want to obtain a kernel matrix for classification. The kernel matrix must be symmetric and positive semidefinite, so that no ...
1
vote
0answers
47 views

Dummy variables for unseen data in R

I got the following problem: When I trained my model I created my dummy variables(before train-test split) in the following way: ...
1
vote
1answer
313 views

What are the approaches to aggregate categorical variables?

I am working on a clickstream dataset. I have come up with the following example dataset to explain my problem: ...
1
vote
1answer
65 views

EDA for analysis of nominal variable with high cardinality

I have a nominal variable (car model) with very high cardinality (~8500 labels) and I would like to analyse its relation with a binary target variable. While I can create logical groups and compare ...
0
votes
1answer
150 views

Categorical data into numeric in excel

I have a large dataset and I would like to convert these categorical data into numeric in binary form to perform k means clustering in R. However, I get an error in value. This is the formula that I ...
1
vote
1answer
61 views

Linear regression model with (categorical) predictor variables

I used LM model with (categorical) predictor variables on my data in r like this (I have count variable as dependent/target variable): ...
2
votes
2answers
139 views

Dealing with a dataset with a mix of continuous and categorical variables

How do the choice of machine learning algorithm and preprocessing change when some of the independent variables are categorical while others are continuous? Can such data be directly applied to the ...
0
votes
1answer
330 views

Converting continuous to categorical variable

What method must be chosen for converting a continuous variable(socio-economic ratio) into a categorical variable, the quantiles are as follows: ...