Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [categorical-data]

Categorical data can take on a limited (usually fixed) number of possible values called categories. Categorical values "label", they do not "measure". Nominal and dichotomous/binary scale types are categorical. Some people consider ordinal scale categorical too.

4
votes
1answer
217 views

How to handle columns with categorical data and many unique values

I have a column with categorical data with nunique 3349 values, in a 18000k row dataset, which represent cities of the world. I also have another column with 145 nunique values that I could also use ...
0
votes
0answers
19 views

Multiple correspondence analysis usage with K-modes

I plan to cluster a categorical survey data set with 30 questions (5 answer choices to each). I am a bit confused about whether MCA is really needed? I would be able to cluster using the K-modes ...
0
votes
0answers
8 views

Regression methods for multi-dimensional categorical input and multi-dimensional real-valued output?

I wonder if there are useful regression methods for multi-dimensional categorical input and multi-dimensional real-valued output. Could random forest be one of those?
0
votes
0answers
15 views

Correlation / regression / association between one categorical variable and two non-independent others

Let's say I want to measure association / correlation between one categorical variables, and two others which are not independent. As an example (not the one I'm using), I have a data set with three ...
1
vote
0answers
19 views

When should embeddings not be used for categorical data? What are their limitations?

I recently came across the concept of embeddings so the concept is still new to me, but it is my understanding that embeddings convert one-hot encoded input data into a dense vector. Vectors ...
1
vote
4answers
86 views

How do I identify clusters that match on categorical data?

I am seeking some directions for a proper path to research the solve for this problem: My company made all our employees take a "StrengthFinders" test, which results in every employee being assigned ...
2
votes
0answers
30 views

How to deal with a potencially multiple categorical variable

I'm build a model that has, as inputs, some categorical variables. I had already dealt with this sort of data before, and applied different techniques as creation of dummy variables and factor scoring....
0
votes
0answers
10 views

Method to detect if categorical variables affect several binary variables

I have a dataset whose columns correspond to three categorical variables plus ~400 binary variables. I would like to asses the impact of these three categorical variables on the binary ones (i.e. know ...
1
vote
1answer
41 views

Dealing with multiple distinct-value categorical variables

So, I've got a dataset with almost all of its columns are categorical variables. Problem is that most of the categorical variables have so many distinct values. For instance, one column have more ...
2
votes
1answer
20 views

Positive semidefinite kernel matrix from Gower distance

I have a dataframe with continuous and categorical variables and I want to obtain a kernel matrix for classification. The kernel matrix must be symmetric and positive semidefinite, so that no ...
1
vote
0answers
29 views

Dummy variables for unseen data in R

I got the following problem: When I trained my model I created my dummy variables(before train-test split) in the following way: ...
1
vote
1answer
75 views

What are the approaches to aggregate categorical variables?

I am working on a clickstream dataset. I have come up with the following example dataset to explain my problem: ...
0
votes
0answers
20 views

what is a good probability for predicting default on loan?

i'm attempting to answer a question for a job interview, where i am given a dataset of customer data showing the customer's bank transactions over the past year, and asked to calculate the probability ...
1
vote
1answer
25 views

EDA for analysis of nominal variable with high cardinality

I have a nominal variable (car model) with very high cardinality (~8500 labels) and I would like to analyse its relation with a binary target variable. While I can create logical groups and compare ...
0
votes
1answer
34 views

Categorical data into numeric in excel

I have a large dataset and I would like to convert these categorical data into numeric in binary form to perform k means clustering in R. However, I get an error in value. This is the formula that I ...
1
vote
1answer
37 views

Linear regression model with (categorical) predictor variables

I used LM model with (categorical) predictor variables on my data in r like this (I have count variable as dependent/target variable): ...
2
votes
2answers
42 views

Dealing with a dataset with a mix of continuous and categorical variables

How do the choice of machine learning algorithm and preprocessing change when some of the independent variables are categorical while others are continuous? Can such data be directly applied to the ...
0
votes
1answer
43 views

Converting continuous to categorical variable

What method must be chosen for converting a continuous variable(socio-economic ratio) into a categorical variable, the quantiles are as follows: ...
1
vote
1answer
36 views

Regression - Unbalanced Categorical Features

I have a data set that has some unbalanced categorical features. I would like to build a regression model to predict a label using machine learning (ML). How do I handle data imbalances in ...
0
votes
0answers
14 views

How to perform T-test and chi square test to my categorical variables like country, education and predict accuracy using logistic regression?

I'm new to Data science. I have been working on a classification project which has columns (Sex, Age, Occupation, Marital Status, education, country, relationship,capital gain, income). Here income('&...
0
votes
1answer
505 views

Correlation between nominal categorical variables

I have two arrays, whose values are nominal categorical variables. Each element represents a zone of a city: in the first vector we have the class each zone belongs to (so these might also be seen as ...
1
vote
1answer
55 views

How to handle different input sizes of an NN when One-Hot-Encoding a categorical input?

let's assume an input dataset that is a mix of categorical values and real values. When preprocessing this data into an appropriate NN input, OHE is recommended because it doesn't assume any order of ...
4
votes
1answer
910 views

Confusion about Entity Embeddings of Categorical Variables - Working Example!

Problem Statement: I have problem making the Entity Embedding of Categorical Variable works for a simple dataset. I have followed the original github, or paper, or other blogposts[1,2,or this 3], or ...
0
votes
1answer
31 views

Queries regarding feature importance for categorical features

Queries regarding feature importance for categorical features: Context: I have almost 185 categorical features and these categorical features have either 2 or 3 or 8 or 1 or sometimes 4 categories, ...
1
vote
3answers
40 views

Should I build a different model for each subset

I have a dataset which has categorical variable class. I am trying to solve a regression problem I am not understanding whether I should build a model on entire dataset and consider variable class as ...
0
votes
1answer
213 views

Regression in Python with many NaN values spread across all columns

I want to do a regression to predict "value" based on the other columns from below example table. The data was collected by single indicator and not across all data points, resulting in many NaN/blank ...
1
vote
1answer
38 views

Anomaly detection in nominal big data

I have to apply an anomaly detection algorithm on big data, the values of each column on my dataframe are nominal and vary over 10000 times, the algorithms I've found only accept numeric values, is ...
1
vote
0answers
37 views

Input explanatory categorical variables along with time series into neural network

I want an advise on the ways to enter time series along with additional variables into convolutional neural network. Story first: I have a dataset of time series with daily energy consumption data (...
1
vote
1answer
27 views

Algorithm for purely categorical data

Looking for an algorithm to deal with purely categorical data. It was suggested to me to look into the K-medoids algorithm. Anyone know if there is a K-medoids algorithm R library(package)?
1
vote
1answer
35 views

Loss is bad, but accuracy increases?

I have a multicategorial classification problem for images. There are 5 (imbalanced) classes for which i use different class weights. In general there are only a few training images per class: ~56-238 ...
0
votes
1answer
11 views

Entropy loss from collapsing/merging two categories

Suppose I am counting occurrences in a sequence. For a classical example, let's say I'm counting how many of each kind of car comes down a highway. After keeping tally for a while, I see there are ...
0
votes
1answer
37 views

What is the the cost of combining categorical variables?

I have 2 categorical variables e.g. state and city. Missing are only in city. As opposed to throwing out all observations with missing values for city or throwing out city all together I was ...
0
votes
1answer
64 views

PANDAS Within Category Normalization

I'm want to normalize sales data of multiple point of sales (POS), Products and weeks. The dataframe looks like this: ...
2
votes
0answers
40 views

“Binary Encoding” in “Decision Tree” / “Random Forest” Algorithms

Is it OK to use Binary Encoding in a dataset containing categorical columns with very high cardinalities? Some facts about my dataset: My dataset has ~170000 rows One of the categoric variables has ...
1
vote
0answers
16 views

Suggestions on using model in production 1 test at a time

I have created an Artificial Neural Network with 4 categorical features and a binary outcome either 1 for suspicious or 0 for non-suspicious: ...
1
vote
1answer
136 views

Prediction with unseen values in categorical variables

I have created an Artificial Neural Network with 4 features. I am at the point where I want to test the model with a live sample of a malicious file path/exe using: ...
0
votes
1answer
51 views

How to deal with missing data for only some categories

Or in other words, data for category A is irrelevant for category B. So it is not present, how can imputing missing data distort/effect learning models broadly. I can't find any logic how to deal ...
0
votes
0answers
13 views

Response variable is nominal.

I have a dataset that has a nominal response variable with about 10 classes. Now I want to train a classification model (such as random forest or XGBOOST). I separated the data into X and y. Now, y is ...
3
votes
2answers
48 views

Dummy variable for Categorical values

The question is in reference to solution of Titanic survival predictionat kaggle . As many have did the similar kind of feature extraction, They have converted some of the numerical features (Age, ...
8
votes
3answers
1k views

How can I do classification with categorical data which is not fixed?

I have a classification problem with both categorical and numerical data. The problem I'm facing is that my categorical data is not fixed, that means that the new candidate whose label I want to ...
2
votes
1answer
51 views

Selecting the right time series model [closed]

Using Python, I am trying to predict the future sales count of a product, using historical sales data. I am also trying to predict these counts for various groups of products. For example, my columns ...
1
vote
0answers
661 views

How to plot a heatmap-like plot for categorical features?

I would greatly appreciate let me know how to plot a heatmap-like plot for categorical features? In fact, based on this post, the association between categorical ...
1
vote
1answer
2k views

How to implement feature selection for categorical variables (especially with many categories)?

I've been trying to get some ideas of how I could treat categorical variables when doing feature selection. Mainly I've been running Random Forest feature importance on Python for which preprocessing ...
0
votes
1answer
97 views

Orange imports numeric features as categorical (“file” widget)

Why are some of my numeric features not being recognized as 'numeric' types AND why can't I reclassify them? I can't share my CSV here but I can assure you those features are indeed numeric (I use ...
1
vote
1answer
190 views

tensorflow categorical data with vocabulary list - Expected binary or Unicode string, got [0,1,2,…]

I'm brand new to machine learning (having just completed the google machine learning crash course) and thought it would be good to try my hand at a Kaggle competition as a good starter to some real ...
1
vote
0answers
12 views

How do i create the same categories across two or more variables(columns) when converting integers to Factors in R?

I did merge the columns, arranged them as rows and then converted them to categories. I just wanted to know if there was a better way in doing this.
2
votes
3answers
95 views

Best practices for selecting categorical features

I'm trying to create a classifier that will predict whether someone will attend an interview or not. Each data point is for a single candidate and contains details such as the location of the ...
0
votes
1answer
233 views
1
vote
3answers
93 views

What is the best way to visualize the relationship two categorical variables

I am currently working on an ambulance dataset and one of my tasks is to find when a patient was misdiagnosed by the call dispatcher. I have two codes; a dispatch code(what the dispatcher believes is ...