Questions tagged [categorical-data]

Categorical data can take on a limited (usually fixed) number of possible values called categories. Categorical values "label", they do not "measure". Nominal and dichotomous/binary scale types are categorical. Some people consider ordinal scale categorical too.

Filter by
Sorted by
Tagged with
0
votes
0answers
23 views

How to run PCA when data contains some categorial features?

Assume that we have a dataset with various features, and some of the features are categorial. And PCA dosn't work good on categorical features. How should I handle such datasets using PCA, what is ...
2
votes
1answer
31 views

How to retrain a K-Modes model based on daily data?

I have read that retraining a model depends highly on what you are trying to achieve. I am conscious that maybe I need to retrain my model daily and after a certain time I have to train the model ...
2
votes
1answer
26 views

Two questions about one-hot encoding: drop first? and features with thousands of categories [closed]

I have two questions about one-hot feature encoding: (1) Is it considered a "best practice" to drop the first (or at least one) one-hot encoded feature when one-hot encoding, like you would ...
0
votes
1answer
877 views

Multiple variable as input and output

I'm trying to predict the possible diagnosis given a consultation reason. I have ID's for all the data. So my data kind of looks like below ...
0
votes
2answers
5k views

Correlation between nominal categorical variables

I have two arrays, whose values are nominal categorical variables. Each element represents a zone of a city: in the first vector we have the class each zone belongs to (so these might also be seen as ...
3
votes
2answers
440 views

How to Set the Same Categorical Codes to Train and Test data? Python-Pandas

NOTE: If someone else it's wondering about this topic, I understand you're getting deeper in the Data Analysis world, so I did this question before to learn that: You encode categorical values as ...
2
votes
1answer
118 views

Logistic Regression Model for categorical features with multiple values in each category

I am working on an insurance use case to build a logistic regression classifier to predict if a policy will lapse or not. The dataset has more than 20 categorical features for a policy. Each ...
1
vote
1answer
177 views

Is there any way to collect categorical features quickly in Julia DataFrames?

I'm using Julia 0.6.3 with Dataframes.jl I was wondering if there was any way to get categorial features easily in Julia? For large datasets it can be impossible to enter everything by hand. My ...
1
vote
0answers
19 views

When doing feature selection, are features like “year”, “month” considered as ordinal features or should I convert them to strings?

I am working on a hotel reservation dataset that has both categorical and numerical (continuous and discrete) features (26 columns, 30244 rows). Target is also categorical and it says if the user &...
0
votes
2answers
40 views

How would I classify this variable?

I am learning about the difference between categorical, ordinal and numerical variables. From what I understand: Categorical variables have 2+ categories without any intrinsic order. Ordinal ...
1
vote
0answers
20 views

Feature Selection Statistical Test for Nominal Response Vs Continuous Predictors? (in R)

I cannot find much information on this, none so far useful. I have a sparse data set with 17K+ columns of continuous gene expressions, an example of a typical column: (3.15, 0, 7.1294, 0, 0, 0, 2300.2,...
1
vote
1answer
629 views

faster alternatives to sparse.model.matrix?

I have a large dataset that is entirely categorical. I'm trying to train with it using xgboost, so I must first convert this categorical data to numerical. So far I've been using sparse.model.matrix() ...
1
vote
1answer
27 views

Handling nominal category features in decision tree

I have been reading some stackoverflow questions on how to handle nominal features for decision tree (sklearn implementation). One of the answer states that : Using a OneHotEncoder is the only current ...
1
vote
0answers
8 views

spark ml StringIndexer vs OneHotEncoder, when to use which?

Confused as to when to use StringIndexer vs StringIndexer+OneHotEncoder. The OneHotEncoder docs say For string type input data, it is common to encode categorical features using StringIndexer first. ...
0
votes
1answer
25 views

Entropy loss from collapsing/merging two categories

Suppose I am counting occurrences in a sequence. For a classical example, let's say I'm counting how many of each kind of car comes down a highway. After keeping tally for a while, I see there are ...
1
vote
1answer
27 views

How to identify if there is a relationship between 5 categorical independent variables to a binary dependent variable?

My dataset has 5 independent variables, each with a value of either Large, Medium or None and a binary dependent variable. The dataset has 67 rows with a split of 17:50. I would like to identify if ...
1
vote
0answers
23 views

An Unsupervised learning method suitable for large categorical data sets

I want to detect anomalies in the bank data set in an unsupervised learning method. However, in the bank data set, all columns except time and amount were categorical data, and about half of them had ...
1
vote
1answer
78 views

Queries regarding feature importance for categorical features

Queries regarding feature importance for categorical features: Context: I have almost 185 categorical features and these categorical features have either 2 or 3 or 8 or 1 or sometimes 4 categories, ...
0
votes
1answer
16 views

Handling categorical data with more over 100 unique classes

I am working with a pure categorical data set. And some classes have more than 100 unique values. I could not find any appropriate encoding possibility. So I created a SQL table, where each value got ...
0
votes
1answer
31 views

Model for predicting duration based on categorical data

I am working on a model which will allow me to predict how long it will take for a "job" to be completed, based on historical data. Each job has a handful of categorical characteristics (all ...
1
vote
1answer
98 views

Anomaly detection in nominal big data

I have to apply an anomaly detection algorithm on big data, the values of each column on my dataframe are nominal and vary over 10000 times, the algorithms I've found only accept numeric values, is ...
0
votes
1answer
1k views

Should the type of Boolean categorical features be numerical or categorical after encoding?

There are categorical features which have two different value in my dataframe next to numerical features. I've converted these categorical values to 0 or 1. I will apply correalation elimination on ...
1
vote
2answers
18 views

How do I assign specific values to categorical variables

I have a Pandas data frame with columns within a survey with the following categorical values - "Increased, Decreased, Neutral". My question is how can I assign specific numerical values to ...
1
vote
2answers
225 views

What are some good methods to forecast future revenue on categorical and value based data?

I have monthly snapshots (3 years) of all the contract data. It includes following information: Contract status [Categorical]: Proposed, tracked, submitted, won, lost, etc Contract stages [...
1
vote
1answer
314 views

Aggregating multiple encoded categorical values

I am trying find commonly used techniques when dealing with high cardinality multi-valued categorical variables. I am currently using a dataset with a feature CATEGORY which has a cardinality of ~20,...
1
vote
2answers
2k views

Dealing with multiple distinct-value categorical variables

So, I've got a dataset with almost all of its columns are categorical variables. Problem is that most of the categorical variables have so many distinct values. For instance, one column have more ...
0
votes
0answers
31 views

Encoding Categorical Data

I am trying to predict a car price from a given dataset. When I am trying to encode the categorical data I am getting an error. I am attaching an image with the error below. Can someone help me know ...
1
vote
0answers
27 views

How to build multiple variable regression having a mix of numerical & categorical features?

There is a need to estimate Annual Average Daily Traffic Volume (AADT). We have bunch of data about vehicles' speeds during several years. It is noticed that AADT depends on the average number of such ...
8
votes
4answers
7k views

One Hot encoding for large number of values

How do we use one hot encoding if the number of values which a categorical variable can take is large ? In my case it is 56 values. So as per usual method I would have to add 56 columns (56 binary ...
2
votes
5answers
128 views

How do you predict a continuous variable when all your independent variables are categorical

I am new to data science and ML. Recently I have been given a sales dataset which contains weekly sales of a fashion brand. It has information about the product like category(t shirt, polo shirt, ...
2
votes
1answer
62 views

Clustering with Only Categorical Features

I am trying to do clustering with a bunch (24) of categorical features. I have done some research and found a lot of people recommending something such as K-Modes. I tried running K-Modes on my data ...
12
votes
3answers
24k views

How to convert categorical data to numerical data in Pyspark

I am using Ipython notebook to work with pyspark applications. I have a CSV file with lots of categorical columns to determine whether the income falls under or over the 50k range. I would like to ...
1
vote
1answer
37 views

one hot encoding target variable in tree and non tree (knn) methods

I am learning about label encoders, one hot encoding etc applied to datasets for classification via KNN and XGBoost type trees. However, I am a bit confused as to whether the target variable should be ...
2
votes
1answer
27 views

Individual differences in response time (RT) experiment - searching for the right test

Given a distribution of response times to different categorical variables, what's the best way to test for individual differences? or more specifically: There are 100 people pressing buttons in 10 ...
2
votes
1answer
125 views

How can I count the number of occurrences of a category in dataset as part of an Sklearn Pipeline

Let us say we have a dataset with a feature such as Surname. arr['Surname'] = ['Smith', 'Jones', 'Johnson', 'Smith'] And I want to encode this categorical info ...
1
vote
2answers
31 views

Linear Regression with Category variables

I'm currently learning and exploring machine learning and understand the basics of linear regression based on two numerical variables, but now I wish to go a little further and need some guidance ...
1
vote
0answers
143 views

Association between Categorical Variables and regression

We perform data analysis and build models. Say, for example, I built a regression model that has more than one predictor (multiple regression). We then check many things: normality, multicollinearity, ...
1
vote
1answer
102 views

Alternatives for MultiLabelBinarizer

There a lot of information on how to handle categorical variables when preprocessing data for ML classification. However, I cannot find any feedback on how to handle categorical variables, where each ...
1
vote
0answers
22 views

How to get correlation between the categories of two categorical variable?

I have a categorical variable with 2 categories ("Health") ('healthy', 'not_healthy') and another categorical variable ("country") with 5 categories ("english", "eua&...
1
vote
1answer
110 views

Which classification model to use on large, high-dimensional dataset?

I face a classification task: with several features a target features is to be predicted. I'm working with python. My dataset includes 60 features from which I picked 16 which I think could be ...
0
votes
1answer
58 views

What to do if a specific label of a category appears only a few times?

Let's say I am trying to predict whether a car will be auctioned or not (not what I'm actually trying to do, but it represents it pretty well) using tabular data. I have the year the car was made, its ...
6
votes
2answers
18k views

Always drop the first column after performing One Hot Encoding?

Since one of the columns can be generated completely from the others, and hence retaining this extra column does not add any new information for the modelling process, would it be good practice to ...
1
vote
2answers
26 views

Different number of features in train vs test when using Label Encoding

This is not a duplicate of Different number of features in train vs test There are some categorical columns in my data, and the cardinality for each of them is large, so I chose to use LabelEncoding ...
0
votes
0answers
26 views

Equitable selection of users through ranking

I am looking to take a dataset largely derived of user input in categorical form, this sign up sheet asks for many data points such as age group, race, sign up date, as well as a few others. My goal ...
2
votes
2answers
89 views

How to leverage description data in multi-class classification (dimensionality reduction)

I'm currently working with a dataset of 55k records and seven columns (one target variable), three of which are nominal categorical. The other three are 'description' fields with high cardinality, as ...
0
votes
0answers
16 views

Which are the features selection techniques depending on the combination on cat num columns in independent and dependent features?

I am very confused: For what I understood I should: Multicollinearity check with Pearson corr and possibly consider to drop multicolliner features Then? I am very confused feature selection should be ...
1
vote
1answer
49 views

DBSCAN on textual and numerical columns

I have a dataset which has two columns: title price sentence1 12 sentence2 13 I have used doc2vec to convert the ...
2
votes
1answer
349 views

ML methods for prediction, using categorical variables and time

Most of the time series analysis tutorials/textbooks I found time series data, usually deal with continuous numerical variables. I am currently trying to solve a problem that deals with multivariate ...
3
votes
2answers
249 views

Anomaly detection using clustering of highly correlated Categorical data

My data has two columns and both are highly correlated e.g. if column1 has value ABC, column2 should be XYZ i.e. ABC-->XYZ. If column2 has anything else its Anomaly. Likewise there are thousands of ...

1
2 3 4 5
7