Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [categorical-data]

Categorical data can take on a limited (usually fixed) number of possible values called categories. Categorical values "label", they do not "measure". Nominal and dichotomous/binary scale types are categorical. Some people consider ordinal scale categorical too.

0
votes
0answers
15 views

OneHotEncoder on multiple columns belonging to same categories for encoding and decoding data

I have multiple columns consisting of categorical variables which are in the form of integer values ranging from 0-4. But, all columns belong to the same category. I tried using OneHotEncoder from ...
0
votes
0answers
10 views

Can I use MANCOVA with categorical data?

By doing an experiment, I would like to analyze the effect of 3 categorical independent variables (with 2 levels each) on three constructs measured with ordinal variables. I also have 2 covariates (...
2
votes
0answers
18 views

“Binary Encoding” in “Decision Tree” / “Random Forest” Algorithms

Is it OK to use Binary Encoding in a dataset containing categorical columns with very high cardinalities? Some facts about my dataset: My dataset has ~170000 rows One of the categoric variables has ...
1
vote
0answers
10 views

Suggestions on using model in production 1 test at a time

I have created an Artificial Neural Network with 4 categorical features and a binary outcome either 1 for suspicious or 0 for non-suspicious: ...
0
votes
0answers
16 views

Having issues with selecting new prediction using new categorical variables for numpy array

I have created an Artificial Neural Network with 4 features. I am at the point where I want to test the model with a live sample of a malicious file path/exe using: ...
0
votes
1answer
29 views

How to deal with missing data for only some categories

Or in other words, data for category A is irrelevant for category B. So it is not present, how can imputing missing data distort/effect learning models broadly. I can't find any logic how to deal ...
0
votes
0answers
8 views

Response variable is nominal.

I have a dataset that has a nominal response variable with about 10 classes. Now I want to train a classification model (such as random forest or XGBOOST). I separated the data into X and y. Now, y is ...
0
votes
0answers
12 views

Must label encoding for categorical variable be sequential?

I came across label-encoding for a categorical variable that encodes {X, Y, Z} to {1, 3, 15}. By default, scikit-learn returns {0, 1, 2}. For logistic regression, would encoding a categorical variable ...
0
votes
0answers
14 views

Can I use MCA on categorical features, and PCA on numeric then combine both for learning

So all is said in the title. I have a mix of both categorical and numeric features, both are more than 20 columns and reside in the same data-set. I am using PCA solution from sklearn.decomposition ...
3
votes
2answers
21 views

Dummy variable for Categorical values

The question is in reference to solution of Titanic survival predictionat kaggle . As many have did the similar kind of feature extraction, They have converted some of the numerical features (Age, ...
7
votes
3answers
133 views

How can I do classification with categorical data which is not fixed?

I have a classification problem with both categorical and numerical data. The problem I'm facing is that my categorical data is not fixed, that means that the new candidate whose label I want to ...
2
votes
1answer
32 views

Selecting the right time series model [closed]

Using Python, I am trying to predict the future sales count of a product, using historical sales data. I am also trying to predict these counts for various groups of products. For example, my columns ...
1
vote
0answers
92 views

How to plot a heatmap-like plot for categorical features?

I would greatly appreciate let me know how to plot a heatmap-like plot for categorical features? In fact, based on this post, the association between categorical ...
1
vote
1answer
272 views

How to implement feature selection for categorical variables (especially with many categories)?

I've been trying to get some ideas of how I could treat categorical variables when doing feature selection. Mainly I've been running Random Forest feature importance on Python for which preprocessing ...
0
votes
1answer
29 views

Orange imports numeric features as categorical (“file” widget)

Why are some of my numeric features not being recognized as 'numeric' types AND why can't I reclassify them? I can't share my CSV here but I can assure you those features are indeed numeric (I use ...
1
vote
1answer
63 views

tensorflow categorical data with vocabulary list - Expected binary or Unicode string, got [0,1,2,…]

I'm brand new to machine learning (having just completed the google machine learning crash course) and thought it would be good to try my hand at a Kaggle competition as a good starter to some real ...
1
vote
0answers
9 views

How do i create the same categories across two or more variables(columns) when converting integers to Factors in R?

I did merge the columns, arranged them as rows and then converted them to categories. I just wanted to know if there was a better way in doing this.
1
vote
1answer
29 views

Best practices for selecting categorical features

I'm trying to create a classifier that will predict whether someone will attend an interview or not. Each data point is for a single candidate and contains details such as the location of the ...
0
votes
1answer
34 views
1
vote
3answers
66 views

What is the best way to visualize the relationship two categorical variables

I am currently working on an ambulance dataset and one of my tasks is to find when a patient was misdiagnosed by the call dispatcher. I have two codes; a dispatch code(what the dispatcher believes is ...
0
votes
1answer
41 views

Anomaly detection using clustering of highly correlated Categorical data

My data has two columns and both are highly correlated e.g. if column1 has value ABC, column2 should be XYZ i.e. ABC-->XYZ. If column2 has anything else its Anomaly. Likewise there are thousands of ...
0
votes
1answer
71 views

Categorical data for sklearns Isolation Forrest

I'm trying to do anomaly detection with Isolation Forests (IF) in sklearn. Except for the fact that it is a great method of anomaly detection, I also want to use it because about half of my features ...
0
votes
1answer
63 views

250 Categorical values

I have a dataset which has only categorical values. As I came across a few articles people suggested that KNN / Random forest would work for dataset like this. Though in R it couldn't handle as if ...
0
votes
1answer
40 views

Convert nominal to numeric variables?

I am trying to develeop an algorithm with sklearn and Tensorflow to predict which car can be offer to each customer. To do that I have a database with the answers of one survey to 1000 customers. An ...
0
votes
0answers
10 views

Is there any way to collect categorical features quickly in Julia DataFrames?

I'm using Julia 0.6.3 with Dataframes.jl I was wondering if there was any way to get categorial features easily in Julia? For large datasets it can be impossible to enter everything by hand. My ...
2
votes
1answer
42 views

Same predictors in test set but I want different outputs

I have a (training) dataset about what TV spectators are watching and for how long. The goal (at new set - the test set) is to predict for how long the TV spectators will watch a specific channel and ...
1
vote
0answers
57 views

What is the relationship between correlation ratio and one-way Anova?

According to the answer to this post, it is recommended to use one-way anova to compute the dependence between a categorical and a numerical variable. Besides, the second answer to this post says ...
0
votes
0answers
9 views

Which method can I use to explore dependences between categorial(nominative) variables?

There is some sociological research data. For example: How can I explore some sort of correlation between income level and district, or which statistical measures based on this data can I calculate? ...
0
votes
1answer
27 views

What are appropriate labels for age categorical labels?

I am converting some age data to categorical variables. What are some appropriate labels? Some people might take offense to using "Young", "Old" or "Millenial", etc. Is there a "standard" list of ...
0
votes
0answers
44 views

Dropping one of the one-hot encoded columns for Gradient Boost Methods/Decision Trees?

If I have the categorical variable like favorite_color and it has unique values red, green, ...
0
votes
0answers
64 views

Clustering of devices in locations?

My question is about using some sort of AI to assess if devices are located in any of a list of venues. I'd ask of machine learning, but so far we're doing this with an expert system, and we are ...
0
votes
0answers
73 views

faster alternatives to sparse.model.matrix?

I have a large dataset that is entirely categorical. I'm trying to train with it using xgboost, so I must first convert this categorical data to numerical. So far I've been using sparse.model.matrix() ...
1
vote
2answers
353 views

One hot encoding large dataset

Initially, I have a dataset where for each row there is user_id and product_ids he bought. In that dataset there are 478191 products bought by different users. In order to discover frequent items ...
2
votes
1answer
167 views

Data scaling before PCA: how to deal with categorical values?

I have to apply PCA on a dataset, which contains both numerical and categorical values. In the preprocessing phase, I converted all the categorical values in numerical, so that the software can deal ...
1
vote
1answer
35 views

Dropping less frequently used categorical data?

I'm new to the datascience field and working on an assignment. I have a dataset with 150K rows with a categorical and numerical data, the target is a boolean. A categorical column consist of quite ...
2
votes
1answer
37 views

how to decide categorical variables for prediction

I have a dataset that contains weekly sales for stores and categories. It looks like this: I would like to apply gradient boosting method to predict weekly sales. My question is: Should I create ...
3
votes
2answers
45 views

Selecting ML algorithm for music composition

I'm a composer and programmer. I'd like to use ML for composing music. There's already research on the general problem of composing by machine in known musical styles. I'm more interested in the ...
2
votes
1answer
24 views

Individual differences in response time (RT) experiment - searching for the right test

Given a distribution of response times to different categorical variables, what's the best way to test for individual differences? or more specifically: There are 100 people pressing buttons in 10 ...
0
votes
1answer
53 views

What is the input space of a neural network (or other supervised learning algorithms)?

While training the neural network (or any other supervised learning algorithms), we supply input variables and corresponding outputs. The input variables can be continuous or discrete (binary in many ...
-1
votes
1answer
104 views

Encoding categorical data 2 different columns

Suppose I have two columns namely Goods, and Quality which are to be one encoded. Goods                   &...
1
vote
2answers
57 views

Is there a quick way to check for multicollinearity between categorical variables in R?

I have a large amount of categorical and dummy variables (36) and I would like to remove a number of them based on their multicollinearity (or just collinearity). Instead of using Chi Square tests ...
1
vote
1answer
35 views

Logistic regression if 3 categories in outcome variable

Logistic regression is generally performed if there are 2 categories in outcome variables. I just tried it for iris dataset with species as y variable which has 3 categories. I used following code: <...
0
votes
0answers
47 views

how to properly organize ordinal categorical data

Let's assume you have ordinal categorical data example: ['small,large,medium ] If you use Label encoding, you will not get output based on of it's value, but just ...
1
vote
1answer
361 views

customer segmentation with categorical variables

I was adviced to write in this group regarding my question about modeling categorical database. I have a customer dataset, which is a survey result. I have 1595 obs. and about 200 columns(200 because ...
3
votes
0answers
281 views

How to combine categorical and continuous input features for neural network training

Suppose we have two kinds of input features, categorical and continuous. The categorical data may be represented as one-hot code A, while the continuous data is just a vector B in N-dimension space. ...
2
votes
0answers
98 views

How to continue incremental learning when a categorical variable has been assigned additional category labels?

Please help answer this question or point me to any resource. There is a model in an environment where training happens with new data and the data is discarded after training is completed. This keeps ...
1
vote
1answer
26 views

How can I make a prediction in a regression model if a category has not been observed already?

I'm researching a regression model to predict a target value that has four features, all of which are categorical. The categories are not fixed, e.g. one is a customer identifier. How could my model ...
0
votes
2answers
98 views

Data binning - Why we need to transform Categorical Variables?

Having a lot of categorical features and other numerics why we need to transform the categorical to binary values? Is it for using the values in mathematics functions of the algorithms? Thanks!
0
votes
0answers
338 views

preprocessing data: mixing categorical, numerical and ordinal data?

I want to apply different kind of ML techniques to user data using scikitlearn. Let's assume that one user got the following ordinal data: ...