Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [categorical-data]

Categorical data can take on a limited (usually fixed) number of possible values called categories. Categorical values "label", they do not "measure". Nominal and dichotomous/binary scale types are categorical. Some people consider ordinal scale categorical too.

0
votes
1answer
17 views

Regression in Python with many NaN values spread across all columns

I want to do a regression to predict "value" based on the other columns from below example table. The data was collected by single indicator and not across all data points, resulting in many NaN/blank ...
1
vote
1answer
13 views

Anomaly detection in nominal big data

I have to apply an anomaly detection algorithm on big data, the values of each column on my dataframe are nominal and vary over 10000 times, the algorithms I've found only accept numeric values, is ...
1
vote
0answers
22 views

Input explanatory categorical variables along with time series into neural network

I want an advise on the ways to enter time series along with additional variables into convolutional neural network. Story first: I have a dataset of time series with daily energy consumption data (...
-1
votes
1answer
32 views

Handling Categorical Data and Numerical Data Together [on hold]

I have a Dataframe containing 3 features: Product_detail, S.I_Units and Value. ...
0
votes
0answers
10 views

categorical features encoding strategy

I have a dataset with all features in categorical(text or numeric). What is the most efficient way to encode categorical data to numeric such that information is not lost. I want to perform feature ...
0
votes
0answers
14 views

How to feed my categorical data into my model?

This may be a basic question, I have a categorical data and I want to feed this into my machine learning model. my ML model accepts only numerical data. What is the correct way to convert this ...
0
votes
0answers
8 views

Impute categorical data of train and test set

I have a big data-set compound mainly of two parts numeric and categorical (10 numeric, 11 categorical). Try to apply a clever technic to impute missings, I use Impute library which is capable of ...
0
votes
0answers
13 views

How to group text categorical data basis on clustering?

So if I have text dataset where I have more than 50 categories. Sample: ...
0
votes
1answer
10 views

Algorithm for purely categorical data

Looking for an algorithm to deal with purely categorical data. It was suggested to me to look into the K-medoids algorithm. Anyone know if there is a K-medoids algorithm R library(package)?
1
vote
1answer
23 views

Loss is bad, but accuracy increases?

I have a multicategorial classification problem for images. There are 5 (imbalanced) classes for which i use different class weights. In general there are only a few training images per class: ~56-238 ...
0
votes
0answers
12 views

One-Hot Encoder and nested categorical features

Suppose we have a dataset with $n$ features $A_1,\cdots,A_n$ that are categorical and are nested. By nested, I mean if you know the categorical value of $A_m$ for observation $x$, then the value for $...
0
votes
1answer
6 views

Entropy loss from collapsing/merging two categories

Suppose I am counting occurrences in a sequence. For a classical example, let's say I'm counting how many of each kind of car comes down a highway. After keeping tally for a while, I see there are ...
0
votes
1answer
26 views

What is the the cost of combining categorical variables?

I have 2 categorical variables e.g. state and city. Missing are only in city. As opposed to throwing out all observations with missing values for city or throwing out city all together I was ...
0
votes
1answer
23 views

PANDAS Within Category Normalization

I'm want to normalize sales data of multiple point of sales (POS), Products and weeks. The dataframe looks like this: ...
0
votes
0answers
21 views

“Histogram and binning” technique for categorical variables publication and implementations

H2O.ai have implemented the "histogram and binning" technique for efficient and accurate tree-building using categorical variables of high cardinality (>100): http://docs.h2o.ai/h2o/latest-stable/h2o-...
0
votes
0answers
15 views

How to deal with ordered set of values with different ranges

I have data to train a model on, a mix of continuous and categorical values, one column is labeled with values like "[0,10], [10,100], [100,500], [500,]". those are ...
0
votes
0answers
13 views

Can I use MANCOVA with categorical data?

By doing an experiment, I would like to analyze the effect of 3 categorical independent variables (with 2 levels each) on three constructs measured with ordinal variables. I also have 2 covariates (...
2
votes
0answers
25 views

“Binary Encoding” in “Decision Tree” / “Random Forest” Algorithms

Is it OK to use Binary Encoding in a dataset containing categorical columns with very high cardinalities? Some facts about my dataset: My dataset has ~170000 rows One of the categoric variables has ...
1
vote
0answers
15 views

Suggestions on using model in production 1 test at a time

I have created an Artificial Neural Network with 4 categorical features and a binary outcome either 1 for suspicious or 0 for non-suspicious: ...
1
vote
1answer
37 views

Prediction with unseen values in categorical variables

I have created an Artificial Neural Network with 4 features. I am at the point where I want to test the model with a live sample of a malicious file path/exe using: ...
0
votes
1answer
43 views

How to deal with missing data for only some categories

Or in other words, data for category A is irrelevant for category B. So it is not present, how can imputing missing data distort/effect learning models broadly. I can't find any logic how to deal ...
0
votes
0answers
10 views

Response variable is nominal.

I have a dataset that has a nominal response variable with about 10 classes. Now I want to train a classification model (such as random forest or XGBOOST). I separated the data into X and y. Now, y is ...
0
votes
0answers
16 views

Must label encoding for categorical variable be sequential?

I came across label-encoding for a categorical variable that encodes {X, Y, Z} to {1, 3, 15}. By default, scikit-learn returns {0, 1, 2}. For logistic regression, would encoding a categorical variable ...
0
votes
0answers
22 views

Can I use MCA on categorical features, and PCA on numeric then combine both for learning

So all is said in the title. I have a mix of both categorical and numeric features, both are more than 20 columns and reside in the same data-set. I am using PCA solution from sklearn.decomposition ...
3
votes
2answers
35 views

Dummy variable for Categorical values

The question is in reference to solution of Titanic survival predictionat kaggle . As many have did the similar kind of feature extraction, They have converted some of the numerical features (Age, ...
7
votes
3answers
224 views

How can I do classification with categorical data which is not fixed?

I have a classification problem with both categorical and numerical data. The problem I'm facing is that my categorical data is not fixed, that means that the new candidate whose label I want to ...
2
votes
1answer
37 views

Selecting the right time series model [closed]

Using Python, I am trying to predict the future sales count of a product, using historical sales data. I am also trying to predict these counts for various groups of products. For example, my columns ...
1
vote
0answers
247 views

How to plot a heatmap-like plot for categorical features?

I would greatly appreciate let me know how to plot a heatmap-like plot for categorical features? In fact, based on this post, the association between categorical ...
1
vote
1answer
658 views

How to implement feature selection for categorical variables (especially with many categories)?

I've been trying to get some ideas of how I could treat categorical variables when doing feature selection. Mainly I've been running Random Forest feature importance on Python for which preprocessing ...
0
votes
1answer
37 views

Orange imports numeric features as categorical (“file” widget)

Why are some of my numeric features not being recognized as 'numeric' types AND why can't I reclassify them? I can't share my CSV here but I can assure you those features are indeed numeric (I use ...
1
vote
1answer
118 views

tensorflow categorical data with vocabulary list - Expected binary or Unicode string, got [0,1,2,…]

I'm brand new to machine learning (having just completed the google machine learning crash course) and thought it would be good to try my hand at a Kaggle competition as a good starter to some real ...
1
vote
0answers
10 views

How do i create the same categories across two or more variables(columns) when converting integers to Factors in R?

I did merge the columns, arranged them as rows and then converted them to categories. I just wanted to know if there was a better way in doing this.
1
vote
1answer
41 views

Best practices for selecting categorical features

I'm trying to create a classifier that will predict whether someone will attend an interview or not. Each data point is for a single candidate and contains details such as the location of the ...
0
votes
1answer
62 views
1
vote
3answers
72 views

What is the best way to visualize the relationship two categorical variables

I am currently working on an ambulance dataset and one of my tasks is to find when a patient was misdiagnosed by the call dispatcher. I have two codes; a dispatch code(what the dispatcher believes is ...
0
votes
1answer
58 views

Anomaly detection using clustering of highly correlated Categorical data

My data has two columns and both are highly correlated e.g. if column1 has value ABC, column2 should be XYZ i.e. ABC-->XYZ. If column2 has anything else its Anomaly. Likewise there are thousands of ...
0
votes
1answer
193 views

Categorical data for sklearns Isolation Forrest

I'm trying to do anomaly detection with Isolation Forests (IF) in sklearn. Except for the fact that it is a great method of anomaly detection, I also want to use it because about half of my features ...
0
votes
1answer
75 views

250 Categorical values

I have a dataset which has only categorical values. As I came across a few articles people suggested that KNN / Random forest would work for dataset like this. Though in R it couldn't handle as if ...
0
votes
1answer
67 views

Convert nominal to numeric variables?

I am trying to develeop an algorithm with sklearn and Tensorflow to predict which car can be offer to each customer. To do that I have a database with the answers of one survey to 1000 customers. An ...
0
votes
0answers
19 views

Is there any way to collect categorical features quickly in Julia DataFrames?

I'm using Julia 0.6.3 with Dataframes.jl I was wondering if there was any way to get categorial features easily in Julia? For large datasets it can be impossible to enter everything by hand. My ...
2
votes
1answer
42 views

Same predictors in test set but I want different outputs

I have a (training) dataset about what TV spectators are watching and for how long. The goal (at new set - the test set) is to predict for how long the TV spectators will watch a specific channel and ...
1
vote
0answers
87 views

What is the relationship between correlation ratio and one-way Anova?

According to the answer to this post, it is recommended to use one-way anova to compute the dependence between a categorical and a numerical variable. Besides, the second answer to this post says ...
0
votes
0answers
11 views

Which method can I use to explore dependences between categorial(nominative) variables?

There is some sociological research data. For example: How can I explore some sort of correlation between income level and district, or which statistical measures based on this data can I calculate? ...
0
votes
1answer
31 views

What are appropriate labels for age categorical labels?

I am converting some age data to categorical variables. What are some appropriate labels? Some people might take offense to using "Young", "Old" or "Millenial", etc. Is there a "standard" list of ...
0
votes
0answers
84 views

Dropping one of the one-hot encoded columns for Gradient Boost Methods/Decision Trees?

If I have the categorical variable like favorite_color and it has unique values red, green, ...
0
votes
0answers
64 views

Clustering of devices in locations?

My question is about using some sort of AI to assess if devices are located in any of a list of venues. I'd ask of machine learning, but so far we're doing this with an expert system, and we are ...
0
votes
1answer
111 views

faster alternatives to sparse.model.matrix?

I have a large dataset that is entirely categorical. I'm trying to train with it using xgboost, so I must first convert this categorical data to numerical. So far I've been using sparse.model.matrix() ...
2
votes
2answers
507 views

One hot encoding large dataset

Initially, I have a dataset where for each row there is user_id and product_ids he bought. In that dataset there are 478191 products bought by different users. In order to discover frequent items ...
2
votes
1answer
311 views

Data scaling before PCA: how to deal with categorical values?

I have to apply PCA on a dataset, which contains both numerical and categorical values. In the preprocessing phase, I converted all the categorical values in numerical, so that the software can deal ...