Questions tagged [encoding]

Encoding in machine learning and data science refers to the process by which non-numeric data is transformed into a numeric representation that can be fed into machine learning algorithms.

Filter by
Sorted by
Tagged with
1
vote
0answers
5 views

quereies related to autoencoder

i want to design an deep auto encoder after following keras tutorial. Input is a simple 2-dimensional image consists of 512 rows and 50 columns matrix My trial code is ...
0
votes
0answers
9 views

Expected input for time-series transformers

I am trying to create a model starting from the Attention is all you need paper. Specifically I want to setup the Encoder/Decoder architecture to predict time-series. I would like to implement it in ...
1
vote
3answers
30 views

One-hot-encoded variables dominating clustering

I am performing some unsupervised clustering with k-means on some transaction data that contains the following information: Customer units purchased in category_1 units purchased in category_1 time ...
0
votes
0answers
19 views

Encoding categories of items purchased for customer segmentation (clustering)

AIM: I am trying to deveop a model that will allow me to understand my customer better by clustering their purchase behaviour. CONTEXT: I have transaction level data that tells me, for a given ...
1
vote
1answer
11 views

Encoding distance variable that is continuous until out-of-range

I have a varaible distance which is continous until a "hard stop" at which we stop measuring the distance itself and just label the distance as "out ...
0
votes
1answer
32 views

Encoding with OrdinalEncoder: TypeError: unhashable type: 'numpy.ndarray'

I'm trying to do a Random Forest in a data-set with numerical and categorical variables in order to obtain a categorical result (two possible classes, column name "predicción"). I'm using ...
0
votes
2answers
23 views

Encoding feature containing both text and string?

I have a feature which has following entries:- | Exterior | | -------- | | Vinyl | | Wd Sdng | | MetalSd | | Wd Sdng | | HdBoard | | BrkFace | | Wd Sdng | ...
1
vote
3answers
61 views

Does One-Hot encoding increase the dimensionality and sparsity of dataset?

There are two ways to convert object datatype into numeric datatype, first is One-Hot encoding and second is simply map the numerical tags to different values. For example for column Age containing ...
1
vote
1answer
27 views

Encoding "histogram bins"

I am currently working on a regression problem where I have one variable (x) of the data in the form of "histogram bins". I.e. I could have value ranges ...
0
votes
0answers
7 views

Best Approach to Encode/Categorize Unrelated Data

I have a data set with multiple columns that I'd like to encode for a model. The data set is as follows. Transaction CPT Code Modifier1 Modifier2 Modifier3 Price 00001 99287 LT 81 50.00 00002 ...
0
votes
1answer
15 views

Encoding when different number of records for each month-One hot or different type?

I am working on a dataset which is pretty small: 1169 records. There is a column called month and it takes the values 'Jan','Feb', or 'March'. The number of records for each month is different. I have ...
0
votes
0answers
24 views

Encoding technique used in Keras ImageDataGenrator class

I would like to know what encoding is used by ImageDataGenerator for encoding the class labels. I have done a lot of research and found that there is a variable called ...
6
votes
3answers
1k views

What is the difference between one-hot and dummy encoding?

I am trying to understand The reason behind encoding (one-hot encoding and dummy encoding) How one-hot and dummy are different from each other
0
votes
0answers
12 views

Dataset that's already encoded, mixed types, methods for EDA and clustering?

I have a dataset that is already encoded (so it looks like table 2) with numbers (ie. each state & flavor have an assigned number, likeliness to try new flavor is on a scale 1-5, etc.) I would ...
0
votes
0answers
10 views

Term document matrix for webpages

Consider the following code for obtaining term-document matrix for given texts ...
0
votes
0answers
9 views

How to apply Helmert Coding in a real Machine Learning model?

My dataset is something like this ...
0
votes
1answer
31 views

Preprocess multi-sample time series data: encode each sample separately or in aggregate?

Let's say I have 3 dense sequences of uniform length. Should I fit a scaler on them separately or together? ...
0
votes
2answers
64 views

How to encode a sentence using an attention mechanism?

Recently, I read about one of the state-of-the-art method called Attention models. This method use a Encoder-Decoder model. It can find a better encoding for each word in a sentence. But how can I ...
2
votes
2answers
81 views

How to handle non ordinal Features like Gender,Language,Region etc? Ordinal Encoding or one-hot encoding?

I see that usually, while preparing the dataset. Usually, data scientists convert non-ordinal features like Gender or Language in a dataset using LabelEncoder/ordinalEncoder. Ideally, they should have ...
1
vote
0answers
25 views

When doing feature selection, are features like "year", "month" considered as ordinal features or should I convert them to strings?

I am working on a hotel reservation dataset that has both categorical and numerical (continuous and discrete) features (26 columns, 30244 rows). Target is also categorical and it says if the user &...
1
vote
1answer
451 views

Should I do one hot encoding before feature selection and how should I perform feature selection on a dataset with both categorical and numerical data

a newbie here. I am currently self-learning data science. I am working on a dataset that has both categorical and numerical (continuous and discrete) features (26 columns, 30244 rows). Target is ...
0
votes
1answer
24 views

Handling categorical data with more over 100 unique classes

I am working with a pure categorical data set. And some classes have more than 100 unique values. I could not find any appropriate encoding possibility. So I created a SQL table, where each value got ...
0
votes
1answer
15 views

Do I need to encode samples during inference?

I recently started saving (pickling) my fitted encoders. The thinking was that I would need them to encode previously unseen samples during inference. Encode training features and labels. Train model ...
0
votes
3answers
167 views

encoding datetime features .. label or onehot?

I am working on a cab booking prediction problem where I need to use datetime aspects like hour,day ,week etc for prediction. As I need to do categorical encoding for the purpose. can anyone help me ...
2
votes
2answers
55 views

Is it acceptable to use label encoding for nominal categorical data when one hot encoding would create too many features?

I'm working on a short data science project to compare the accuracy of different classification methods. The groups decided to use and compare Random Forest, Naive Bayes and SVM. The dataset we are ...
0
votes
1answer
23 views

How to do encode this target vector containing strings

Consider a target vector like ...
1
vote
2answers
41 views

Linear Regression with Category variables

I'm currently learning and exploring machine learning and understand the basics of linear regression based on two numerical variables, but now I wish to go a little further and need some guidance ...
0
votes
1answer
30 views

Encode the days of week as numeric variable

I would like to understand if there is the possibility to encode the days of the week as a single numerical column to preserve the ordinal relationship between the days. My task is a classification ...
0
votes
0answers
22 views

Binary classification or Single-class classification for data with Boolean label

In supervised learning. For simple prediction/classification problems like Will it rain tomorrow? or more serious one like disease diagnosis I often encountering ...
2
votes
1answer
149 views

Multi-Feature One-Hot-Encoder with varying amount of feature instances

Let's assume we have data instances like this: ...
1
vote
0answers
24 views

Modeling time spent on different websites

Apologies in advance for the poor title. I didn't know exactly how to phrase what I want in a succinct manner so hopefully I can elaborate a bit more. I have a dataset where I have customer_id, ...
0
votes
0answers
7 views

Factors in choosing a continuous encoder?

Apart from heuristically treating an encoder like a hyperparameter, how can one decide which encoder to use for continuous features? Rule out encoders that do/ don't accept negative values. Can we ...
0
votes
1answer
23 views

Cleaning NaNs with averages pre or post split? [duplicate]

I have a column with some NaNs in it and I want to replace those NaNs with the average/median/mode. Technically, the validation/ test data has never been seen before - so how could I include it in the ...
7
votes
5answers
901 views

How do I encode the categorical columns if there are more than 15 unique values?

I'm trying to use this data to make a data analysis report using regression. Since regression only allows for numerical types, I then need to encode the categorical data. However, most of these have ...
2
votes
2answers
378 views

Choose-many categorical features: alternatives to one-hot encoding?

I'm building a model to predict the lifetime value of a client based on the relational data we have on them. The user table has a bunch of one-to-many child tables that might be predictive. Grossly ...
0
votes
1answer
32 views

How would I approach training a model and encoding this categorical data

So I have the following data: I have one series where each word has a value that describes the average review score that would get. For example, if the word "excellent" showed up in reviews ...
1
vote
1answer
26 views

Encoding Tags for Random Forest

I have the following data set: I want to use attributes Tags and Authors to classify each record into their respective Rating. In order to do so I want to use a random forest classifier. My concern ...
0
votes
1answer
28 views

Is label-encoded data quantitative or qualitative?

If you label-encode something which is qualitative, like brand of toothpaste or colour of hair, would you describe the resulting data as quantitative since it is now expressed in numbers? Or would you ...
0
votes
1answer
31 views

Handling encoding of a dataset which has more than total 2000 columns

Whenever we have a dataset to be pre processed, before feeding it to the model we convert the categorical values to numerical values for which we generally use LabelEncoding, One Hot encoding etc ...
1
vote
0answers
28 views

How does the R implementation of RandomForest split nodes on categorical data?

The R implementation of RandomForest can take in categorical features as factors and train and predict on these features without encoding. Normally, I use the python implementation from scikit-learn ...
1
vote
2answers
104 views

Is there a RandomForest implementation that handles categorical data without encoding in python?

I am working on a binary classification project with both continuous and categorical features. I know that the R implementation of RandomForest can handle categorical data passed in as factor type ...
0
votes
1answer
48 views

How to encode high cardinality categorical data?

I have a dataset of 1600 rows and 28 columns. Only one column is partially complete with 1300 records. The rest is NaN. I did a value count of this columns and it has 84 different categories that are ...
1
vote
2answers
520 views

is it better to correlate and encode or encode and correlate?

I have one doubt like is it better to perform label encoding and check for the correlation or should I 1st perform correlation and do label encoding? Because when I tried it both ways I'm getting ...
1
vote
1answer
216 views

What is multi-hot encoding?

I was read and paper for machine learning, and i found this term "multi-hot encoding" without explanation. Can you help me please? the paper: https://arxiv.org/abs/2001.06917
0
votes
2answers
842 views

Encoding categorical data with pre-determined dictionary

in case feature encoding, if I'd like to encode my values based on my pre-determined dictionary, how do I do that? For instance, say, I've values as ...
0
votes
1answer
38 views

Variables for SVM [closed]

I would like to predict if an email is spam or not spam based on the information that I have, i.e. date, email address, subject and text. Three of these parameters are text data, so they would need to ...
0
votes
1answer
28 views

How keras.layers.embedding learn word embeddings?

I was trying some tensorflow tutorials and see that in all of them they use layers.embedding to learn these word embeddings, but how are these learned? , with a NN? which arquitecture? , or word2vec? ...
2
votes
1answer
97 views

Explanation about i//2 in positional encoding in tensorflow tutorial about transformers

I was implementing the transformer architecture in tensorflow. I was following the tutorial : https://www.tensorflow.org/tutorials/text/transformer#setup_input_pipeline They implement the positional ...
1
vote
1answer
52 views

How to encode ordinal data before applying linear regression in STATA?

I have a data set that has student performance marks (continuous and dependent variable), Teacher Qualification (Ordinal and independent variable containing categories: Masters, Bachelors, High School)...
3
votes
1answer
185 views

On gradient boosting and types of encodings

I am having a look at this material and I have found the following statement: For this class of models [Gradient Boosting Machine algorithms] [...] it is both safe and significantly more ...