Stack Exchange Network
Stack Exchange network consists of 181 Q&A communities including
Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.
Visit Stack Exchange
Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
Anybody can ask a question
The best answers are voted up and rise to the top
2 years, 11 months ago
I have a csv with both categorical and float dtypes. I want to do the following:
For each categorical column i will use pandas to compute the unique values (
pd.unique()) that are present in the column. say
u_l for a column
I will use the
len(u_l) to decide upon the dimension of embeddings that i use for a particular categorical column that i want i embed (this step is the reason i cannot use tensorflow_transform)
I want to create some stateful node that can map
category (token) value to embeddings index thus subsequently i can lookup the embedding from embeddings matrix that i created in step 2
I dont know how to go about doing it currently. A very inelegant solution i can see is using tensorflow_datasets:
encoder = tfds.features.text.TokenTextEncoder(u_l,decode_token_separator=' ')
concatenate the entire column using space delimiter (
c_l is one string now) and then using
This is a very basic thing that i think tensorflow would be able to do relatively easily. Please guide me to the right solution
Apr 26, 2020 at 19:21
775 1 1 gold badge 4 4 silver badges 13 13 bronze badges
Embedding() layers take sequences as inputs. If there is no time dependency in your data (such as for time series datasets or language corpora) then those layers cannot be used. What kind of data do you have?
Alternatively, you can use
dimensionality reduction, one-hot encoding or, in some cases, label encoding. I personally suggest dimensionality reduction whenever possible, even if it's the most computationally intensive choice.
Apr 26, 2020 at 21:16
5,897 2 2 gold badges 12 12 silver badges 50 50 bronze badges
By clicking “Accept all cookies”, you agree Stack Exchange can store cookies on your device and disclose information in accordance with our
Accept all cookies
Necessary cookies only