Encoding feature containing both text and string?

I have a feature which has following entries:-

| Exterior |
| -------- |
| Vinyl    |
| Wd Sdng  |
| MetalSd  |
| Wd Sdng  |
| HdBoard  |
| BrkFace  |
| Wd Sdng  |


and so on.

I am assuming that Wd Sdng is a text value and other values are string (but please correct me if I am wrong).

How do I encode this feature since it has both text and string values?

Should I perform a OneHotEncoding or should I perform some kind of NLP encoding (Tfidf etc)??

• Is it possible to clarify the question better with some real data points? Nov 2 '21 at 6:00
• @jdsuryap edited. Basically my question is, since I have both text (Wd Sdng) and string (vinyl, MetalSd and others) values, how do I encode this feature? Nov 2 '21 at 7:13

2 Answers

so sorry, but you are confusing peeps with "text" vs "string" ..looking at your examples, they are all "strings" and in this case using any pre trained embedding (glove, word2vec ) wont work very well since most of your strings will come up with 0 while looking up the global embeddings.

One hot will have to be done at character level (meaning at least a 26 dimension vector with 1 non zero) BUT this will be super sparse so, u ll need to take care of that as well ..the best bet is , if you had enough samples, train a local glove embedding (there are scripts available on their git page)

• But isn't Wd Sdng a text value as it comprises of 2 strings? Nov 2 '21 at 17:17
• nope ..basically your datatype definition depends on what you want to do with the data..in this case you want to process all of them together right ? treat everything as string and try any embedding / encoding technique..dont get lost in semantics :) ..good luck Nov 3 '21 at 4:36
• I want to perform a regression task on this data. So I should treat Wd Sdng as a string and perform One hot encoding? Nov 3 '21 at 5:03

After hours of searching online, I think I finally got the answer. The observation Wd Sdng is not a text type. It is considered a string type (don't know why!) and to encode it, the best thing would be to join both the words, as in WdSdng and then apply usual encoding techniques (not text encoding!).