0
$\begingroup$

I have a feature which has following entries:-

| Exterior |
| -------- |
| Vinyl    |
| Wd Sdng  |
| MetalSd  |
| Wd Sdng  |
| HdBoard  |
| BrkFace  |
| Wd Sdng  | 

and so on.

I am assuming that Wd Sdng is a text value and other values are string (but please correct me if I am wrong).

How do I encode this feature since it has both text and string values?

Should I perform a OneHotEncoding or should I perform some kind of NLP encoding (Tfidf etc)??

$\endgroup$
2
  • $\begingroup$ Is it possible to clarify the question better with some real data points? $\endgroup$
    – jdsurya
    Nov 2 '21 at 6:00
  • $\begingroup$ @jdsuryap edited. Basically my question is, since I have both text (Wd Sdng) and string (vinyl, MetalSd and others) values, how do I encode this feature? $\endgroup$
    – spectre
    Nov 2 '21 at 7:13
0
$\begingroup$

so sorry, but you are confusing peeps with "text" vs "string" ..looking at your examples, they are all "strings" and in this case using any pre trained embedding (glove, word2vec ) wont work very well since most of your strings will come up with 0 while looking up the global embeddings.

One hot will have to be done at character level (meaning at least a 26 dimension vector with 1 non zero) BUT this will be super sparse so, u ll need to take care of that as well ..the best bet is , if you had enough samples, train a local glove embedding (there are scripts available on their git page)

$\endgroup$
3
  • $\begingroup$ But isn't Wd Sdng a text value as it comprises of 2 strings? $\endgroup$
    – spectre
    Nov 2 '21 at 17:17
  • $\begingroup$ nope ..basically your datatype definition depends on what you want to do with the data..in this case you want to process all of them together right ? treat everything as string and try any embedding / encoding technique..dont get lost in semantics :) ..good luck $\endgroup$ Nov 3 '21 at 4:36
  • $\begingroup$ I want to perform a regression task on this data. So I should treat Wd Sdng as a string and perform One hot encoding? $\endgroup$
    – spectre
    Nov 3 '21 at 5:03
0
$\begingroup$

After hours of searching online, I think I finally got the answer. The observation Wd Sdng is not a text type. It is considered a string type (don't know why!) and to encode it, the best thing would be to join both the words, as in WdSdng and then apply usual encoding techniques (not text encoding!).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.