0
$\begingroup$

So I have the following data:
I have one series where each word has a value that describes the average review score that would get.
For example, if the word "excellent" showed up in reviews with a score of 2,3,5,4 it would gain a value of 3.5.

I also have a list of the words contained in a review, and the review scores of each of those written reviews.
For example,

Unique_words
["good","clean","hotel","enjoyed","stay","here"]

score
4

(These are ofc simplified examples, my actual data is a lot longer)

I also have the original reviews, from which the unique_words are taken from.

The question is, how would I use this data in order to train a machine-learning algorithm to predict what score a review would get, given the unique words contained inside it.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

You need to encode categorical variables as dummies.

This means to create new features for each type of category and then assigned either a 1 (where a record has that category) or 0 (to each record that doesn't have that category). With some examples, this should look something like this

                        good  clean  hotel  enjoyed  stay  here
I am good               1     0      0      0        0     0
My face is clean        0     1      0      0        0     0
This is a clean hotel   0     1      1      0        0     0
I enjoyed a good meal   1     0      0      1        0     0
Don't stay here         0     0      0      0        1     0

In python you can use the pandas function pd.get_dummies()

Improve this answer
$\endgroup$
2
  • $\begingroup$ Yeah, I was hoping there would be a way to avoid this. Since I've got more than 50k unique words that I've assigned the value to, creating a new feature for each word makes the DataFrame really big and slow to run through. Also side question; is there a way (or even a benefit) to keep count of how many times a certain word shows up in the review. One could assume that a review that has the word bad five times is lower than one that mentions it once. $\endgroup$
    – SirAchesis
    Commented Nov 18, 2020 at 6:18
  • $\begingroup$ I don't know of any other way but make sure you are using sparse matrices. Also, take a look at tfidf which weights things based on how often they occur. $\endgroup$
    – Taylrl
    Commented Nov 18, 2020 at 17:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.