I have dataset of 100 000 words labeled by surname(is last name / not last name) Example:

kitchen | 0

kennedy | 1


I tried extract lenth of word, count of each letter and such simple features to build random forest classifier, but it didn't work. How to solve this task?

  • $\begingroup$ Please specify what you are trying to classify (what does 0 and 1 denotes?). Feature engineering is a task that requires domain expertise. $\endgroup$ – Yohanes Alfredo Nov 18 '19 at 8:36
  • $\begingroup$ 1 denote that the word is last name (Smith, Johnson, Williams, Jones, Brown etc.) 0 denote that the word is not last name (card, phone, computer, airplane etc.) $\endgroup$ – mender1 Nov 18 '19 at 8:47
  • $\begingroup$ Do you really need to use randomforest? for this case you will lose the sense (tech and chet is essentially similar using feature generated like that) of each of your sample and hence poor performance. $\endgroup$ – Yohanes Alfredo Nov 18 '19 at 9:07
  • $\begingroup$ I have no idea how to generate feature another way, becouse I have no context for the words. But I need to solve this using ML $\endgroup$ – mender1 Nov 18 '19 at 9:14
  • $\begingroup$ You can use RNN, if that is the case. Simply use RNN and use letters as your input. That is the simplest working approach I can think of. $\endgroup$ – Yohanes Alfredo Nov 18 '19 at 9:16
from keras.preprocessing.text import Tokenizer
from keras.preprocessing.sequence import pad_sequences
from keras.models import Sequential
from keras.layers import LSTM, Embedding, GlobalAveragePooling1D, Dense
from keras import backend as K
import numpy as np

words = ['kennedy','smith','kitchen','caterpillar']
label = [1,1,0,0]

tokens = Tokenizer(30, char_level=True)

X = tokens.texts_to_sequences(words)
X = pad_sequences(X)

model = Sequential()
model.add(Embedding(30, 100, mask_zero=True, input_shape=(None,)))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam')


So this is a very simple implementation built with keras. So the model is very simple it is only to turn sequence of letters into tokens (e.g. transform A->1, B->3, etc. This is automatically handled by keras tokenizer), put embedding on top of these tokens to get your text as sequence of vectors and then feed the sequence of input with LSTM, and simply apply classification on top of the final LSTM output.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.