The length of human_vocab
is 18377.
The length of input X
is 1000
I'm trying to run to_categorical
np.array(list(map(lambda x: to_categorical(x, num_classes=len(human_vocab)), X)))
Is this the same if i apply:
onehot_encoder = OneHotEncoder(sparse=False)
onehot_encoder.fit_transform(X)
the output of onehot_encode is (1000, 9739)
shouldn't it be (1000,18377)
?
human_vocab
contains the corpus data that X doesnt contain
I'm trying to find the replacement of to_categorical
due to memory issues to create hot vector