I am new to neural network. I'm trying to train word embeddings without using word2vec package.
Using titles from reddit worldnews dataset I'm have done some CBOW representation.
For window size three here is some of my outputs:
0 Context : ['scores', 'killed', 'pakistan'] ---> Target: clashes 1 Context : ['japan', 'resumes', 'refuelling'] ---> Target: mission 2 Context : ['us', 'presses', 'egypt'] ---> Target: gaza 3 Context : ['presses', 'egypt', 'gaza'] ---> Target: border
For Vocab size = 513, I've collected 369 of target words against 369 of 3-grams context words. Each context word is one-hot codded of length 513.
Therefore, my dataset length becomes:
X.shape = (369, 3, 1, 513) Y.shape = (369, 1, 513)
Now I'm having trouble in fitting the data in neural network. My Neural Network model is constructed with keras.
# create model model = Sequential() model.add(Dense(100, input_dim=1, init=369 'uniform' , activation= 'sigmoid' )) model.add(Dense(1, init= 'uniform' , activation= 'sigmoid' )) model.compile(loss= 'binary_crossentropy' , optimizer= 'sgd' , metrics=['accuracy']) #train history = model.fit(X, Y, nb_epoch=100)
ValueError: Error when checking input: expected dense_9_input to have 2 dimensions, but got array with shape (369, 3, 1, 513)