I have a problem of multi class classification and I'm using a simple 2-Layer Bi-directional LSTM with keras.
The model in a simple form:
Bidirectional LSTM (64) Bidirectional LSTM (64) Dense (128) Activation Sigmoid Dense (14) Activation Softmax
I have a raw and skewed dataset so I'm doing all the pre-processing myself to balance it.
- Firstly, I replicated the original data and produced 3*OriginalData to produce more examples of certain classes that weren't represented.
- Then I performed classification and the loss and accuracy history are depicted below for both training and validation sets
Accuracy built up after 25 epochs, but loss had major variations, so I decided to replicate more of my data so I produced 10*OriginalData, and performed classification again.
Now my loss and accuracy behaved so much worse.
My question is whether the problem lies in my model, or at the replication of data. Is it possible for a model to work well with less data but not so good when it has so many? Or maybe I'm causing it to overfit by providing over-replicated data?
Note: By replicating data, I don't mean duplicating, but because I work with audio data, I play with different pitch shifts.