I have currently fine tuned the BERT model on some custom data and I want to conduct some more experiments to increase the accuracy.
My original dataset consists of a pair of sentences (like MRPC dataset). I want to increase the accuracy of classification by adding some numerical features (which I will separately calculate). I wanted to know if I could train bert using this after I have already fine tuned it ?
I have read about some solutions people have proposed in the past like : 'Extracting word embeddings (extract_features.py on Bert GITHUB) and combining that with my custom data to feed a single layer CNN network.' I dont want to lose out on the accuracy the BERT network is providing me by only extracting features from the pretrained model.
So is there any way I can create a kind of hybrid model which first fine tunes BERT and then I add my features and it feeds into another model for an improved classification ?
As I am new to tensorflow and Deep learning, please let me know if there is something fundamentally wrong in my understanding. Thank you for your help