I am trying to implement BERT using HuggingFace - transformers implementation.

I am following two links: by analytics-vidhya and by HuggingFace

If we consider inputs for both the implementations: 1) by analytics-vidhya They have used three inputs here:

token_inputs = Input((MAX_SEQUENCE_LENGTH), dtype=tf.int32, name='input_word_ids')
mask_inputs = Input((MAX_SEQUENCE_LENGTH,), dtype=tf.int32, name='input_masks')
seg_inputs = Input((MAX_SEQUENCE_LENGTH,), dtype=tf.int32, name='input_segments')

where as in by HuggingFace , input has not been divided for ids, mask, and segments.

2) In by analytics-vidhya, they have created own tokenizer , can we use inbuild tokenizer glue_convert_examples_to_features (as in here)

3) fit method can have both argument;x and y, in one object ? like this

bert_history = bert_model.fit(bert_train_dataset, epochs=3, validation_data=bert_validation_dataset)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.