2
$\begingroup$

I installed bert-as-service (bert-as-service github repo) and tried encoding some sentences in Japanese on the multi_cased_L-12_H-768_A-12 model. It seems to work as I am getting vectors of length 768 per word but np.shape() shows this for each sentence:

np.shape(vec_j[0]): (25, 768)
np.shape(vec_j[1]): (25, 768)
np.shape(vec_j[2]): (25, 768)
np.shape(vec_j[3]): (25, 768)
np.shape(vec_j[4]): (25, 768)
type: <class 'numpy.ndarray'>

My sentences are short so there is quite a bit of padding with 0's. Still, I am unsure why this model seems to have a maximum sequence length of 25 rather than the 512 mentioned here: Bert documentation section on tokenization

"Truncate to the maximum sequence length. (You can use up to 512, but you probably want to use shorter if possible for memory and speed reasons.)"

$\endgroup$
4
  • $\begingroup$ Hi @mLstudent33, I am using Bert-as-service as well, but I only get (1, 768) size vector for each sentence. Would you mind sharing your code snippet, say on Github or with me via email? Thanks! $\endgroup$ Jan 9 '20 at 18:43
  • $\begingroup$ Hi can you look through my repos? My github username is nyck33. $\endgroup$ Jan 9 '20 at 20:51
  • $\begingroup$ Thanks. But I see you have 50 repos. Which one should I look into? $\endgroup$ Jan 9 '20 at 21:48
  • $\begingroup$ It says I used bert-as-service in the question with a link. $\endgroup$ Jan 9 '20 at 22:30
1
$\begingroup$

The default setting for max_seq_len is 25 as seen here under heading Server API: bert-as-service readme

There is an open issue regarding this on the Github repo here and the creator seems to be implementing a feature: bert-as-service issues

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.