My Training data look like this .
I have to recognize 4 class for each sentence. Any algorithm , which have some learning parameters Means not rule based approach . So which method is good for my problem ?
$\begingroup$
$\endgroup$
5
-
$\begingroup$ This data was processed manually wasn't it? Tokenization is inconsistent: why is "7 Up / Pepsi" one word? (same thing for "AI A", "AI am"). The first problem to solve is this: if the tokens (words) are not obtained automatically in the training data, the model will fail with any test data where tokenization is not done the same way. For the record there's no obvious indication that a NER system could use, and a rule-based method would probably work well to recognize "size" and "unit". $\endgroup$– ErwanNov 23, 2020 at 12:27
-
$\begingroup$ Yes you are correct , data is prepared manually . I have to work on my data this is just to show how my data looks like . But in my code I will clean my data . I want to know any NER which work better here . First I supoosed to use Bi-LSTM here , But then i thought length of sentence is not large . So it may be difficult for Bi-LSTM to learn . Is there any other method for my problem ?? $\endgroup$– ar shNov 25, 2020 at 10:08
-
$\begingroup$ You could try Conditional Random Fields, this is the traditional approach for sequence labeling. The main issue is that there are no textual indications, so it's likely that the system will only be able to recognize the brands and products seen in the training data. $\endgroup$– ErwanNov 25, 2020 at 11:22
-
$\begingroup$ Thanks for your suggestion . Now I am try to solve this problem by classify each word of sequence in 'brand' or 'product' . For 'units' or 'size' I will use rule base . And filter them out from sentence . So after this only brand or product will left . I also want to know any other algorithm which work here good ? $\endgroup$– ar shNov 26, 2020 at 4:54
-
$\begingroup$ That would be regular supervised classification, there are many choices but I don't know which one would suit your data better. An important question is what are going to be the features, it's not clear to me which ones would help the classification. $\endgroup$– ErwanNov 26, 2020 at 16:52
Add a comment
|