New answers tagged sentiment-analysis
1
First of all, in problems like you brought, you typically start with preprocessing. Specifically, in your case, you need to normalize it. That means, using some processing, you have to change both Are you there and are you there??? to just are you there. By doing that, you are removing duplicate examples there. Now, unlike Erwin, I suggest that in general. ...
2
In general it's not recommended to get rid of duplicates because it modifies the distribution of the data and this could bias the model. In other words, if the final application (or any test data) is expected to contain cases like these in similar proportions then it is preferable to train the model with these cases.
So the duplicates by themselves are not ...
2
A few thoughts:
The evaluation method is not clear, in particular what are the evaluation scores shown, is it f1 score?
Why do you need to improve "by at least 5%"? Do you know the results of another system on the same data? If not it doesn't really make sense to aim for a particular performance value: performance depends a lot on the data, it's ...
Top 50 recent answers are included