How does BERT work for Aspect-Based sentiment analysis?

Question

I have recently used a package to perform Aspect-Based Sentiment Analysis (ABSA) through a BERT model.

Briefly, the model takes two inputs:

• words that constitute the aspects
• a sentence on which we want to perform the ABSA

The BERT-based model outputs a sentiment list with three integers representing the positive, the neutral and the negative scores.

I would like to understand more about how this kind of model works.

I have seen many posts on Medium that are too general and read some papers that give the basic functioning for grated.

So, if anyone can explain extensively how bert-based ABSA works, it would be appreciated.

Thank you!

Answer 1

So, from what I have understood there are two ways to perform ABSA:

• Aspect category detection + Aspect category sentiment classification

• Aspect target extraction + Aspect target sentiment classification

ABSA with BERT involves using pre-trained BERT language model as a feature extractor for a supervised learning task. The model is then fine-tuned on a labeled set. For example, let's consider a set of reviews where each review is annotated with the aspect being discussed and the sentiment associated with the aspect.

During fine tuning, BERT takes the review text as input and produces a sequence of contextualized embeddings for each token in the input. These embeddings are often fed into a classification head, which predicts the aspect and sentiment of the review.

To perform the ABSA, the best model is typically fine-tuned using a multi-task learning approach, where the model is trained to perform multiple related tasks simultaneously such as aspect extraction and sentiment classification. This approach allows the model to learn shared representations that can capture both aspects and sentiment information in the input text.

In conclusion, ABSA with BERT involves fine tuning the pre-trained model on a labelled dataset to learn to identify aspects and sentiments in reviews. The contextualized embeddings generated by BERT are used to capture the meaning of the review text, while the classification head predicts the aspect and sentiment associated with each aspect.