I am currently working on a dataset that belongs to the restaurant and food delivery domain. After completing sentiment analysis and quantification, I now need to select a Change Point Detection Algorithm and detect a shift in the sentiment signal in the reviews on each category of restaurant. The signal will be a score that is the difference between the positivity and negativity of the reviews in a certain time frame. I have considered 3 sentiments that are positive, neutral and negative and am thus contemplating the use of a multivariate time series. Since, I have the complete dataset at hand, I will be conducting an offline change point detection algorithm to detect a shift in the sentiment of the reviews in the pre-covid and post-covid time. Please provide me some help as to how do I select an algorithm. I am very confused and a few factors to consider with links to resources are welcome.
2 Answers
The problem with Change point detection (or positive/negative trend detection), is that it depends on many things, including noise and sensitivity through time. For instance, you cannot send an alert when the scores just started being negative in one day. You have to wait a few days to see if the trend is actually bad or not. Consequently, you have to tune your model to define the "few days" and the "criticality". That's why I would recommend to visualize every category trends by using a general score system (ex: sum of positive +1, neutral 0 and negative -1) and applying a smoothing function (ex: Kalman filter, with different noise reduction values). In this way, you should be able to detect the required sensitivity to detect when the situation is becoming critical or better. For the smoothing function, you can use pykalman. After evaluating the right noise reduction and the right quantity of days N, you can apply the diff function to measure the difference of the filtered curve in the last N days.
-
1$\begingroup$ Thankyou @Nicolas M so much for this. I really appreciate the time you took to help me on this. May I know if this is the criterion based approach? If yes, where do the other approaches like the Bayesian appear in your structure of thinking about it. $\endgroup$– NishaJul 2, 2021 at 16:42
-
$\begingroup$ You are totally right to choose a mathematical criterion approach like Bayesian. But I would recommend first to observe the resulting curves and set a quantity of days that would "make sense" to you (could be more or less one week or two in your case). $\endgroup$ Jul 3, 2021 at 7:46
-
You haven't specified the programming language you are using,so I'll provide different options.
The "ruptures" package in python would be a good option that would allow you to try out several different change point detection methods to your data. Before using it maybe it would be helpful to read this review, where the rupture methods are further explained - understanding the methods better will help you on the method selection.Another review paper that summarises and explains change point detection methods is "A Survey of Methods for Time Series Change Point Detection".
Alternatively if you are using R you could use the "changepoint" package (reference manual here)
Finally, from the description of the task I believe simpler change point detection methods could also work e.g. I would start with trying CUSUM method (available in R here and it is also integrated in the signal processing library in matlab)
