I am very new to nlp. I am doing a text segmentation task and for evaluating my model I need to calculate Pk and Windiff scores. My question is what is the ideal value for window size (k) for Pk score because different window sizes give different results. I am using this function nltk.metrics.segmentation.pk. Thanks.
1 Answer
I am also new to NLP, but according to some research papers:
Window size k should follow the next expresion:
This applies for WinDiff and Pk scores.