My objective is to estimate differences between how five political parties use moral words in their tweets and speeches. To that end, I have a dictionary that I pass to each tweet text / audio transcription through regex (this is important because audio transcriptions are somewhat more noisy and I cannot use bag of words) and get the frequencies with which each moral value is mentioned. Afterwards, I will use Tukey HSD intervals to estimate differences between parties. Nevertheless, my biggest concern here is whether I should compare absolute or relative frequencies of words. Relative frequencies seem like the right choice, because they allow to know how much is each moral value being used controlling for text / audio length. But on the other hand, absolute differences are interesting (especially for the particular case of audios, which can be noisy and not fully capture the total length of the texts and audios according to whitespaces). Is there any guideline to follow here?
$\begingroup$
$\endgroup$
1
-
1$\begingroup$ Welcome to DataScienceSE. I agree that relative freq seems a better option. The argument about noisy audio doesn't seem really strong to me: errors are going to impact absolute or relative frequency equally, afaik. $\endgroup$– ErwanAug 5, 2022 at 12:42
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Some point I can think of
- Find word similarity using gensim. your dictionary may not contain all words
