I'm trying to write a framework to compare a set of labels such as (for a sample of 5 yes/no answers to a question)
[0, 1, 1, 1, 0] to a series of features to determine correlation. For numerical non-sparse features, like "number of words" or "average word length", I know I can use a variance-covariance matrix and get a sense for whether or not "number of words" or "average word length" is an informative feature for a model to answer the question.
I'd like to be able to do the same thing for term frequency (let's say using CountVectorizer in scikit-learn), but the resultant covariance matrix will be rather large and will only indicate whether or not that particular term is an informative feature. How do I get some kind of "collapsed" or "aggregate" measure of correlation? Is this even possible?