class sklearn.feature_extraction.text.TfidfVectorizer( *, input='content', encoding='utf-8', decode_error='strict', strip_accents=None, lowercase=True, preprocessor=None, tokenizer=None, analyzer='word', stop_words=None, token_pattern='(?u)\b\w\w+\b', **ngram_range=(1, 1)**, **max_df=1.0**, **min_df=1**, max_features=None, vocabulary=None, binary=False, dtype=<class 'numpy.float64'>, norm='l2', use_idf=True, smooth_idf=True, sublinear_tf=False)
These are the hyperparameters of sklearn's TF-IDF vectorizer.
The ngram_range parameter takes in the possible range of ngrams to extract from the given documents.
I couldn't help but wonder how does the vectorizer come up with the possible ngrams (say bigrams or trigrams) for the documents supplied as word vector features. Does it use only min_df and max_df to come up with them or is there something deeper than what meets the eye going on underneath?