2
$\begingroup$

Running an unsupervised plot of my data, I noticed a hyperbolic ('boomerang') shape:

dim=2

vectorizer = TfidfVectorizer(min_df=5, max_df = 0.4, stop_words = 'english')
train_tf_idf = vectorizer.fit_transform(bunch_train.data)
svd = TruncatedSVD(n_components=dim,random_state=42)
svd_train = svd.fit_transform(train_tf_idf)
svd_train = Normalizer().fit_transform(svd_train)

labels = {
        0:'alt.atheism',
        1:'comp.graphics',
        2:'sci.med'}
y = np.vectorize(labels.get)(bunch_train.target)

with plt.style.context('seaborn-whitegrid'):
    plt.figure(figsize=(12, 8))
    for lab, col in zip(('alt.atheism', 'comp.graphics', 'sci.med'),
                        ('blue', 'red', 'green')):
        plt.scatter(svd_train[y==lab, 0],
                    svd_train[y==lab, 1],
                    label=lab,
                    c=col)

    plt.title('2D SVD on TF-IDF - 3-NewsGroups',size=28)
    plt.xlabel('PC1')
    plt.ylabel('PC2')
    plt.legend(loc='upper left',prop={'size': 20})
    plt.tight_layout()
    plt.show()

enter image description here

I suspect it has something to do with the Normalizer - when deleting the following line:

svd_train = Normalizer().fit_transform(svd_train)

the data plots like this:

enter image description here

$\endgroup$
  • $\begingroup$ it's not just an ellipse, it's the unit square; the obvious result of normalization! The aspect ratio of the plot is merely off. $\endgroup$ – Emre Aug 23 '17 at 18:06
1
$\begingroup$

The problem here is that you are normalizing the output of SVD. It might be clear that this output does not need to be normalized if you consider the output of each step:

  1. Vectorizer: [n_items x n_words] Matrix. Unnormalized.
  2. TFIDF: [n_items x n_words] Matrix with rows normalized to unit Euclidean norm.
  3. SVD: [n_items x dim] Matrix. Columns correspond to each row's loading onto two orthonormal unit vectors.

In short, the output of the third step (SVD) is meaningful as-is and normalization obscures that meaning.

As to why the normalized output appears curved? The math to show this seems a bit involved but my intuition is that normalization is mapping the row weightings onto the ellipsoid manifold used in SVD.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.