11
$\begingroup$

t-SNE, as in [1], works by progressively reducing the Kullback-Leibler (KL) divergence, until a certain condition is met.
The creators of t-SNE suggests to use KL divergence as a performance criterion for the visualizations:

you can compare the Kullback-Leibler divergences that t-SNE reports. It is perfectly fine to run t-SNE ten times, and select the solution with the lowest KL divergence [2]

I tried two implementations of t-SNE:

  • python: sklearn.manifold.TSNE().
  • R: tsne, from library(tsne).

Both these implementations, when verbosity is set, print the error (Kullback-Leibler divergence) for each iteration. However, they don't allow the user to get this information, which looks a bit strange to me.

For example, the code:

import numpy as np
from sklearn.manifold import TSNE
X = np.array([[0, 0, 0], [0, 1, 1], [1, 0, 1], [1, 1, 1]])
model = TSNE(n_components=2, verbose=2, n_iter=200)
t = model.fit_transform(X)

produces:

[t-SNE] Computing pairwise distances...
[t-SNE] Computed conditional probabilities for sample 4 / 4
[t-SNE] Mean sigma: 1125899906842624.000000
[t-SNE] Iteration 10: error = 6.7213750, gradient norm = 0.0012028
[t-SNE] Iteration 20: error = 6.7192064, gradient norm = 0.0012062
[t-SNE] Iteration 30: error = 6.7178683, gradient norm = 0.0012114
...
[t-SNE] Error after 200 iterations: 0.270186

Now, as far as I understand, 0.270186 should be the KL divergence. However I cannot get this information, neither from model nor from t (which is a simple numpy.ndarray).

To solve this problem I could:

  1. Calculate KL divergence by my self,
  2. Do something nasty in python for capturing and parsing TSNE() function's output [3].

However:

  1. would be quite stupid to re-calculate KL divergence, when TSNE() has already computed it,
  2. would be a bit unusual in terms of code.

Do you have any other suggestion? Is there a standard way to get this information using this library?

I mentioned I tried R's tsne library, but I'd prefer the answers to focus on the python sklearn implementation.


References

[1] http://nbviewer.ipython.org/urls/gist.githubusercontent.com/AlexanderFabisch/1a0c648de22eff4a2a3e/raw/59d5bc5ed8f8bfd9ff1f7faa749d1b095aa97d5a/t-SNE.ipynb

[2] http://homepage.tudelft.nl/19j49/t-SNE.html

[3] https://stackoverflow.com/questions/16571150/how-to-capture-stdout-output-from-a-python-function-call

$\endgroup$
4
$\begingroup$

The TSNE source in scikit-learn is in pure Python. Fit fit_transform() method is actually calling a private _fit() function which then calls a private _tsne() function. That _tsne() function has a local variable error which is printed out at the end of the fit. Seems like you could pretty easily change one or two lines of source code to have that value returned to fit_transform().

| improve this answer | |
$\endgroup$
  • $\begingroup$ Essentially what I could do is to set self.error = error at the end of _tsne(), in order to retrieve it from the TSNE instance afterwards. Yes, but that would mean changing sklearn.manifold code, and I was wondering if the developers thought of some other ways to get the information or if not why they didn't (i.e.: is 'error' considered useless by them?). Furthermore, if I changed that code I would need all the people running my code to have the same hack on their sklearn installations. Is that what you suggest, or did I get it wrong? $\endgroup$ – joker Jul 17 '14 at 16:07
  • $\begingroup$ Yes, that is what I suggested as a possible solution. Since scikit-learn is open source, you could also submit your solution as a pull request and see if the authors would include that in future releases. I can't speak to why they did or didn't include various things. $\endgroup$ – Trey Jul 17 '14 at 17:10
  • 2
    $\begingroup$ Thanks. If anybody else is interested in this, github.com/scikit-learn/scikit-learn/pull/3422. $\endgroup$ – joker Jul 18 '14 at 15:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.