t-SNE, as in , works by progressively reducing the Kullback-Leibler (KL) divergence, until a certain condition is met.
The creators of t-SNE suggests to use KL divergence as a performance criterion for the visualizations:
you can compare the Kullback-Leibler divergences that t-SNE reports. It is perfectly fine to run t-SNE ten times, and select the solution with the lowest KL divergence 
I tried two implementations of t-SNE:
Both these implementations, when verbosity is set, print the error (Kullback-Leibler divergence) for each iteration. However, they don't allow the user to get this information, which looks a bit strange to me.
For example, the code:
import numpy as np from sklearn.manifold import TSNE X = np.array([[0, 0, 0], [0, 1, 1], [1, 0, 1], [1, 1, 1]]) model = TSNE(n_components=2, verbose=2, n_iter=200) t = model.fit_transform(X)
[t-SNE] Computing pairwise distances... [t-SNE] Computed conditional probabilities for sample 4 / 4 [t-SNE] Mean sigma: 1125899906842624.000000 [t-SNE] Iteration 10: error = 6.7213750, gradient norm = 0.0012028 [t-SNE] Iteration 20: error = 6.7192064, gradient norm = 0.0012062 [t-SNE] Iteration 30: error = 6.7178683, gradient norm = 0.0012114 ... [t-SNE] Error after 200 iterations: 0.270186
Now, as far as I understand, 0.270186 should be the KL divergence. However I cannot get this information, neither from model nor from t (which is a simple
To solve this problem I could:
- Calculate KL divergence by my self,
- Do something nasty in python for capturing and parsing
TSNE()function's output .
- would be quite stupid to re-calculate KL divergence, when
TSNE()has already computed it,
- would be a bit unusual in terms of code.
Do you have any other suggestion? Is there a standard way to get this information using this library?
I mentioned I tried R's tsne library, but I'd prefer the answers to focus on the python sklearn implementation.