I am wondering what's the best way to handle outliers when using non-supervised clustering algorithms?
If you have outliers, the best way is to use a clustering algorithm that can handle them.
For example DBSCAN clustering is robust against outliers when you choose minpts large enough. Don't use k-means: the squared error approach is sensitive to outliers. But there are variants such as k-means-- for handling outliers.