Consider we have two classes of points. Both of them come from a multivariate normal distribution with an unrestricted covariance matrix. Let's assume, that the densities of those distributions do not overlap much, i. e. $p_1(x) \cdot p_2(x) \approx 0$.
Will methods like TSNE, UMAP, PCA project those points (in a general case, since some of those methods are stochastic) into two visually separate clusters on a 2D plane?
I know this question is very broad, but I was wondering if there is a reason to search for clusters using gaussian mixture fitting or k-means in a high dimensional data, if TSNE projection shows no cluster structure. Or to put it even simpler: I know non-gaussian and non-oval clusters (embedded on some complex manifold) could exist if dimensionality reduction projection is a single blob of points, but I am not sure if this excludes the possibility of the clusters being multvariate normal.