4

I would get a sufficiently large random/representative sample and cluster that. To see what is such a sample, you will have to get two such samples and cluster them to get cluster solutions c1 and c2. If the matching clusters of c1 and c2 have the same model parameters, then you probably have representative samples. You can match the clusters by looking at ...


2

Yes, this is a classic Graph-Based Clustering problem in which each location is a node and you have the distances between them. Forgetting about concept of graphs and graph-based algorithm which might be complicating, I directly jump to your answer. The most well-known algorithm is Spectral Clustering. There are tones of tutorial out there and it is well ...


2

It's definitely viable, just that there is catch 22. In order to get this representative sample from your dataset, you have to sample from every cluster. But if you already can sample from every cluster, you already know them, hence you don't need unsupervised learning.


2

matplotlib already takes care of coloring adjancent clusters with different colors. But, I believe it uses a unique color for each cluster. If that's the case, 400 colors would be too much. There might be better ways, but worst case, try this: We want to color with minimum number of colors. Hence, the problem turns to a graph coloring problem in which, we ...


1

No, as there is randomness in the method implementation, for example here (in LdaModel of the gensim library). Hence, it can affect your final result in each run. Therefore, if you want to keep the result reproducible, you can set the random_state property of the model to a constant seed (see the documentation for more details).


1

I found that taking the centroids of each cluster, running k-nearest-neighbors, and then applying https://en.wikipedia.org/wiki/Greedy_coloring works well. Just keep increasing K until the clusters stand out. Edit: following @Fatemeh Asgarinejad's suggestion, use the minimum distance from a cluster centroid to a member of the other clusters as the distance ...


1

Dynamic Time Warping might be what you're looking for: it measures similarity between two time series based on the optimal alignment between the two sequences. For example point $i$ in sequence 1 might be better aligned with point $i+3$ in sequence 2 based on the evolution of the sequences (as opposed to Euclidean distance which would always compare $i$ in ...


1

The meaning of $\epsilon$ is that of the neighbourhood size. The neighbourhood of a point $p$, denoted by $N_{\epsilon}(p)$, is defined as the $N_{\epsilon}(p) = \{q \in D | dist(p,q) \leq \epsilon \}$. Here $D$ is a database of $n$ objects (points) and $q$ a query point. So what you Professor probably wants you to do is to evaluate goodness of clustering ...


1

You are describing semi-supervised learning where the training dataset is only partially labeled. One common set of techniques to solve that problem is active learning. In active learning, there is a learning loop where the algorithm makes predictions and a human corrects those predictions. Pre-clustering is a specific active learning technique to address ...


1

I strongly recommend to start with a simple statistical data analysis. In this approach you take the Moving Average/Median of signals and if one signal shows a magnitude -/+ 3 times standard deviation, you mark it as anomaly. Please have a look at this answer for Python code.


1

Welcome to the community! You may want to refer to a tutorial on Agglomerative Hierarchical Clustering before reading this answer. My explanation is more practical. Assume the data below: from scipy.cluster.hierarchy import ward, fcluster from scipy.spatial.distance import pdist import numpy as np import matplotlib.pyplot as plt from matplotlib.text import ...


1

The simplest solution for such a task coming to my mind is to do a simple kmeans clustering (or batch variants) using the exact same metrics as planned for the later hierarchical clustering step (in your case eucledian / minkowski with p=2). For the initial kmeans step you chose the number of clusters k such that a distance computations on those cluster ...


1

Let's say that your dataset is a bunch of temperatures with a bunch of features related to when/where the measurements were made. A measurement is a global outlier if it diverges from the distribution of temperatures regardless of the features (when and where) because that measurement is far off the global distribution (100°C for example). The example makes ...


Only top voted, non community-wiki answers of a minimum length are eligible