2
$\begingroup$

I'm looking for real datasets on which I could test my DBSCAN algorithm implementation, that is, a dataset of points in (ideally 2 dimmensional) space, or a set of nodes and info about the distances between them.

I have looked on SNAP and CRAWDAD for such datasets, like datasets of road networks with distances, or cities with GPS coordinates, etc, but I haven't found any!

I know that the DBSCAN is said to be one of the best algorithims of it's kind on real data, but can't seem to find the real data sets people use...

Suggestions?

$\endgroup$
4
$\begingroup$

If you want to test whether your algorithm works as expected, I'd use sklearn datasets. They allow you to create simple synthetic 2D data with certain properties: circles, half moons, etc.

enter image description here

If you want "real" datasets, here is an interesting resource found after a brief search:

It seems to be a collection of datasets used in the literature.

Otherwise, I'd recommend you to look for image segmentation datasets, for instance, maps, as they make good candidates for DBSCAN. Kaggle is good place to search, so is the Google Dataset Search tool

$\endgroup$
1
$\begingroup$

Kaggle has some nice datasets available, including the classic Iris dataset. Take a look and pick one that looks interesting.

There are some impactful real-world data sets there, including COVID-19 related data sets. Something on the lighter side might be this scrubbed Iris data set posted not long ago.

EDIT: to elaborate on COVID-19, Kaggle has the COVID-19 Open Research Dataset (CORD-19), a nice 2 GB data set created by the Allen Institute for AI (Allen as in Paul Allen of Microsoft fame) with many partners. It's a great first place to start. They also have a nice COVID-19 data set from John Hopkins University. There must be 100+ COVID-19 data sets. This link should bring up the search feature.

$\endgroup$
5
  • $\begingroup$ @C8H10N402 : I would love to use a Covid-19 dataset. Could you elaborate? $\endgroup$ – Joshua Benabou May 7 '20 at 12:45
  • $\begingroup$ @C8H10N402 : those data sets, as well as ones in kaggle are excellent. however i'm not sure what quanities i should be using to do cluster analysis. i thought about doing a 3-d cluster analysis to determine clusters of countries with many coronavirus cases using lattitude, longitude, and total # cases. however this would require me to define a distance function (weighted distance of great circle distance and diff in total # cases), which is somewhat arbitrary. Any ideas for something simpler? $\endgroup$ – Joshua Benabou May 9 '20 at 3:55
  • $\begingroup$ In that case Iris dataset would be a simpler start; there are two very clear clusters (two species will be in one cluster, the third in its own cluster). The Iris dataset is available on Kaggle and elsewhere. $\endgroup$ – C8H10N4O2 May 10 '20 at 6:02
  • $\begingroup$ @C8H10N402: thanks for the suggestion. I've done the Iris dataset, it's super simple. Covid data turns out to be less evident! I was also hoping to compare strongly connected component cluster algorithms such as Tarjan with DBSCAN on a real dataset. This would require a dataset to be in the form of a graph, with distances between nodes (edge wights for example). But I can't think of any real data sets that would take that form... $\endgroup$ – Joshua Benabou May 11 '20 at 0:13
  • 1
    $\begingroup$ Another repository of datasets that comes to mind is Movebank, a data repository of animal movement datasets. Clustering comes in to play when for example trying to distinguish commuting vs. foraging, for example bats flying to a lake to forage. Site is here: datarepository.movebank.org $\endgroup$ – C8H10N4O2 May 12 '20 at 4:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.