How do I right feature selection for DBSCAN?

Question

I want to use DBSCAN to recognize any clusters within all text elements from the DOM tree of any webpage. For example all menu items shall be clustered separatey to all main content or footer elements.

My features for now are: - DOM path to the text element - visual rendered CSS properties (used PhantomJS) of the text element - the middle x and y position of the text element - the width of the text element

Before using DBSCAN I use StandardScaler for my feature vector.

But I have no good results even after trying channging eps.

Now the question: Can DBSCAN handle att all such different kind of features like OneHotEncoding for DOM path combined (!) with features like positions or widths in pixel? Should I weight some features before using StandardScaler that all have a similar value?

Answer 1

Automatic weighting will likely not be enough.

For examples standard scaler will assign twice as much weight to the one-hot encoded parts than to the other attributes. Plus, it is based on the assumption that all variables should have the same (assuming they are all normal distributed) weight. But should they all have the same weight? Does it even make sense to use distances of one-hot encoded data?

You are not ready to use DBSCAN (or any other clustering) if you don't have a reliable distance yet that discriminates similar from dissimilar instances.