Outliers Approach

Question

Having a schema which the majority of the values are IDs. Like this example (this isn't my real data):

ID   SCHOOL_ID   CLASSE_ID   STUDENT_ID   GRADE
1       1            1           1          17
2       1            1           2          10
3       1            1           3          4
4       1            2           19          11
5       1            2           21         8
...    ...          ...         ...         ...

Which one of this can be a better approach to detect outliers using SQL: - Standard Deviation + Average - Try to implement an clustering algorithm

I'm a little bit confusing about this...

Thanks

Answer 1

Student ID (and ID) doesn't make sense as a column to cluster on because it's not continuous, and is unique and high cardinality too so isn't even usable as a categorical value.

Clustering school and class ID could make a little sense if converted to a one-hot encoded value, but it's also probably high cardinality.

I think you may need to question whether those are even meaningful dimensions to cluster on. You might just drop them.