1
$\begingroup$

Suppose I have a dataset containing feature values I don't all agree with, as another person selected them. An example problem could be classifying 'good' or 'bad' music using subjective features, e.g., emotion. If an emotion feature of a particular sample has a value of 'sad,' whereas I think 'peaceful' is a much better representation, would that count as feature noise?

Suppose I then re-labeled the samples as 'good' or' bad' by my standards, without touching the subjectively 'wrong' features. Will a classifier still be able to accurately classify 'good' music based on my taste, despite the features?

Although, the fact that subjective features can even be used in ML problems is something I assumed, as I'm unable to find any related resources. I'm also relatively new to ML, so please do correct me.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

Welcome to datascience.stackexchange,

One way to look at the subjectivity problem is a model per subjective scenario (in your case - per person). Then, each classification could use some kind of voting mechanism between all models' classifications.

In that sense, using your friend's model on "yourself" would be more of in-accuracy than noise, and changing the labels might be more of a noise (like a noisy sensor in a system, that collected the wrong target value).

An alternative might be to group-by song observations from different persons, aggragating per-song-features by some averaging. Thus, if we have enough unique sensors (i.e. persons) a song might get an average emotion value of 6.5 and would classify closer to peaceful.

I guess it comes down to - to what extent an individual's opinion can be represented in a collected opinions' data-set, and how would that matter for the user-story you were asking about.

This is a good example of how an early ml-project's domain-expertise with regard to a well-defined user-story - can make the difference.

The user story here might be: classifying an individual's song's score, while the domain-expert might state how, if any, it's possible to classify an indivudal's taste based on the number of unique-sensors.

Regarding the emotion averaging, in case it's not possible to project it on a numeric scale (domain-expert issue), see my post here regarding n-hot encoding, for exactly this scenario (encoding not just the existense of a category-value, but also its weight)

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ That's an excellent insight, thank you. I find it odd that there aren't many resources about the subjectivity problem. If my understanding is correct, both methods you mentioned require validation or aggregation of subjectivity from different people (or unique sensors). Assuming that's not available, would my model be any good, in the sense of classifying what I think is good or bad music? $\endgroup$
    – KEiixel
    Apr 4, 2022 at 1:10
  • $\begingroup$ You're welcome. In the current state of the model subjectivity is the unique input per song, so there's no way to guaranty a per-person target without a per-person model. If we can find a way to represnt a subjectiviy-per-person as an input - we'll have a shot. This would then become more of a Recommendation System. And there - I believe you'll find more knowledge about subjectivity. $\endgroup$
    – mork
    Apr 4, 2022 at 9:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.