0
$\begingroup$

I'm new to machine learning. I have the following scenario:

I have five individuals that are each carrying an accelerometer. That sensor measures movement/acceleration on a scale from 0 to 255, 0 being no movement, 255 being max movement (at a 5-minutes interval). Some individuals carry sensors that are more sensitive, and some that are less sensitive. As such, some individuals' sensors will provide higher values, and some individuals' sensors will provide lower values, for the same movements.

Using a classification algorithm (i.e., discriminant analysis), I want to train a model that can identify different types of behaviour, based on the movement data. So I observe the individuals and use their observed behaviour as the response variable, and the movement data as the explanatory variables. I want to create one singular model, that can identify any future individual's behaviour based on their movement data. My question is: How do I have to normalise the training data to create such a global model, when the sensors all have different "sensitivities"? And how do I have to normalise the data from future wearers, to successfully classify their behaviour with the model?

The only way that I can think of would be to figure out the global mean and global standard deviation for all the 5 training individuals, and try to fit the training individuals' data to fit the global distribution. But I'm not sure how I would go about that.

Thank you kindly!

$\endgroup$

1 Answer 1

2
$\begingroup$

This is a tricky, common problem. The fundamental issue is that a bias may exist between training and production (deployment) data. This video (at ~minute 19) describes a case where an AI X-ray system which works great in lab but fails in real hospital due to this problem.

Back to your question, there is no certain way to tell in advance what mean/std (plus other unexpected variables in the wild) is best; your best bet would be to use whatever you have, and continuously collect and update your model with more data.

$\endgroup$
2
  • $\begingroup$ Thank you for your feedback! One solution I've now been considering would be to zero-center each batch by removing the individuals' data point mean from each of this individuals data points. Whether that would decrease the problem, I don't know. $\endgroup$ May 5 at 5:48
  • 1
    $\begingroup$ Worth a try. Maybe zero-center and divide by SD (standardize) too? Anyway, general truth is you never know for unseen data from different source. $\endgroup$
    – lpounng
    May 5 at 6:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.