Let's take the task of evaluating very short dance movements (phrases) using sensor data (accelerometer and gyro from an iPhone device) as an example. If the model's confidence is 100% on a particular dance phrase, it does not necessarily follow that the user performed this movement phrase perfectly.
Given this task that consists of very short movements (1-2sec), given that a very high-quality dataset (sensor data) is under disposal, given that the model has very high accuracy in classifying these movement phrases (actions) would it be fair to assume that this action classifier can also serve as a movement evaluator?
For example, we can set a threshold of 50% and evaluate the movements based on the model's confidence, i.e. if the model is 40% confident that this movement (we know the ground-truth beforehand) is X we say that the user didn't perform the movement correctly but if the model has a 90% confidence we say that the movement was performed correctly. In other words, we give feedback to the user about his performance based on the model's confidence.
Or it still doesn't matter and we can't simply draw this conclusion (robust action classifier = potential action evaluator)?
Alternatively, how much sense it would (theoretically) make if I feed certain data qualitative characteristics, such as the 25th, 50th, and 75th percentile (certain spikes at these points make up for the quality of my kind of data) as well as the mean and S.D. for each sensor as features to an attention model reasoning that, since I feed these as input features to the model, the classifier's prediction might now have been slightly nudged to an evaluator's prediction?
