0
$\begingroup$

I have several expert persons performing the same specific action (for example, squat or leap forward) multiple times. Say 5 persons do 100 squats each. They have an accelerometer attached to the same body parts. I record the accelerometer readings and get 100*5 = 500 data samples. They do it for multiple different actions (squat, push up, leap forward, etc). The way they record the action is as follows:

  1. Start recording (push button)
  2. Do the action
  3. Stop recording (push button)

Now I need to see that another person is doing the actions in correct order. For example: squat, leap forward, stand up, drop down, push up. I take his accelerometer data and continuously feed it to my classifier that needs to tell me if he now has done exactly a squat action, not a leap or a push up. So, when the first action, namely squat, was identified, I check against leap forward and so on.

There are several problems with this:

  1. These data samples have different amount of values, since somebody is squatting a bit slower, others do it a bit faster. So, some data samples have 250 XYZ values, other have 220 or 270 etc. (in range of +-50). What I do for now is make stricter rules. I discard all the data samples that exceed 250 readings and for ones that have less values than 250, I append the values from the beginning to the end so that it gives 250 in total. Works fine, since there is a windup for every action where person is standing still for a brief moment before he performs the action. This is not optimal, because the experts need to redo the action, if they were too slow (the windup was too long) + I append fake data. What would be a better solution to handle this?
  2. For now I am using Random Forest, AdaBoost classifiers with low/high pass filtered accelerometer data that I map to 750 columns (250 X, 250 Y, 250 Z) with 1 class column. So the prediction tells me something like 70% leap, 25% squat, 5% push up. The classification is sometimes wrong or not precise enough. Thus, I was thinking of extracting some features from my signal series and feed them to the algorithm instead. My problem is that I do not know what features to extract.

The majority of papers that I found focus on human activity recognition to differentiate between walking, running, ascending and descending stairs. They were not very helpful in regard that they have continuous data flow of a person walking/running for hours and they use much more sample data. In contrast, with my task I have data set instances that are separate from each other.

I am not asking to solve those whole tasks for me, just guide me into some direction with a good explanation why it might be useful.

Thank you in advance!

$\endgroup$
1
$\begingroup$

Firstly for every expert you need to create a separate model because activity of one expert is totally different from other expert.

Addressing your first problem :

Appending fake accelerometer data will completely bias the results to one activity because at worst 50 points out of 250 i.e. 20% data is being augmented at your end. Rather you go with the number of points obtained for particular action performed by the expert, without appending fake/augmented data to the obtained accelerometer data. You can discard the points above 250, that will not affect the prediction much.

Addressing your second problem:

You can go for various statistical feature extraction such as (X,Y,Z)max, (X,Y,Z)min, (X,Y,Z)mean, (X,Y,Z)std. You can also use SMA(signal magnitude area) = |X|+|Y|+|Z| - The SMA variable is used to distinguish mobility (activity) and rest period in a time series.

You can validate the correlation between the features and and the activity classes.

$\endgroup$
  • $\begingroup$ Thank you, very useful! So if I have 5 classifiers for 5 experts, how do i use them together to check if another person is doing the correct action? Won't the models be overfit, because they will basically be stuck with the behavior of one particular person and not much room to breathe if there is any deviation. About discarding the data. The thing is, when the expert pushes the button, he then gets ready to perform an action for like 0.5-0.75 seconds (removing the hand from the button and visual feedback from the button click takes time) $\endgroup$ – t0a0 Sep 27 at 9:35
  • $\begingroup$ The feedback is instant but human reaction takes time. So at the beginning of the data set, the first 50-75 values are kinda the same, its inside the 75-220 values range where the actual action happens and then u need about 0.3 seconds (30 values) to click the button to stop recording. So sometimes if i discard the values after 250 I will actually discard the important ones, whilst the ones in the beginning are always quite same for any action for any person. $\endgroup$ – t0a0 Sep 27 at 9:36
  • 1
    $\begingroup$ I mentioned that the model should be different is cauz of the deviation in a values of XYZ for same activity. For exampl, If A expert does push ups as a newbie and B expert does push ups as a trained professional, values of XYZ are totally different for the two experts and our model for some different activity, in future, may classify it to push ups because of lot of deviation of values in the same class. So basically if you use the above features as training features in your model, you will observe this type of behaviour and with raw acceleration data, this is totally impossible to visualise. $\endgroup$ – abheet22 Sep 27 at 9:45
  • $\begingroup$ I see your point. It makes sense than to have 1 expert and force another person to perform the action in exactly the same manner as the expert. $\endgroup$ – t0a0 Sep 27 at 9:53
  • $\begingroup$ And regarding discarding the value, you need to make decision depending on the nature of data (as mentioned in your comment) you obtain at the start or at the end. Basically for this you need to validate all these decision based on the ground level truth. $\endgroup$ – abheet22 Sep 27 at 9:56
1
$\begingroup$

Regarding your first problem: I would suggest that you do not discard the the amount of values even if it goes more than or less than 250. Instead what you can do is ; aggregate the values of accelerometer over a time intervals and then tie it up to a single action.

Regarding your second problem: You have around 750 columns of the data. It would be very difficult to use random forest algorithm on it and get higher accuracy on it. You have to apply dimension reduction and then feature extraction techniques. You can go ahead with PCA (principal component analysis) f 750 independent variables. Reduce it down to 2 or 3 variable and check how much variance these reduced variables can explain. if it less than 60%. You can apply T-SNE algorithm to extract features more on it.

P.S. to check if your reduced variable can explain your dependent variables (like squat, sitting, pushup), you can plot the scatter plot of reduce variable values and then color the values based on your dependent variable. you can click below link to understand what I am saying

https://blog.bioturing.com/2018/06/18/how-to-read-pca-biplots-and-scree-plots/

$\endgroup$
  • $\begingroup$ thanks a lot for your input. I will check out the algorithms you mentioned regarding the second problem. I didn't quite get what you meant regarding the first one. Could you please elaborate or give an example? $\endgroup$ – t0a0 Sep 27 at 8:31
  • 1
    $\begingroup$ I am assuming your data have 750 columns and 200-270 rows for a single activity. $\endgroup$ – Vikas Jangra Sep 27 at 8:35
  • $\begingroup$ Not exactly. First when i record the data i have 3 columns (for X, Y, Z) and ~250 rows (readings over 2.5seconds with 100 readings per second). then i take 250 X rows and put it in 0-249 columns, then 250 Y rows and put it in 250-499 columns, then i take 250 Z rows and put it in 500-749 columns that results in 750 columns. So i map 250r X 3c matrix to 1r X 750c. This is for one data sample (1 person recorded 1 action once). So for 100 data samples I will have a matrix of 100 rows X 750 columns $\endgroup$ – t0a0 Sep 27 at 8:40
  • 1
    $\begingroup$ You have to first define the time interval for an activity and that has to be a constant for all activity. Then, take the statistics(median, variance and standard deviation) for each activity for that time interval. That would constitute you one row of data. For example suppose you fix that 2 seconds is the time to complete squat or pushup. you can take data in those two seconds and take statistics. columns for your data would be statistics like (mean-x, mean-y, mean-z, SD-X, SD-Y, SD-Z). Each row of 2 seconds will take label of squat or pushup. Hope this helps. $\endgroup$ – Vikas Jangra Sep 27 at 8:58
  • 1
    $\begingroup$ I understand that some people do the activity faster than other people. For that you can take the median value of time taken to complete that particular activity. You can try with median value first and then see if you getting proper accuracy with it or not. $\endgroup$ – Vikas Jangra Sep 27 at 9:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.