4
$\begingroup$

I'm fairly new to machine learning so please pardon me if I mess up terms. I'm working on a wearables project that involves hand gesture detection, looking for any pointers on potential algorithms that would be applicable.

Basically there will be sensors that pick up tendon movement at the wrist to provide input, so for simplicity I'm assuming that would mean 5 features (one for each finger), each with values ranging from say 0-100.

There are also 5 classes that the gesture could be classified as. The user would do each type of gesture several times to provide training data, and then for each gesture class I'd need an algorithm to determine the best model for it based on that training data.

After initial training, whenever the user does a gesture it would input the sensor readings into each model to see which one is the closest match. Additionally, since this is a wearable anything that optimizes for low memory and low processing power would be best.

If anyone has any input on either the type of models or the algorithm with which to generate them that I should look into I'd appreciate it!

Edit: Here is some sample data from 4 sensors during 3 different gestures. These are basically just pressure readings from sensors lying along tendons on the wrist.

$\endgroup$
3
$\begingroup$

It might be really useful to provide a bit more details regarding the actual data, produced by your sensor. Can you provide a simple example for any simple gesture?

Anyway, let's try to address the problem in the following way:

  • sensor can be represented as a function f(t)
  • range of the function will be {1,2,3,..100}

So we can draw something like the following (can we?):

enter image description here

You can extract various high-level features out of that data, but I would suggest to go with inflection points as the most straightforward option:

enter image description here

Those points, basically, define a pattern. I'll use a figure from the different answer to illustrate the idea. For example, the naive pattern below is used to identify a circle.

enter image description here

In your case situation is a bit simpler, because sensor output is the only data you can work with. Naturally, you can combine multiple sensors in order to define a bit more complex interaction scenarios:

enter image description here

Also you can take into account f'(t) in order to distinguish between slow and fast movements, etc.

Update

Thanks for the data. It does look like these three gestures looks pretty much the same from the data perspective (see visualization below).

enter image description here

It really looks like several features are required to identify the gesture.

$\endgroup$
  • $\begingroup$ Awesome, thanks for the response. I edited my question to add sample data $\endgroup$ – Luke Nov 19 '15 at 3:04
  • $\begingroup$ @Luke, did you manage to solve the problem? See update, I visualized the data (sorry for the delay). $\endgroup$ – Renat Gilmanov Jan 24 '16 at 19:37
0
$\begingroup$

The types of methods that would work depend on the nature of the data being captured in relation to the output classification. If the gestures are linearly separable in the 5-dimensional input space, a simple logistic regression is cheap and fast.

Alternatively a weak but very cheap method would be to find the mean input vector for each of the gestures, and simply decide which gesture vector the input is closest to at any point. The downside there you might need another class for "not gesturing."

It's also possible to gather a lot of test data from many users to pre-train a model, and long as readings don't vary too drastically between users,you wouldn't need to train on a by-user basis (on the wearable itself). In most cases prediction is efficient, so you wouldn't be too limited by computational power.

$\endgroup$
  • $\begingroup$ Thanks for the response, edited my question to include data $\endgroup$ – Luke Nov 19 '15 at 3:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.