# Detecting pushups based on pose data

I've been playing with Google's MLKit, and decided to detect push ups.

As a quick test, I took the position of the left shoulder, and plotted the Y Axis. Here's how a variety of trials look:

Five pushups

6 pushups, slightly farther away from the phone

No pushups at all

6 pushups, far aware and with a different orientation

## Good news

If I look at these graphs, I can visually intuit pretty well how many pushups someone does.

## Challenge

I am not quite sure how I could detect that algorithmically.

I thought about using something like "slope changes", but with just that I would be detecting a lot of noise too. I thought about using a fourier transform, but from some basic playing, I wasn't able to clearly differentiate the "pushup" signal from the rest.

The two big problems I think are:

• a. The "pushup" waves can differ in both width and height, based on how close the user is to the phone
• b. I can't rely on just slopes to figure this out, and need some sort of "wave" detector

## Current Plan

Currently, I'm thinking about doing some special heuristics:

• Maybe I can use other info, like:
• coordinates of the hips (to make sure they are within the same latitude as the shoulder)
• X coordinate info
• I could use the above two pieces of information, to "remove" the bad areas
• (any place that has a big movement in X coordinates, or hip is not aligned, can't be a pushup), than maybe I could rely on something like a "significant" slope change.

## Question

Right now I am in the position where there's a lot of stuff I don't know I don't know. What do you think of my "current plan?". Would you approach it differently? Is there some method you would look into?

Current exploration so far: https://github.com/stopachka/pushups-data-exploration/blob/main/exploration.ipynb