0
$\begingroup$

A year or two ago I wrote a keylogger that has been quietly running in the background of my computer. Each line consists of a timestamp, a keycode, and a value representing modifier keys (e.g. ctrl, shift, etc.).

Separately, I have a web app that I use to manually make notes about how I spend my time: TaskRanger.com

Task Ranger

As you can see in this screenshot, Task Ranger is basically a daily log of work done. Each day consists of a list of intervals. Each interval consists of a start time, a duration, and a string briefly describing the work I did during that period. The description also contains a twitter style hashtag that manually classifies the type of work I was doing.

I'm interested in combining both of these data sources to create a classifier that can take a raw log of keystrokes as input and output a sequence of tags indicating the type of work I did over time. For example, I might have been doing research from 11am to 1pm, then planning a project from 1pm to 2:30, then alternating between coding and debugging for the next few hours, etc.

The intent is to basically build a personal profiler. I want to become more productive. And I think I ought to be able to treat my workflow as a software program: figure out what the bottlenecks are, drill down into them, and focus my improvement efforts on those areas.

I tried to build some kind of naive classifier myself without much luck. One problem is I couldn't find an easy way to figure out when one kind of work ends and the next kind begins (segmentation I think it's called?). I think this whole classification task might be a lot harder than it initially seems. I have this vague idea that a Hidden Markov Model should work. Basically the type of work that I'm doing is a hidden state, and the keylogs provide clues as to that state. But I know next to nothing about signal processing or machine learning, so I'm not sure if I'm on the right track here.

Any advice or suggestions on how to build this would be appreciated. Some sample code demonstrating something similar would be especially useful.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Looks like you have a classification problem. A simple way to solve this is with a linear regression model. Here is how I would do that with the data you've provided:

1) Determine a "unit" of time, for example 1 minute of keystrokes. Once this process is complete you can tweak the unit of time to see if different intervals give you better results (5 minutes, 10 minutes, etc...)

2) Split all keystrokes into 1 minute chunks, and categorize those chunks as work, study, play, etc... Now you basically have a bunch of character strings that are categorized into activity types.

3) Create variables to describe your chunks. Word count, character count, wpm, # of times "lol" was typed, etc...Get creative.

4) Create a test set and training set of chunks.

5) Create a linear regression model to predict type of activity based upon the variables you created.

This is a pretty simple solution but if you can come up with good variables to distinguish your activity types it can work.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Yeah, I basically tried to do that with keystrokes per minute. It didn't work well. The problem is that... well, consider writing a long email for one hour. You don't hit the keyboard at a consistent rate. You'll tend to have bursts of activity followed by pauses where you think or look up some piece of information you want to reference. The keystrokes-per-minute graph will be spikey. Looking at one minute slice could be completely different from another even they they are both the same activity. In other words, context matters here. This is why I think a HMM might be a good approach. $\endgroup$
    – Jesse Aldridge
    Commented Mar 7, 2019 at 8:37
  • $\begingroup$ I am not familiar with HMM's but you could try using variables other than keystrokes per minute. For example you could use word counts of key words, such as Dear, Sincerely, report, etc... I am not sure if your keylogger captures key combos but you could also look for giveaway hotkeys, too. For example Alt+Tab or Ctrl+Shift. $\endgroup$
    – bstrain
    Commented Mar 8, 2019 at 5:00
  • $\begingroup$ Tried that, and in practice that doesn't work either. For instance, many of the words you type while coding are valid English words. You Alt+Tab in many different contexts. So on. $\endgroup$
    – Jesse Aldridge
    Commented Mar 9, 2019 at 1:05
  • $\begingroup$ My understanding is that a HMM is basically a way of saying that if you were coding 1 minute ago, then there is a 75% chance you are still coding, a 10% chance you switched to debugging, and 15% chance you switched to something else entirely different. You basically take features like you are describing and combine them with this learned state-change model and you get something that can account for context efficiently by doing a single linear scan through the sequence. But both the math and the implementation are a bit over my head. I guess there's no path forward but to learn the details. $\endgroup$
    – Jesse Aldridge
    Commented Mar 9, 2019 at 1:10
  • 1
    $\begingroup$ If you want percent chances you could stack several logistic regression models on top of one another. If you want a fun project with which to learn HMMs go for it but if you want the simplest solution stick to regression - they are powerful tools! $\endgroup$
    – bstrain
    Commented Mar 10, 2019 at 10:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.