I have a series of timestamps that represent the time a user clicked a certain button.

My goal is to detect the automated clicks, so I need to find recurring patterns in the data that may point to an automated script. The majority of the data is by regular users. I don't need to detect them online, I just need to detect them in historical data. I also have user_id to make a distinction between users.

Is there a default approach to detecting recurring patterns in a largely unstructured dataset?

Most things I've found are about anomaly detection on a certain value paired with a timestamp but my problem is not really anomaly-detection as I need to find patterns amongst unpatterned data

  • $\begingroup$ Welcome to the site! Do you need to detect it as it happens? Are there multiple buttons you need to track? This is an anomaly detection problem. $\endgroup$ – Emre Aug 21 '17 at 8:47
  • $\begingroup$ I don't need to detect it as it happens, I just need to detect it in historical data. It's one event/button I need to track. $\endgroup$ – Deb Aug 21 '17 at 8:48
  • $\begingroup$ Do you also have some training data that distinguishes bots from users? $\endgroup$ – Emre Aug 21 '17 at 8:56
  • $\begingroup$ @Emre Unfortunately not $\endgroup$ – Deb Aug 21 '17 at 8:59

The question can be super difficult if you only have the number of clicks per time-stamp. The reason is that you might find many different recurrent patterns! For instance certain time periods in year, month or day (according to the functionality of website) may attract a certain temporal pattern of clicks. But if you have more relevant features the story is different.

After all I will point out some directions and hope it helps:

Autocorrelation may initially give you a good insight about what is happening there.

Then go for Time-Warping methods to define similarity function and seek it along time-series.

Embedding in high-dimensional spaces disclose the temporal recurrent pattern of a time-series. It's mostly from physics literature and is called State-Space Recunstruction. This method transfers your time-series into a m-dimensional space using this and there you can see the time-series as a dynamic system which may help in detecting recurrent dynamics.

  • $\begingroup$ Thank you for the info, I will checkout the links! Other relevant data I have are user id's so I am sure to only find patterns from a single user/bot. $\endgroup$ – Deb Aug 21 '17 at 9:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.