I am software engineer but have zero data science background, so apologies for this basic question.

I would like to find correlations between different behaviors of my daily life and different outcomes. Some examples would be:

  • How does "amount of screen time" correlate with "time to fall asleep"?
  • How does "amount of exercise" correlate with "amount of sleep"
  • How does "amount of alcohol" correlate with "amount of REM sleep"
  • How does "amount of sunlight exposure" correlate with "daily energy"

[these are basic examples of fairly well known correlations, the goal is to find new surprising correlations]. I would end up capturing as much data as possible on sleep, eating, exercise and any other relevant activities.

  • Due to the nature of the data, I imagine the effects of some variables could build up over time. For example, 1 night of bad sleep might not be a big deal but 5 would probably have a large impact on other variables.
  • Also, some effects may be caused by a combination of variables. For example, "energy level" could be affected by "amount of food", "amount of sleep" and "amount of exercise"

My current approach is to capture a set of these variables on a daily basis, write a script to put the data into a pandas DataFrame. Each day would be a row and each variable would be a column. Then I would use DataFrame.corr to find pairwise correlations. But this approach is very simple and limiting. It won't be able to find any deep (long period) or complex (multiple variable) patterns.

What other approaches/methods should I consider for this problem?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.