-2
$\begingroup$

I have downloaded some data to learn about machine learning and distributed computing. I used WinZip to uncompress a .bz2. Now I have text (opened in Notepad) which looks like this (this file is long, this is just a sample):

    {"parent_id":"t3_5yba3","created_utc":"1192450635","ups":1,"controversiality":0,"distinguished":null,"subreddit_id":"t5_6","id":"c0299an","downs":0,"archived":true,"link_id":"t3_5yba3","score":1,"author":"bostich","score_hidden":false,"body":"test","gilded":0,"author_flair_text":null,"subreddit":"reddit.com","edited":false,"author_flair_css_class":null,"name":"t1_c0299an","retrieved_on":1427426409}
    {"score_hidden":false,"body":"much smoother.\r\n\r\nIm just glad reddit is back, #reddit in mIRC was entertaining but I had no idea how addicted I had become. Thanks for making the detox somewhat short.","author_flair_text":null,"gilded":0,"link_id":"t3_5yba3","score":2,"author":"igiveyoumylife","author_flair_css_class":null,"name":"t1_c0299ao","retrieved_on":1427426409,"edited":false,"subreddit":"reddit.com","ups":2,"controversiality":0,"parent_id":"t3_5yba3","created_utc":"1192450639","downs":0,"archived":true,"distinguished":null,"subreddit_id":"t5_6","id":"c0299ao"}
{"author":"Arve","link_id":"t3_5yba3","score":0,"body":"Can we please deprecate the word \"Ajax\" now? \r\n\r\n(But yeah, this _is_ much nicer)","score_hidden":false,"author_flair_text":null,"gilded":0,"subreddit":"reddit.com","edited":false,"author_flair_css_class":null,"retrieved_on":1427426409,"name":"t1_c0299ap","created_utc":"1192450643","parent_id":"t1_c02999p","controversiality":0,"ups":0,"distinguished":null,"id":"c0299ap","subreddit_id":"t5_6","downs":0,"archived":true}
{"author_flair_text":null,"gilded":0,"score_hidden":false,"body":"[deleted]","score":1,"link_id":"t3_5yba3","author":"[deleted]","name":"t1_c0299aq","retrieved_on":1427426409,"author_flair_css_class":null,"subreddit":"reddit.com","edited":false,"ups":1,"controversiality":0,"created_utc":"1192450646","parent_id":"t3_5yba3","archived":true,"downs":0,"subreddit_id":"t5_6","id":"c0299aq","distinguished":null}
{"edited":false,"subreddit":"reddit.com","author_flair_css_class":null,"name":"t1_c0299ar","retrieved_on":1427426409,"author":"gigaquack","link_id":"t3_5yba3","score":3,"body":"Oh, I see. Fancy schmancy \"submitting....\"","score_hidden":false,"author_flair_text":null,"gilded":0,"distinguished":null,"subreddit_id":"t5_6","id":"c0299ar","downs":0,"archived":true,"created_utc":"1192450646","parent_id":"t1_c0299ah","controversiality":0,"ups":3}
{"author_flair_text":null,"gilded":0,"body":"testing ...","score_hidden":false,"author":"Percept","score":1,"link_id":"t3_5yba3","retrieved_on":1427426409,"name":"t1_c0299as","author_flair_css_class":null,"edited":false,"subreddit":"reddit.com","controversiality":0,"ups":1,"parent_id":"t3_5yba3","created_utc":"1192450656","archived":true,"downs":0,"subreddit_id":"t5_6","id":"c0299as","distinguished":null}
{"ups":3,"controversiality":0,"parent_id":"t1_c0299ar","created_utc":"1192450658","archived":true,"downs":0,"subreddit_id":"t5_6","id":"c0299at","distinguished":null,"author_flair_text":null,"gilded":0,"score_hidden":false,"body":"I like it. One more time...","score":3,"link_id":"t3_5yba3","author":"gigaquack","name":"t1_c0299at","retrieved_on":1427426409,"author_flair_css_class":null,"subreddit":"reddit.com","edited":false}
{"ups":2,"controversiality":0,"parent_id":"t1_c02999j","created_utc":"1192450662","archived":true,"downs":0,"id":"c0299au","subreddit_id":"t5_6","distinguished":null,"gilded":0,"author_flair_text":null,"score_hidden":false,"body":" try refreshing yor cache, that worked for me \n//edit: trying to edit","score":2,"link_id":"t3_5yba3","author":"mcm69","retrieved_on":1427426409,"name":"t1_c0299au","author_flair_css_class":null,"edited":false,"subreddit":"reddit.com"}
{"controversiality":0,"ups":3,"parent_id":"t1_c0299at","created_utc":"1192450670","downs":0,"archived":true,"distinguished":null,"subreddit_id":"t5_6","id":"c0299av","body":"K. I lied. Just one more...","score_hidden":false,"author_flair_text":null,"gilded":0,"author":"gigaquack","score":3,"link_id":"t3_5yba3","author_flair_css_class":null,"name":"t1_c0299av","retrieved_on":1427426409,"edited":false,"subreddit":"reddit.com"}
{"edited":false,"subreddit":"reddit.com","author_flair_css_class":null,"retrieved_on":1427426409,"name":"t1_c0299aw","author":"deki","score":2,"link_id":"t3_5yba3","body":"I also wonder what the differences are...","score_hidden":false,"gilded":0,"author_flair_text":null,"distinguished":null,"id":"c0299aw","subreddit_id":"t5_6","downs":0,"archived":true,"parent_id":"t3_5yba3","created_utc":"1192450681","controversiality":0,"ups":2}
{"created_utc":"1192450683","parent_id":"t1_c0299av","controversiality":0,"ups":3,"subreddit_id":"t5_6","id":"c0299ax","distinguished":null,"archived":true,"downs":0,"author":"gigaquack","score":3,"link_id":"t3_5yba3","author_flair_text":null,"gilded":0,"score_hidden":false,"body":"So addictive...","edited":false,"subreddit":"reddit.com","name":"t1_c0299ax","retrieved_on":1427426409,"author_flair_css_class":null}
{"downs":0,"archived":true,"distinguished":null,"subreddit_id":"t5_6","id":"c0299ay","controversiality":0,"ups":1,"created_utc":"1192450687","parent_id":"t3_5yba1","author_flair_css_class":null,"retrieved_on":1427426409,"name":"t1_c0299ay","subreddit":"reddit.com","edited":false,"body":"I can't post a story to proggit - I got \"you are trying to submit too fast\" on my first submission.","score_hidden":false,"author_flair_text":null,"gilded":0,"author":"llimllib","score":1,"link_id":"t3_5yba1"}
{"gilded":0,"author_flair_text":null,"score_hidden":false,"body":"Alright I'm done.","author":"gigaquack","score":3,"link_id":"t3_5yba3","name":"t1_c0299az","retrieved_on":1427426409,"author_flair_css_class":null,"subreddit":"reddit.com","edited":false,"controversiality":0,"ups":3,"parent_id":"t1_c0299ax","created_utc":"1192450691","archived":true,"downs":0,"subreddit_id":"t5_6","id":"c0299az","distinguished":null}
{"distinguished":null,"subreddit_id":"t5_6","id":"c0299b0","downs":0,"archived":true,"created_utc":"1192450696","parent_id":"t3_5yba3","ups":12,"controversiality":0,"edited":false,"subreddit":"reddit.com","author_flair_css_class":null,"retrieved_on":1427426409,"name":"t1_c0299b0","score":12,"link_id":"t3_5yba3","author":"igiveyoumylife","score_hidden":false,"body":"Is anyone else's \"recommended\" page completely empty? Mine is, and it is usually jam-packed.","author_flair_text":null,"gilded":0}
{"downs":0,"archived":true,"distinguished":null,"id":"c0299b1","subreddit_id":"t5_6","ups":1,"controversiality":0,"parent_id":"t1_c02999j","created_utc":"1192450700","author_flair_css_class":null,"name":"t1_c0299b1","retrieved_on":1427426409,"subreddit":"reddit.com","edited":false,"body":"[deleted]","score_hidden":false,"author_flair_text":null,"gilded":0,"link_id":"t3_5yba3","score":1,"author":"[deleted]"}
{"parent_id":"t3_5yba3","created_utc":"1192450705","controversiality":0,"ups":2,"id":"c0299b2","subreddit_id":"t5_6","distinguished":null,"archived":true,"downs":0,"author":"[deleted]","score":2,"link_id":"t3_5yba3","author_flair_text":null,"gilded":0,"score_hidden":false,"body":"Ok, I guess we need to submit comments to test? ","edited":false,"subreddit":"reddit.com","retrieved_on":1427426409,"name":"t1_c0299b2","author_flair_css_class":null}
{"name":"t1_c0299b3","retrieved_on":1427426409,"author_flair_css_class":null,"subreddit":"reddit.com","edited":false,"author_flair_text":null,"gilded":0,"body":"I can't submit any stories -- even the first time, i get, \"you are trying to submit too fast\" .. anyone else seeing this?\n\nEdit: This appears to be fixed.","score_hidden":false,"author":"raldi","score":5,"link_id":"t3_5yba3","archived":true,"downs":0,"id":"c0299b3","subreddit_id":"t5_6","distinguished":null,"controversiality":0,"ups":5,"parent_id":"t3_5yba3","created_utc":"1192450709"}
{"archived":true,"downs":0,"subreddit_id":"t5_6","id":"c0299b4","distinguished":null,"controversiality":0,"ups":1,"created_utc":"1192450737","parent_id":"t1_c0299a5","retrieved_on":1427426409,"name":"t1_c0299b4","author_flair_css_class":null,"subreddit":"reddit.com","edited":false,"author_flair_text":null,"gilded":0,"body":"Working fine with normal Adblock.","score_hidden":false,"author":"deki","score":1,"link_id":"t3_5yba3"}
{"subreddit_id":"t5_6","id":"c0299b5","distinguished":null,"archived":true,"downs":0,"parent_id":"t3_5yba3","created_utc":"1192450739","controversiality":0,"ups":12,"subreddit":"reddit.com","edited":false,"retrieved_on":1427426409,"name":"t1_c0299b5","author_flair_css_class":null,"author":"jezmck","link_id":"t3_5yba3","score":12,"author_flair_text":null,"gilded":0,"score_hidden":false,"body":"can't see beta.reddit.com from here..."}
{"downs":0,"archived":true,"distinguished":null,"id":"c0299b6","subreddit_id":"t5_6","ups":2,"controversiality":0,"parent_id":"t1_c02999j","created_utc":"1192450748","author_flair_css_class":null,"retrieved_on":1427426409,"name":"t1_c0299b6","subreddit":"reddit.com","edited":false,"body":"I had a problem as well, with my comment being lost. Then I did a \"super-refresh\" (shift-Reload) and it seemed to work.","score_hidden":false,"gilded":0,"author_flair_text":null,"score":2,"link_id":"t3_5yba3","author":"sickofthisshit"}
{"distinguished":null,"id":"c0299b7","subreddit_id":"t5_6","downs":0,"archived":true,"parent_id":"t3_5yba3","created_utc":"1192450748","ups":1,"controversiality":0,"edited":false,"subreddit":"reddit.com","author_flair_css_class":null,"name":"t1_c0299b7","retrieved_on":1427426409,"score":1,"link_id":"t3_5yba3","author":"hfaber","body":"hm?","score_hidden":false,"author_flair_text":null,"gilded":0}
{"subreddit":"reddit.com","edited":false,"retrieved_on":1427426409,"name":"t1_c0299b8","author_flair_css_class":null,"score":1,"link_id":"t3_5yba3","author":"paternoster","author_flair_text":null,"gilded":0,"score_hidden":false,"body":"I think so...","id":"c0299b8","subreddit_id":"t5_6","distinguished":null,"archived":true,"downs":0,"parent_id":"t1_c0299b2","created_utc":"1192450770","ups":1,"controversiality":0}
{"distinguished":null,"id":"c0299b9","subreddit_id":"t5_6","downs":0,"archived":true,"parent_id":"t1_c0299ac","created_utc":"1192450771","ups":2,"controversiality":0,"subreddit":"reddit.com","edited":false,"author_flair_css_class":null,"name":"t1_c0299b9","retrieved_on":1427426409,"link_id":"t3_5yba3","score":2,"author":"igiveyoumylife","score_hidden":false,"body":"well... I read Ron Paul's website.","gilded":0,"author_flair_text":null}
{"parent_id":"t1_c0299ap","created_utc":"1192450772","controversiality":0,"ups":2,"distinguished":null,"subreddit_id":"t5_6","id":"c0299ba","downs":0,"archived":true,"author":"zoomzoom83","link_id":"t3_5yba3","score":2,"score_hidden":false,"body":"No","gilded":0,"author_flair_text":null,"edited":false,"subreddit":"reddit.com","author_flair_css_class":null,"name":"t1_c0299ba","retrieved_on":1427426409}
{"gilded":0,"author_flair_text":null,"body":"Cry me a river.  Most patents and\r\ncopyrights these days are theft  of\r\npublic domain under the color of\r\nlaw, to an absurd degree.\r\n\r\nMickey Mouse copyright extended to\r\n100 years as a favor to everyone's\r\nfavorite political donor, Will Eisner.\r\nAmazon's one click patent.\r\n\r\nIt's probably been 20 years \r\nsince the Constitutional authority\r\nto grant patents and copyrights\r\nactually served, on the whole, to\r\n\"advance the arts and commerce\",\r\nrather than to stifle them.\r\n","score_hidden":false,"link_id":"t3_2zxa6","score":1,"author":"donh","retrieved_on":1427426409,"name":"t1_c0299bb","author_flair_css_class":null,"subreddit":"reddit.com","edited":false,"ups":1,"controversiality":0,"parent_id":"t3_2zxa6","created_utc":"1192450781","archived":true,"downs":0,"id":"c0299bb","subreddit_id":"t5_6","distinguished":null}
{"parent_id":"t1_c0299b2","created_utc":"1192450789","controversiality":0,"ups":1,"distinguished":null,"subreddit_id":"t5_6","id":"c0299bc","downs":0,"archived":true,"author":"tashbarg","link_id":"t3_5yba3","score":1,"body":"jup","score_hidden":false,"author_flair_text":null,"gilded":0,"edited":false,"subreddit":"reddit.com","author_flair_css_class":null,"name":"t1_c0299bc","retrieved_on":1427426409}
{"retrieved_on":1427426409,"name":"t1_c0299bd","author_flair_css_class":null,"subreddit":"reddit.com","edited":false,"gilded":0,"author_flair_text":null,"body":"As long as we can still say \"Web 2.0\"...","score_hidden":false,"link_id":"t3_5yba3","score":2,"author":"newton_dave","archived":true,"downs":0,"subreddit_id":"t5_6","id":"c0299bd","distinguished":null,"ups":2,"controversiality":0,"parent_id":"t1_c0299ap","created_utc":"1192450790"}
{"ups":3,"controversiality":0,"parent_id":"t1_c0299b7","created_utc":"1192450797","downs":0,"archived":true,"distinguished":null,"id":"c0299be","subreddit_id":"t5_6","score_hidden":false,"body":"what's new except the asynchronous submit (which alone isn't really worth being dubbed 'new comment system')?","author_flair_text":null,"gilded":0,"link_id":"t3_5yba3","score":3,"author":"hfaber","author_flair_css_class":null,"name":"t1_c0299be","retrieved_on":1427426409,"edited":false,"subreddit":"reddit.com"}
{"subreddit":"reddit.com","edited":false,"author_flair_css_class":null,"retrieved_on":1427426409,"name":"t1_c0299bf","score":6,"link_id":"t3_5yba3","author":"paternoster","score_hidden":false,"body":"Yep","gilded":0,"author_flair_text":null,"distinguished":null,"id":"c0299bf","subreddit_id":"t5_6","downs":0,"archived":true,"parent_id":"t1_c0299b0","created_utc":"1192450809","ups":6,"controversiality":0}
{"name":"t1_c0299bg","retrieved_on":1427426409,"author_flair_css_class":null,"subreddit":"reddit.com","edited":false,"author_flair_text":null,"gilded":0,"score_hidden":false,"body":"[deleted]","score":1,"link_id":"t3_5yba3","author":"[deleted]","archived":true,"downs":0,"id":"c0299bg","subreddit_id":"t5_6","distinguished":null,"ups":1,"controversiality":0,"parent_id":"t3_5yba3","created_utc":"1192450827"}
{"subreddit_id":"t5_6","id":"c0299bh","distinguished":null,"archived":true,"downs":0,"created_utc":"1192450831","parent_id":"t1_c0299az","controversiality":0,"ups":2,"subreddit":"reddit.com","edited":false,"retrieved_on":1427426409,"name":"t1_c0299bh","author_flair_css_class":null,"author":"newton_dave","link_id":"t3_5yba3","score":2,"author_flair_text":null,"gilded":0,"body":"You can stop anytime you want.\n\nReally.","score_hidden":false}
{"distinguished":null,"id":"c0299bi","subreddit_id":"t5_6","downs":0,"archived":true,"created_utc":"1192450849","parent_id":"t1_c02999p","controversiality":0,"ups":3,"subreddit":"reddit.com","edited":false,"author_flair_css_class":null,"retrieved_on":1427426409,"name":"t1_c0299bi","author":"lyon","link_id":"t3_5yba3","score":3,"body":"hmm only worked after refresh...","score_hidden":false,"gilded":0,"author_flair_text":null}
{"body":"okay, dokay, lets see how the comment work now ...  PING !!!","score_hidden":false,"author_flair_text":null,"gilded":0,"link_id":"t3_5yba3","score":2,"author":"joe24pack","author_flair_css_class":null,"name":"t1_c0299bj","retrieved_on":1427426409,"subreddit":"reddit.com","edited":false,"ups":2,"controversiality":0,"parent_id":"t3_5yba3","created_utc":"1192450869","downs":0,"archived":true,"distinguished":null,"id":"c0299bj","subreddit_id":"t5_6"}
{"distinguished":null,"subreddit_id":"t5_6","id":"c0299bk","downs":0,"archived":true,"created_utc":"1192450870","parent_id":"t3_5yba3","controversiality"
$\endgroup$
  • 3
    $\begingroup$ There are gazillion ways to mine this data. Do you have a problem statement in mind? $\endgroup$ – Dawny33 Dec 15 '15 at 3:47
3
$\begingroup$

Data Science is not an algorithm to run on your data. It is the process that helps you answer a specific question. The key to be a data scientist is to ask the right questions. So, first, since you want to be familiar with machine learning, examine your data and try to understand what they can answer for you.

Examples:

  1. Can I cluster the reddit questions in categories?
  2. Can you predict the score of a question based on the number of comments?
  3. Can you predict the sub-reddit that a user will ask a question?
| improve this answer | |
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.