1
$\begingroup$

My dataframe contains 7000 rows with the following library login details: login time, libraryid (userid), login date, fined/not.

This data is for a local library, I want to create a model which predicts a persons visit to library. Say, the chance that the member with libraryid=1015 will visit the library tomorrow. The library has had many regular visitors over the past 2-3 yrs.

I tried the following:

clf = RandomForestClassifier(n_estimators=30)
clf.fit(df_train[features], df_train['a'])

Since the date is involved it is not getting the required result.

       login_time             id    login_date  fined
0      2016-02-29 23:28:58    1015  2016-02-29  1
1      2016-03-01 00:19:27    4890  2016-03-01  1
2      2016-03-01 04:40:17    1020  2016-03-01  0
3      2016-03-01 04:41:05    9134  2016-03-01  1
4      2016-03-01 05:00:27    7798  2016-03-01  1
5      2016-03-01 05:01:21    1325  2016-03-01  1
6      2016-03-01 05:02:22    5017  2016-03-01  1
7      2016-03-01 05:05:47    2730  2016-03-01  1
8      2016-03-01 05:09:57    8125  2016-03-01  1
9      2016-03-01 05:10:47    8604  2016-03-01  1
10     2016-03-01 05:16:26    9033  2016-03-01  1
11     2016-03-01 05:17:23    7096  2016-03-01  1
12     2016-03-01 05:18:02    1022  2016-03-01  1
13     2016-03-01 05:19:33    1015  2016-03-01  1
14     2016-03-01 05:19:48    3087  2016-03-01  1
15     2016-03-01 05:21:40    5641  2016-03-01  0
16     2016-03-01 05:28:25    5548  2016-03-01  1
17     2016-03-01 05:31:54    8970  2016-03-01  0
18     2016-03-01 05:58:52    7781  2016-03-01  1
19     2016-03-01 06:09:12    5859  2016-03-01  1
20     2016-03-01 06:13:28    1017  2016-03-01  1
21     2016-03-01 06:14:35    8138  2016-03-01  1
22     2016-03-01 06:15:41    1018  2016-03-01  1
23     2016-03-01 06:17:40    9355  2016-03-01  1
24     2016-03-01 06:18:10   10583  2016-03-01  1
25     2016-03-01 06:20:44    2394  2016-03-01  0
26     2016-03-01 06:29:17   10168  2016-03-01  1
27     2016-03-01 06:31:11    4235  2016-03-01  0

I used ARIMA model, the model is taking too much time for running, any other solutions please.

Improve this question
$\endgroup$
2
  • $\begingroup$ Does 0 represent the time he walked in(swiped in at the library)? Does 2 represent the date? am I right? I think to apply classifier, you need more demographics data for each and every user to use the classifier. if we use time as an input field then the tree built(random forest classifier) would use that as an important feature. $\endgroup$
    – Toros91
    Jan 9, 2018 at 6:37
  • $\begingroup$ yes david 0 is the walkin time and 2 is the respective date. $\endgroup$
    – Sam Joe
    Jan 9, 2018 at 7:40

1 Answer 1

Reset to default
5
$\begingroup$

First and foremost, you need to reformat your data into what's called a balanced panel structure. For each day in your training data, each user should have a record for that day associated with an indicator variable for whether or not they visited. If every record in your training data corresponds to a visit, you're not giving those classifiers much to work with. You're model needs to know what a user's status looks like on days when they don't visit to be able to discriminate between days when they will and won't visit. The indicator variable, denoting "this user visited today", is the target of your prediction. Not the date or time column, if that's what you were trying.

Additionally, you're going to need to do some feature engineering. This process comprises the bulk of time and creativity expended on most data science projects, and the amount of effort you will put in will make or break your project. For each record in the data set, you should calculate things like:

  • What is the day of the week/month/year?
  • how many times did the user visit on this day of the week in the last k days?
  • how many times did the user visit on weekdays/weekends in the last k days?
  • How many days has it been since the users last visit?
  • How many times has the user visited the library in the last k days?
  • how many times has the user visited the library and paid a fine in the last k days?
  • How many users visited in the last k days?
  • Is the library open this day (e.g. it could be a national holiday)?
  • Are the public schools currently in session? On holiday? In finals?
  • How many days since the beginning of the most recent semester?
  • How many days until the semester ends?

Better yet, if you're not limited to what you've shown us...

  • How many books are currently in the users possession? How many are late?
  • If the user visited today, how much would they owe in fines?
  • how many books did they check out in their last visit?
  • how many books have they checked out in the last k days?
  • how long has this person been a member at the library?
  • has their membership expired?
  • in the last k years, have they allowed their membership to expire?
  • have they recently updated their address?
  • How much have they donated in the last k years?
  • What fraction of books checked out in the last 5 years by this user have belonged to each dewey decimal class?

These are just some ideas off the top of my head. Don't let yourself be limited by my suggestions.

If you really only have the data you've shown us, a project you can try that would be more likely to give you good results would be to predict just the number of people who will visit the library on a particular day, rather than which actual people. If this sounds like it might be useful to you, check out Poisson models.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thanks for your detailed reply, I am having only this much data, using the unique library id, prdict. this id may be a regular visitor at a regular time say morning 9:00 am, the data set is obtained from the log, so we have no other info about the other dependencies like holidays etc. if that person is visited the data is stored in the data set. since no other data is available, I not considering other factors. from their data (above) can we make a prediction model? like 1050 visits regularly from past XX days ..with a reliable value... at this time, so 1050 will visit library tomorrow.. $\endgroup$
    – Sam Joe
    Jan 9, 2018 at 8:15
  • 2
    $\begingroup$ If you weren't provided it: holidays is something you can and need to infer. Visit the library website. Check your local holiday calendar. Hell, find days with zero visits. Figure it out. "I'm not considering other factors" is just lazy. The first thirteen suggestions I gave you are all variables you can infer from your current dataset. The bulk of data science isn't throwing different algorithms at the problem, it's combining different data sources and engineering features. Reformat your data as a balanced panel, add an indicator for the target, and start engineering features. $\endgroup$
    – David Marx
    Jan 9, 2018 at 8:15
  • $\begingroup$ sorry David, as i mentioned this a local library and it works all 365 days, i checked the full dataset.that is the reason i have not considered holidays. I can find days with zero visits.also the fine, they deduct from their previous deposit. $\endgroup$
    – Sam Joe
    Jan 9, 2018 at 8:20
  • $\begingroup$ You have the date of the visit: how can you not infer what day of the week it was, or what month, or how many days into the year? You have historical visits: how can you not count the number of times someone visited over a fixed interval? Even if the library hours don't change (they do), you don't think the time of year or holiday affects attendance? I'm not going to do all of this for you: I've already held your hand quite a bit with my long list of suggested features. Whether you put the work in or not is up to you: don't expect the model to work for you if you won't work for it. $\endgroup$
    – David Marx
    Jan 9, 2018 at 8:40
  • $\begingroup$ i am working on arima model, got some results and anlysing it.cool. $\endgroup$
    – Sam Joe
    Jan 9, 2018 at 8:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.