1
$\begingroup$

I have a dataset of tens of thousands of appointments. Appointments have a created date and scheduled date. Something like this:

ID   Created      Scheduled
1    08/01/2020   08/05/2020
2    08/01/2020   08/07/2020
3    08/02/2020   08/04/2020
...

I'm trying to predict the probability of all possible schedule dates based on a created date in the future. So basically, if a customer created an appointment tomorrow (August 20th), what's the probability they would schedule their appointment for August 21th, August 22nd, August 23rd, etc. In theory, customers can create appointments into perpetuity but practically no one makes an appointment more than ~2 months in advance.

Some observations I've made are:

  • Customers prefer making appointments on weekends
  • Customers prefer making appointments during the last or first few days of the month
  • Most appointments are scheduled within 2 weeks of the day the customer creates the appointment

I've been struggling with this problem. I first tried to just look at how many days out a customer schedules their appointment. Something like 10% of the time it's the next day, 15% of the time it's 2 days out, etc. But that didn't account for how customers preferred making appointments on weekends and the start/end of the month. So it was incredibly inaccurate.

I'm frankly stumped about how to approach this problem. I would appreciate it if people have ideas on how I can go about this. Thank you! Please let me know if anything here is unclear.

Improve this question
$\endgroup$

3 Answers 3

Reset to default
0
$\begingroup$

Create new features specifying days of the week.

df['Created'] = pd.to_datetime(df['Created'])
df['Scheduled'] = pd.to_datetime(df['Scheduled'])
df['Scheduled_day_of_week'] = df['Scheduled'].dt.day_name()
df['Created_day_of_week'] = df['Created'].dt.day_name()

Create a new column day difference which will be the difference between the two days.

Then you can visualize for each day what would be the day difference. Similarly you can try for the date.

This process is called Feature Engineering.

Improve this answer
$\endgroup$
6
  • $\begingroup$ Thanks! These features definitely make sense. Going back to the August 20th question, would the idea be to predict when the appointments will be scheduled for by looking at how previous data I have for historical data I have for how appointments made on the 20th of a month on a Thursday were scheduled? $\endgroup$
    – Lakshay Akula
    Aug 20, 2020 at 20:58
  • $\begingroup$ Yeah you should make predictions on observational data to see the trend in booking appointment. $\endgroup$
    – prashant0598
    Aug 21, 2020 at 15:56
  • $\begingroup$ Is there a more intelligent model I could use? I tried this, but it's not very accurate since there aren't many days of samples to use. For August 20th, for example, there are two days in the last couple years which are also Thursdays on the 20th. [Link to data on Thursdays on the 20th] (timeanddate.com/calendar/weekday-thursday-20) $\endgroup$
    – Lakshay Akula
    Aug 21, 2020 at 16:15
  • $\begingroup$ Don't look for particular day date combination rather use day as variable and predict the date difference which will ultimately give you result. Hope this answers your question. $\endgroup$
    – prashant0598
    Aug 21, 2020 at 16:27
  • $\begingroup$ Thanks! The challenge is that if I only use the day of the month as a variable, I'm missing out on the fact that customers like to book on the weekend, so my predictions will be inaccurate. $\endgroup$
    – Lakshay Akula
    Aug 21, 2020 at 20:38
0
$\begingroup$

Just imagine practically when and why a customer would like to make their appointment on a particular day.
There can be many reasons :

  1. Which day is it : Customer are usually free on weekends.
  2. *Price * : Suppose the prices are usually low at a month's end.
  3. Time between appointment being made and scheduled.

You have to take these in account for a model to be accurate and effective.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks! Price isn't relevant in my case, but these definitely are important features. What options would you recommend in terms of models? $\endgroup$
    – Lakshay Akula
    Aug 20, 2020 at 20:59
  • $\begingroup$ Random Forests,SVM's are great to use if you have somewhat less data. You can use ANN if you have sufficient data. $\endgroup$
    – Shiv
    Aug 23, 2020 at 6:54
  • $\begingroup$ Can I use a random forest or SVM if the problem isn't a classification problem? $\endgroup$
    – Lakshay Akula
    Oct 12, 2020 at 4:06
  • $\begingroup$ I wrote svm and random forest as a general example. They are classification algorithms. $\endgroup$
    – Shiv
    Oct 14, 2020 at 7:35
0
$\begingroup$

A neural network can be trained with x as created date and y as appnt scheduled date.

This is a complex dataset so a neural network regression is recommended. Convert all dates into numerical figures in days from a common reference origin like 1/1/1970 or whichever was start date of your clinic.

Neural Network logits output would predict likely appnt scheduled date

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.