I have the following sample dataset (the actual dataset is over 10 million records)
Passenger Trip 0 Mark London 1 Mike Girona 2 Michael Paris 3 Max Sydney 4 Martin Amsterdam 5 Martin Barcelona 6 Martin Barcelona 7 Mark London 8 Mark Paris 9 Martin New york 10 Max Sydney 11 Max Paris 12 Max Sydney ... ... ...
And I wanted to get the destination frequently travelled by a passenger !
I was playing around in Jupyter and got the expected data with the following approach
series_px = df_px_dest.groupby('Passenger')['Trip'].apply(lambda x: x.value_counts().head(1)) df_px = series_px.to_frame() df_px.index = df_px.index.set_names(['UName', 'DEST']) df_px.reset_index(inplace=True) def getNextPossibleDestByUser(pxname,df=df_px): return df.query('UName==@pxname')['DEST'].to_string(index=False)
While the response is fine. I have few doubts now
1) What's the best way to expose the method (say in this case getNextPossibleDestByUser) as a API (pass customer name as input and get the destination as output) ?
2) Whenever the API is being called , does that mean all the 10 million records gets processed each time ? Are there anyway to optimise that ?
3) Rather than dataframe (pandas) query approach can I consider some ml models or utility functions from say scikit to solve the same problem ?