2
$\begingroup$

Given a pandas dataframe with a timestamp index, sorted. I have a label and I need to find the closest index to that label. Also, I need to find a smaller timestamp, so the search should be computed in the minor timestamps. Here is my code:

import pandas as pd
import datetime

data = [i for i in range(100)]
dates = pd.date_range(start="01-01-2018", freq="min", periods=100)
dataframe = pd.DataFrame(data, dates)

label = "01-01-2018 00:10:01"
method = "pad"
tol = datetime.timedelta(seconds=60)
idx = dataframe.index.get_loc(key=label, method="pad", tolerance=tol)

print("Closest idx:"+str(idx))
print("Closest date:"+str(dataframe.index[idx]))

the searching is too slow. Is there a way to improve it?

$\endgroup$
1
$\begingroup$

This probably doesn't take into account that dates are sorted and thus performs as O(n). Try using binary-search on dates, that would perform as O(log(n)) (avoid implementing your own, look for a standard module, maybe it exists in numpy).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.