I'm trying to run add_datepart() which converts a df column from a datetime64 to many columns in place
Year', 'Month', 'Week', 'Day', 'Dayofweek', 'Dayofyear','Ismonthend', 'Ismonthstart', 'Isquarterend', 'Isquarterstart', 'Isyearend', 'Isyearstart' . etc
I'm using is Grocery Sales dataset. Total dates i believe in that are 125497040. What should I do to run this operation?
Every time I run this piece of code. The kernel dies(out of memory which is 17.2 GB RAM)
So I tried breaking down this data frame in smaller parts and then running
add_datepart but still the same result
I wrote this code
def add_datepart_large(temp_df, size): list_df = [temp_df[i:i+size] for i in range(0,temp_df.shape,size)] for i in range(len(list_df)): add_datepart(list_df[i], 'date') return pd.concat(list_df) final_df = add_datepart_large(df_all, 100000)
If after running this code, the kernel dies. What's wrong?