I have a question about memory usage.
I want to do 4 things:
1) make a dataframe from one of several columns from a datasource, say a json string 2) make the third column of the original dataset the index to the dataframe 3) change the name of another column 4) change the series i've created to a dataframe
My question is about memory efficiency. It seems that for step 1), I am first loading a whole dataframe, then run a concat command to concatenate the columns I want.
For step 2, I again need to resave the new dataframe as another object.
For step 3, it seems to stick so nothing there.
Please advise on a more efficient way to go about this, if that exists.
df = pd.DataFrame(jsonobject) df = df.set_index("columnC") df.index.names= ["foo"] df1 = df["foo"].map(lambda x:x["id"]) df2 = pd.DataFrame(df1)