Replacing rows of dataframe with rows of another dataframe that have the same index

Question

I have a dataframe that has rows with indices 0 to 128 and a smaller dataframe with indices 4, 8, 105, and 107.

I made edits to the rows in the smaller dataframe and am now trying to replace rows indexed 4, 8, 105, and 107 in the large dataframe with rows indexed 4, 8, 105, and 107 in the smaller dataframe.

Why can I not just do:

bigDF[smallDF.index] = smallDF

How would I accomplish this replacement? Thank you!

Answer 1

pandas.DataFrame.update is what you are looking for. This modifies in place the provided DataFrame using non-NA values from another DataFrame. Use overwrite=True if you want to copy also np.nan values. Aligns on indices, meaning that in your case it overwrites all rows with matching indices.

>>> df = pd.DataFrame({'A': [1, 2, 3],
...                    'B': [400, 500, 600]})
>>> new_df = pd.DataFrame({'B': [4, 5, 6],
...                        'C': [7, 8, 9]})
>>> df.update(new_df)
>>> df
   A  B
0  1  4
1  2  5
2  3  6

In your case:

bigDF.update(smallDF)

Answer 2

In your code, you passed smallDF.index directly. The interpreter considers this as a list of column names. So your code will be interpreted like below. But 4,8,105,107 are not columns. They are indexes.

bigDF[[4,8,105,107] = smallDF

To locate a row based on the index, use the loc function. Try using below.

bigDF.loc[smallDF.index]=smallDF