I have a dataset of patients from which I want to predict whether patient suffering from diabetes or not.
In that I have a DataFrame res_total_Df with columns suppose
[patient_id, urine output, Blood pressure] and
another DataFrame key_df with columns suppose
[patient_id , haemoglobin, Blood pressure].
I want to merge into single dataFrame in which common columns values should be added as list(for which later I would take mean). Resultant dataFrame would be
[patient_id, urine output, haemoglobin, Blood pressure].
How can I achieve it using pandas. the number of columns in second dataFrame can vary because I am extracting them from the text. I cannot paste the dataFrame snapshot because it is health data.
Here is my code
def find_common_col(res_total_df, key_df): # patient_id is always common common_list = ['patient_id'] for i in res_total_df.columns: if i in key_df.columns: if i not in common_list: common_list.append(i) common_col = find_common_col(res_total_Df, key_df) res_total_Df = pd.merge(res_total_Df, key_df, how='outer', on=common_col)
The problem with above code it has duplicates rows. Suppose Blood pressure value for patient_id
1993 in res_total_Df is 180 and in key_df is 200. Then it adds two rows one with value 180 and other with value 200 for patient_id 1993.
It want Blood pressure for patient_id 1993 as [180, 200].