3
$\begingroup$

Using Python's sklearn module,

 from sklearn.metrics import classification_report
 y1_predict = [0, 1, 1, 0]
 y1_dev = [0, 1, 1, 0]
 report_1 = classification_report(y1_dev, y1_predict)
 y2_predict = [1, 0, 1, 0]
 y2_dev = [1, 1, 0, 0]
 report_2 = classification_report(y2_dev, y2_predict)

Is there a way to combine (maybe just an average) report_1 and report_2 I'm looking for an implementation like:

 report_average = average(report_1,report_2)

Or does this have to be done manually? I was hoping that printing the report_average would have average values between the two reports.

Here's a MWE of the accepted answer:

    from sklearn.metrics import classification_report
    import pandas as pd
    import numpy as np
    from functools import reduce
    def report_average(*args):
        report_list = list()
        for report in args:
            splited = [' '.join(x.split()) for x in report.split('\n\n')]
            header = [x for x in splited[0].split(' ')]
            data = np.array(splited[1].split(' ')).reshape(-1, len(header) + 1)
            data = np.delete(data, 0, 1).astype(float)
            avg_total = np.array([x for x in splited[2].split(' ')][3:]).astype(float).reshape(-1, len(header))
            df = pd.DataFrame(np.concatenate((data, avg_total)), columns=header)
            report_list.append(df)
        res = reduce(lambda x, y: x.add(y, fill_value=0), report_list) / len(report_list)
        return res.rename(index={res.index[-1]: 'avg / total'})


    y1_predict = [0, 1, 1, 0]
    y1_dev = [0, 1, 1, 0]
    report_1 = classification_report(y1_dev, y1_predict)
    y2_predict = [1, 0, 1, 0]
    y2_dev = [1, 1, 0, 0]
    report_2 = classification_report(y2_dev, y2_predict)

    report_ave = report_average(report_1,report_2)

    print(report_ave)

Which yields

             precision  recall  f1-score  support
0                 0.75    0.75      0.75      2.0
1                 0.75    0.75      0.75      2.0
avg / total       0.75    0.75      0.75      4.0
$\endgroup$
  • $\begingroup$ I would have made a comment but I do not have enough reputation. If you set output_dict to True for classification_report() my function will take a list of dics and the label names and returns a pandas DataFrame with each cell containing a string "mean +- std" over all the dics you passed. def report_average(class_reports=None, label_names=None): sum_up = np.zeros((len(class_reports),len(label_names)+3,4)) for i, report in enumerate(class_reports): j = 0 for class_name, results in report.items(): h = 0 for metric, value in results.items(): sum_up[i,j,h] = value h += 1 j += 1 report_mean = np.m $\endgroup$ – Hannes Feb 22 '19 at 16:35
1
$\begingroup$

It maybe a little bit complicated, since I convert the reports to pandas.DataFrame for calculation. But I think it's worth it, because it works well with two or more report as well. Try below:

import pandas as pd
import numpy as np
from functools import reduce
def report_average(*args):
    report_list = list()
    for report in args:
        splited = [' '.join(x.split()) for x in report.split('\n\n')]
        header = [x for x in splited[0].split(' ')]
        data = np.array(splited[1].split(' ')).reshape(-1, len(header) + 1)
        data = np.delete(data, 0, 1).astype(float)
        avg_total = np.array([x for x in splited[2].split(' ')][3:]).astype(float).reshape(-1, len(header))
        df = pd.DataFrame(np.concatenate((data, avg_total)), columns=header)
        report_list.append(df)
    res = reduce(lambda x, y: x.add(y, fill_value=0), report_list) / len(report_list)
    return res.rename(index={res.index[-1]: 'avg / total'})

output:

report_average  = report_average(report_1, report_2)
print(report_average)
             precision  recall  f1-score  support
0                 0.75    0.75      0.75      2.0
1                 0.75    0.75      0.75      2.0
avg / total       0.75    0.75      0.75      4.0

report_3 = report_2
report_average  = report_average(report_1, report_2,report_3)
print(report_average)
             precision    recall  f1-score  support
0             0.666667  0.666667  0.666667      2.0
1             0.666667  0.666667  0.666667      2.0
avg / total   0.666667  0.666667  0.666667      4.0
| improve this answer | |
$\endgroup$
  • $\begingroup$ Looks a bit long, but seems to work ok, thanks! $\endgroup$ – Charles May 9 '18 at 23:21
0
$\begingroup$

Just another way to do this when the reports (as_dict) are passed as list. This will return the result as a dictionary.

def report_average(reports):
    mean_dict = dict()
    for label in reports[0].keys():
        dictionary = dict()

        if label in 'accuracy':
            mean_dict[label] = sum(d[label] for d in reports) / len(reports)
            continue

        for key in reports[0][label].keys():
            dictionary[key] = sum(d[label][key] for d in reports) / len(reports)
        mean_dict[label] = dictionary

    return mean_dict
| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.