Appending DataFrames to lists in a dictionary - why does it seem like the list is being referenced by each new DataFrame?

Question

I have a DataFrame that pairs one or more labels to a sample group and id, for a given sample stored in a database at SampleGroup/SampleID:

There are ~100 labels. I want to create binary models to do classification on each label, and then run these models in parallel to do multi-class classification. To store these models, I am creating a dictionary of form

{label_1:[df_1, model_object_1],
 label_2:[df_2, model_object_2],
...,
label_n:[df_n, model_object_n]
}

Where each df is a DataFrame of the form above, except that the value of the 'Labels' column is replaced with a 1 or 0, depending on whether dictionary key 'label_i' is in the original label list for that row. Here's the code that (should) do that, that has been giving me some trouble:

models = dict.fromkeys(target_labels, [])

for label in target_labels:
    label_list = []
    for multi_label_list in df['Labels']:
        if label in multi_label_list:
            label_list.append(1)
        else:
            label_list.append(0)

    data = {
        'SampleGroup':df['SampleGroup'].copy(),
        'SampleID':df['SampleID'].copy(), 
        'Labels':label_list
    }

    models[label].append(pd.DataFrame(data=data, index=df.index))
    print(len(models[label]))

When I run this, each new binary label_list that is created for a label gets appended to every model in the dictionary, as if I'm creating a reference to the same label_list (similar to how df2 = df would create a reference to df, instead of a copy). The output of the above code tells the story clearly:

[

I managed to hack a fix for this by assigning each new DataFrame to the key instead of appending it to the key's value list:

models[label] = (pd.DataFrame(data=data, index=df.index))

What property of DataFrames (or perhaps native Python) am I invoking that would cause this to work fine, but appending to a list to act strangely?

Answer 1

The problem is when you create the dictionary models using models = dict.fromkeys(target_labels, []), you actaully only created one empty list, and all keys point to that list. Everything you append goes into that same list.

For example,

models = dict.fromkeys('abcd', [])
print(models)
print(models['a'] is models['b'])
models['a'].append(3)
print(models)

will return

{'d': [], 'a': [], 'b': [], 'c': []}
True
{'d': [3], 'a': [3], 'b': [3], 'c': [3]}

Notice that models['a'] is models['b'] is True.

You can create the dictionary instead using

models = dict([(key, []) for key in 'abcd'])
print(models)
print(models['a'] is models['b'])
models['a'].append(3)
print(models)

Now it returns

{'d': [], 'a': [], 'b': [], 'c': []}
False
{'d': [], 'a': [3], 'b': [], 'c': []}

Notice that models['a'] is not models['b'].