I have a list of Pandas DataFrames and I want to perform some operations on them. To be more precise, I want to clean their names and add new column. So I have written the following code:

import numpy as np
import pandas as pd
from janitor import clean_names
rng = np.random.RandomState(2019)
dataset = [pd.DataFrame(rng.randint(0, 10, (3, 3)), columns = ['Column A', 'Column B', 'Column C']) for i in range(4)]
for df in dataset:df = df.clean_names()df['column_d'] = df['column_a'] + df['column_b']

However, the changes are not preserved despite explicite assignment. The following code returns the original DataFrame:

dataset[1]Column A    Column B    Column C
0          8           5           3
1          0           2           5
2          7           8           5

What am I missing?

This is what is happening:

for df in dataset:

This makes df to refer to an item in the list in each iteration.

df = df.clean_names()

df.clean_names() returns a new object, different from df itself. The assignment makes df to refer to that new object instead of the original.

df['column_d'] = df['column_a'] + df['column_b']

This changes df in place but it is not the original object, the original remains untouched.

You should get what you want by using an index to the list to actually replace each item. Something like this:

for i, df in enumerate(dataset):df = df.clean_names()df['column_d'] = df['column_a'] + df['column_b']dataset[i] = df

Not the prettiest thing in the world but I don't have time to think of something better.