2
$\begingroup$

I have the following code written using the pandas library. I would like to know if there are ways to optimize the code.

for column in df:
 for index, row in df[column].iteritems():
    if type(row) == str:
        if 'R$' in row:
            n = row.replace('R$', '')
            n = n.replace(' ', '')
            n = n.replace('.', '')
            df[column].iloc[index] = float(n)

Just want to remove unwanted string parts.

$\endgroup$
2
$\begingroup$

You can replace the symbol in dataframe without iterating yourself.

df = df.replace({'R\$': ''}, regex=True)

Then change the type of columns that can be numeric. If you don't know which are those columns, use this that will automatically change the type to numeric and ignore those that cannot be changed.

df = df.apply(pd.to_numeric, errors='ignore')

When you use replace and in general many other pandas features, it doesn't update your dataframe. It creates a new, temporary dataframe. So, you either need to assign it back to your original dataframe or use inplace=True wherever it is available, like:

df.replace({'R\$': ''}, regex=True, inplace = True)

$ is a special character in regex, so you need to escape it. That's why the backslash before it.

import pandas as pd

dic = {'feature1': 'R$ aaa bb', 'feature2': 1} 
df = pd.DataFrame(dic, index=[0,1])

print(df)

>>  feature1  feature2
0  R$ aaa bb         1
1  R$ aaa bb         1

df = df.replace({'R\$': ''}, regex=True)

print(df)

>>  feature1  feature2
0   aaa bb         1
1   aaa bb         1
$\endgroup$
  • $\begingroup$ For some reason it does not work. $\endgroup$ – Rodrigo Nader Nov 1 '17 at 7:56
  • $\begingroup$ Do you have any error? Just keep in mind that you need to assign back the df... Updated my answer $\endgroup$ – Tasos Nov 1 '17 at 8:01
  • $\begingroup$ No, it just doesn't work. The 'R$' are still there. $\endgroup$ – Rodrigo Nader Nov 1 '17 at 8:02
  • $\begingroup$ Check my updated answer. If you still have the same behaviour, please provide a sample of your dataframe so we can replicate it. $\endgroup$ – Tasos Nov 1 '17 at 8:05
  • $\begingroup$ Yes, I tried that as well, but it's not the case, still not working. We can check this by simply creating a dataframe like this: dic = {'feature1': 'R$ aaa bb', 'feature2': 1} df = pd.DataFrame(dic, index=[A,B]) I tried it to see if was some problem with the dataframe, but it still didn't work. $\endgroup$ – Rodrigo Nader Nov 1 '17 at 8:19

Not the answer you're looking for? Browse other questions tagged or ask your own question.