I want to make the following Python data-processing code more efficient by replacing for loops. Is there any way to vectorize code like this?
I have a DataFrame object
df
that looks somewhat like:names number bob 5 sara 10 bob 8 foo 12 moo 16
I want to subset the DataFrame to find out all the rows associated with each name, and then perform an operation on
number
. This is what I am doing now:for myName in set(df['names']): nameSubset = df.loc[df['names']==myName] operation(nameSubset['number'], **args) '''Basically,perform an operation on the `number` column of nameSubset.'''
Is there any way to make this code run faster? Theoretically, this could be made faster if, instead of running through each myName at a time, the computer could process several myNames at any given moment. I'm not sure how to vectorize/parallelize my code to make this happen though.
group_by(names)
, then do an operation onnumber
- is that an aggregation/summary operation (e.g. sum, max, count) or an operation on each individual number, e.g. compute some value)? $\endgroup$