12
$\begingroup$

I often find myself writing code like the following (oversimplfied example)

df = read_csv('customer_data_export.csv')
df2 = df.query("date > '2017-01-10'")
data = df_filtered.groupby('transaction_id').sum()
plot_data = pivot_table(data, columns='weekday', rows='n_items')
# Etc etc

Basically the problem is that while it's relatively easy to come up with semantic names for columns (as random variables) it's hard for me to come up with meaningful names for each step of transformed dataframe. Additionally I prefer to have short names to make the code easier to type. (Working in Jupyter notebook, the tab-completion isn't the best).

What are some best practices that people follow with this kind of thing?

Improve this question
$\endgroup$
3
  • 2
    $\begingroup$ Method chaining is probably a better alternative if you don't care about the intermediate data frames. It is explained beautifully in this post: tomaugspurger.github.io/method-chaining.html $\endgroup$
    – hssay
    May 16, 2017 at 8:15
  • 1
    $\begingroup$ thanks for the blog link, looks like this is the way to go, (functional chaining for simple transformations, and separate semantically-named functions for more complex transformations, passed through with pipeline) $\endgroup$
    – Max Flander
    May 17, 2017 at 1:20
  • $\begingroup$ by the way why do i have negative votes for this question ? i have been considering this questions for a long time, i actually wrote a blog post about it (pre pipeline) at themrmax.github.io/2015/10/12/… . what could i do to improve my question ? $\endgroup$
    – Max Flander
    May 17, 2017 at 1:22

2 Answers 2

Reset to default
5
$\begingroup$

Why not give them a name describing their purpose?

df_csv = read_csv('customer_data_export.csv')
df_date_filtered = df.query("date > '2017-01-10'")
df_grouped_by_trans_id = df_date_filtered.groupby('transaction_id').sum()

#cleanup
rm(df_csv, df_date_filtered)

plot_data = pivot_table(df_grouped_by_trans_id, columns='weekday', rows='n_items')
Improve this answer
$\endgroup$
0
$\begingroup$

I do add suffix to df_ based on each step.

df_cust = pd.read_csv('customer.csv')
df_clean = df_cust.dropna()

It is also helpful to keep a raw copy of df as df_raw to reference back in case I want to.

df_raw = df_csv.copy()
df_csv.dropna(inplace=True)
Improve this answer
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.