12
$\begingroup$

I often find myself writing code like the following (oversimplfied example)

df = read_csv('customer_data_export.csv')
df2 = df.query("date > '2017-01-10'")
data = df_filtered.groupby('transaction_id').sum()
plot_data = pivot_table(data, columns='weekday', rows='n_items')
# Etc etc

Basically the problem is that while it's relatively easy to come up with semantic names for columns (as random variables) it's hard for me to come up with meaningful names for each step of transformed dataframe. Additionally I prefer to have short names to make the code easier to type. (Working in Jupyter notebook, the tab-completion isn't the best).

What are some best practices that people follow with this kind of thing?

$\endgroup$
3
  • 2
    $\begingroup$ Method chaining is probably a better alternative if you don't care about the intermediate data frames. It is explained beautifully in this post: tomaugspurger.github.io/method-chaining.html $\endgroup$
    – hssay
    Commented May 16, 2017 at 8:15
  • 1
    $\begingroup$ thanks for the blog link, looks like this is the way to go, (functional chaining for simple transformations, and separate semantically-named functions for more complex transformations, passed through with pipeline) $\endgroup$ Commented May 17, 2017 at 1:20
  • $\begingroup$ by the way why do i have negative votes for this question ? i have been considering this questions for a long time, i actually wrote a blog post about it (pre pipeline) at themrmax.github.io/2015/10/12/… . what could i do to improve my question ? $\endgroup$ Commented May 17, 2017 at 1:22

2 Answers 2

5
$\begingroup$

Why not give them a name describing their purpose?

df_csv = read_csv('customer_data_export.csv')
df_date_filtered = df.query("date > '2017-01-10'")
df_grouped_by_trans_id = df_date_filtered.groupby('transaction_id').sum()

#cleanup
rm(df_csv, df_date_filtered)

plot_data = pivot_table(df_grouped_by_trans_id, columns='weekday', rows='n_items')
$\endgroup$
0
$\begingroup$

I do add suffix to df_ based on each step.

df_cust = pd.read_csv('customer.csv')
df_clean = df_cust.dropna()

It is also helpful to keep a raw copy of df as df_raw to reference back in case I want to.

df_raw = df_csv.copy()
df_csv.dropna(inplace=True)
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.