# How to reshape or clean data to be able to visualize it with violin plots?

My end goal is to visualize some data using a violin plot or something similar using Python.

I have the following data in a file (test.csv). The first column is a list of species. The other columns determine abundance of the species at a certain latitude (e.g. how abundant is species A at altitude 1000, 2000?). (Ignoring units for now.) How can I plot this as a violin plot (or something similar)?

test.csv

species,1000,2000,3000,4000,5000,6000,7000
species_A,0.5,0.5,,,2,1,2
species_B,0.5,1,0.5,0.5,1,1,10
species_C,1,1,10,3,15,4,5
species_D,15,3,2,1,0.5,1,3


The Python code I tried so far is below. This does not work because it only plots the distribution of altitudes, which is the same for all species (because they were all sampled from the same set of altitudes).

file = "test.csv"

# convert columns to list
colnames = list(df.columns)
colnames.remove("species")

# Transform the data so that I have a dataframe with only three columns: species, Altitude, and Count
df = pd.melt(df, id_vars=['species'], value_vars=colnames, value_name="Count", var_name="Altitude")
df.species = df.species.astype('category')
df.Altitude = df.Altitude.astype('int')

# Plot the data
sns.violinplot(x="species", y="Altitude", data=df)
plt.title("Abundance of Species at Various Altitudes")
plt.grid(alpha=0.5, ls="--")
plt.xticks(rotation=90)

# show graph
plt.show()
$$$$

• It sounds like you have three variables: species, location, and abundance. Is this correct? Such data are not amenable to a box plot or violin plot, as those would be for just the categorical species variable and one of the numerical variables. My first visualization would be a scatter plot of the location and abundance with different colors and/or shapes for the species: lions in red, tigers in blue, and bears in black, for instance.
– Dave
Jun 15 at 9:53
• @Dave My data counts the number of species at a given altitude: x species A at altitude y. To me (at least), it doesn't seem too different than the life_exp vs continent plot this this tutorial. Jun 15 at 16:10
• The x-axis in their chart is the continent; the y-axis is the life_exp. You have a third dimension. I still recommend a bivariate scatter plot with your categorical variable denoted by colors, shapes, or both. That captures all three of your variables.
– Dave
Jun 15 at 16:12
• @Dave My data could be reshaped to two variables: species and location. Using my example data: there would be 2 entries for species A at altitude 7000, 1 entry at altitude 6000, etc. (The 0.5 entries can either be omitted or rounded up/down.) Jun 15 at 16:13
• Then make a violin chart with categorical species on the x-axis and numerical altitude on the y-axis. That is analogous to the plot in the link, but it does omit the abundance variable.
– Dave
Jun 15 at 16:15

I ended up creating a new Pandas DataFrame using the code below. I wash hoping for something simpler or more elegant.

# Create a new dataframe
df_2d = pd.DataFrame()
for _, sp in df.iterrows():
count = 0 if np.isnan(sp['Count']) else int(np.ceil(sp['Count']))
df_2d = df_2d.append([{"species": sp["species"], "Altitude": sp["Altitude"]}] * count)


You can make the "ungrouped" dataframe by reindexing on a repeated index:

df_2d = df.loc[df.index.repeat(
df["Count"].fillna(0).astype(int)
)]
`

There should be a more direct way to generate a plot, but I don't know it. That your latitudes are discretized might not help.