10
$\begingroup$

I have a dataset in following structure inserted in a CSV file:

Banana  Water   Rice
Rice    Water
Bread   Banana  Juice

Each row indicates a collection of items that were purchased together. For example, the first row denotes that the items Banana, Water, and Rice were purchased together.

I want to create a visualization like the following:

example visualization

This is basically a grid chart but I need some tool (maybe Python or R) that can read the input structure and produce a chart like the above as output.

$\endgroup$
6
$\begingroup$

I think what you probably want is a discrete version of a heat map. For example, see below. The red colors indicate the most commonly purchased together, while green cells are never purchased together. heat map

This is actually fairly easy to put together with Pandas DataFrames and matplotlib.

import numpy as np
from pandas import DataFrame
import matplotlib
matplotlib.use('agg') # Write figure to disk instead of displaying (for Windows Subsystem for Linux)
import matplotlib.pyplot as plt

####
# Get data into a data frame
####
data = [
  ['Banana', 'Water', 'Rice'],
  ['Rice', 'Water'],
  ['Bread', 'Banana', 'Juice'],
]

# Convert the input into a 2D dictionary
freqMap = {}
for line in data:
  for item in line:
    if not item in freqMap:
      freqMap[item] = {}

    for other_item in line:
      if not other_item in freqMap:
        freqMap[other_item] = {}

      freqMap[item][other_item] = freqMap[item].get(other_item, 0) + 1
      freqMap[other_item][item] = freqMap[other_item].get(item, 0) + 1

df = DataFrame(freqMap).T.fillna(0)
print (df)

#####
# Create the plot
#####
plt.pcolormesh(df, edgecolors='black')
plt.yticks(np.arange(0.5, len(df.index), 1), df.index)
plt.xticks(np.arange(0.5, len(df.columns), 1), df.columns)
plt.savefig('plot.png')
| improve this answer | |
$\endgroup$
  • $\begingroup$ Many thanks :) Can I create this using Spark Mllib? $\endgroup$ – João_testeSW Oct 7 '16 at 8:22
  • $\begingroup$ @João_testeSW You probably can, but I'm unfamiliar with Spark. $\endgroup$ – apnorton Oct 7 '16 at 15:08
  • $\begingroup$ did you recommend any IDE for executing this code? $\endgroup$ – João_testeSW Oct 7 '16 at 15:44
  • $\begingroup$ @João_testeSW If you save this in a file as "somescript.py", you can run it with "python3 somescript.py" on the terminal. No IDE needed, but if you load it into some Python-capable IDE it should run. $\endgroup$ – apnorton Oct 7 '16 at 15:47
  • $\begingroup$ thanks ;) I'll see if I can use it into Pyspark, if yes then I can edit the post with the solution ;) $\endgroup$ – João_testeSW Oct 7 '16 at 15:58
3
$\begingroup$

For R, you can use library ArulesViz. There is nice documentation and on the page 12, there is example how to create this kind of visualization.

The code for that is as simple as this:

plot(rules, method="grouped")
| improve this answer | |
$\endgroup$
  • $\begingroup$ While it's not what the OP is looking for, there's a great example visualization using this library here: algobeans.com/2016/04/01/… $\endgroup$ – user35581 Mar 22 '18 at 17:40
0
$\begingroup$

With Wolfram Language in Mathematica.

data = {{"Banana", "Water", "Rice"},
        {"Rice", "Water"},
        {"Bread", "Banana", "Juice"}};

Get pairwise counts.

counts = Sort /@ Flatten[Subsets[#, {2}] & /@ data, 1] // Tally
{{{"Banana", "Water"}, 1}, {{"Banana", "Rice"}, 1}, 
 {{"Rice", "Water"}, 2}, {{"Banana", "Bread"}, 1}, 
 {{"Bread", "Juice"}, 1}, {{"Banana", "Juice"}, 1}}

Get indices for named ticks.

indices = Thread[# -> Range[Length@#]] &@Sort@DeleteDuplicates@Flatten[data]
{"Banana" -> 1, "Bread" -> 2, "Juice" -> 3, "Rice" -> 4, "Water" -> 5}

Plot with MatrixPlot using SparseArray. Could also use ArrayPlot.

MatrixPlot[
 SparseArray[Rule @@@ counts /. indices, ConstantArray[Length@indices, 2]],
 FrameTicks -> With[{t = {#2, #1} & @@@ indices}, {{t, None}, {t, None}}],
 PlotLegends -> Automatic
 ]

enter image description here

Note that it is upper-triangular.

Hope this helps.

| improve this answer | |
$\endgroup$
0
$\begingroup$

You can do this in python with the seaborn visualization library (built on top of matplotlib).

data = [
  ['Banana', 'Water', 'Rice'],
  ['Rice', 'Water'],
  ['Bread', 'Banana', 'Juice'],
]

# Pull out combinations
from itertools import combinations
data_pairs = []
for d in data:
    data_pairs += [list(sorted(x)) + [1] for x in combinations(d, 2)]
    # Add reverse as well (this will mirror the heatmap)
    data_pairs += [list(sorted(x))[::-1] + [1] for x in combinations(d, 2)]

# Shape into dataframe
import pandas as pd
df = pd.DataFrame(data_pairs)
df_zeros = pd.DataFrame([list(x) + [0] for x in combinations(df[[0, 1]].values.flatten(), 2)])
df = pd.concat((df, df_zeros))
df = df.groupby([0, 1])[2].sum().reset_index().pivot(0, 1, 2).fillna(0)

import seaborn as sns
from matplotlib.pyplot import plt
sns.heatmap(df, cmap='YlGnBu')
plt.show()

The final dataframe df looks like this:

enter image description here

and the resulting visualization is:

enter image description here

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.