# Regarding join in pyspark

Given a rdd in pyspark, $$A=[(0, 1), (1, 2), (1, 3), (3, 0)]$$

I am trying to find self join of A such that the second element of previous pair is equal to first element of next pair i.e.

$$result=[(0, 2), (0, 3), (1, 0)]$$

I tried taking cartesian but it is giving other possibilities.
Also I tried with reverse of the above and then joining but it is also giving many use cases.
Please let me know the correct approach to solve the above.

• I meant the logic behind this that shouldn't be like Cartesian product. I'm gonna delete my answer to make your post up. Aug 17, 2019 at 19:48
• Cartisian products gives you all the possible products while what you are aiming to get is based on transitivity among tuples in a set. This code can be a simple also time/energy consuming approach when there are many samples for this problem, but the logic behind is true. Time: 𝑂(𝑛2) O ( n 2 ) result = [] for i in A: for j in A: if i[1] == j[0]: print(i[0],j[1]) result.append((i[0],j[1])) There should be better ways, I will refresh the post If I find. Aug 17, 2019 at 19:48
• @FatemehAsgarinejad I got the answer by taking reverse of the above list and then using inner join of rdd to get all corresponding transitive pairs. After removing duplicates i.e. (b,a) and same edges (a,a) or (b,b) got the resulting rdd. Thanks for the help. Aug 18, 2019 at 17:55