2
$\begingroup$

Good day, everyone!

I have a problem where I have to program something like this:

I have some arbitrary number of events, let's call then Event A, B, C, D, E, F, ... and so on.

Now they occur in well defined sequences, just like below:

[A, A, C, A, A, A, F, B, A]

That's just a random sample, I have a data of thousands of such event sequences, and there are going to be 9 events in every sequence.

What I'm trying to do is to calculate the probability of each individual event happening. Like:

  • what's the probability of event A occurring first
  • what's the probability of event B occurring, given that the previous event was A
  • what's the probability of event C occurring, given that the previous two events were A and B

So far I've gone through the maths behind conditional/joint probabilities, markov chains, but I can't get to come up with the programming part. Are there any leads to where I can get my hands on some framework, library, or a tool which will help me run the simulation with all of my sequences, and give any finalised probabilities?

Thanks!

P.S: For those who are aware of MLB, I'm just trying to find the probabilities of certain scores being made in all the 9 innings based on past scores. If I calculate the simple probability of a score being made, I get a very high probability of zero, which is not the case. I only have the data for the box scores.

$\endgroup$
  • 1
    $\begingroup$ Sounds like textbook RNN territory to me. Checkout this article, part of which details an RNN predicting the probabilities of the next letter in a sequence being one of H, E, L or O. For example, if letter one is H and letter 2 is E, the model will be pretty confident letters three and 4 should both be L and letter 5 should be O. karpathy.github.io/2015/05/21/rnn-effectiveness $\endgroup$ – Dan Scally Aug 16 '19 at 12:13
  • 1
    $\begingroup$ Are you looking for Apriori Algorithm? $\endgroup$ – Mr. Sigma. Aug 16 '19 at 14:58
  • $\begingroup$ @Dan Sally, thanks, I'll go through this. $\endgroup$ – Syed Ali Hamza Aug 16 '19 at 15:00
  • $\begingroup$ @Mr.Sigma, I'm not sure if this problem can be mapped onto Apriori, because as per my understanding, Apriori might not consider the order of the sequence, but only the counts of certain events happening in the same sequence. $\endgroup$ – Syed Ali Hamza Aug 16 '19 at 15:02
1
$\begingroup$

My answer is inspired by this question in Code Review SE

import random
import string
from collections import Counter

def test_data(n, m, choices):
    return [[random.choice(choices) for _ in range(m)] for _ in range(n)]

data = test_data(50000, 9, ["A", "B", "C", "D", "E", "F"])

def subsequence_counts(sequences):
    return Counter([seq for seq in map(''.join, sequences)])

counter = subsequence_counts(data)

print(counter['BCEEECFAF'])

What the above code does:

1) The test_data function, creates a n lists with m items using elements of the choices parameters.

In specific, we create 50000 lists with 9 items length using the capital letters A, B, C, D, E, F

This is just for testing the code

2) The function subsequence_counts uses a list comprehension and a Counter to keep track of every single combination of the sublists we have.

3) Finally, you can access the Counter and get the frequency of any event you want.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks, this is something which looks a bit closer to what I have, but I still am looking for something which can return me the individual probabilities of each event within the list, based on the conditional probabilities of the previous events occurring in the same sequence. $\endgroup$ – Syed Ali Hamza Aug 17 '19 at 17:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.