1
$\begingroup$

I have many different strings of text. These strings of text are labels for particular things. But these labels are sloppy, sometimes one label is used for many different things. For example:

"Brown foxes edition 1999 series 1-6 EDI"

"Light [old] seasons 1,2,3,4 other gibberish"

I would like to answer the question: "If the label contains a series, does that serie contain the value N?" For the examples above 6 would be included in the first one, but not the second.

Initially I thought of using regexes but that quickly grew out of hand. Digits appear everywhere, people can get very creative with separators and the location of the series in the label is not fixed. There are many different ways the labels denote series.

What I can do, however, generate labels with series. I'll just grab a bunch of separators, a start and an end digit and iterate. This gives me a nice labeled training set.

Naive Bayes comes to mind for this problem but I'm not sure what good features would be.

update

Let me try to clarify. Given a label and given a number determine if that number is contained within that label.

For example: Given I'm looking for season 2 (the number). Does "Pioneer One 2011 seasons 1-3" contain season 2?

Improve this question
$\endgroup$
4
  • $\begingroup$ Machine learning on algorithm-generated data sounds wrong. For basic functionality I would pursue the regular expressions approach instead. $\endgroup$
    – Valentas
    Commented Sep 30, 2015 at 8:05
  • $\begingroup$ I have difficulties in getting your question, can you be more explicit? $\endgroup$
    – user10169
    Commented Sep 30, 2015 at 8:05
  • $\begingroup$ @Valentas I'm under the impression that happens all the time with vision related machine learning. Slight alterations are made to the training data to inflate their numbers. $\endgroup$
    – harm
    Commented Sep 30, 2015 at 11:03
  • $\begingroup$ Could you provide a representative sample of the data with all of the known series types you currently have , then perhaps people could help you define a detection method or pattern matching algorithm ? $\endgroup$
    – image_doctor
    Commented Oct 2, 2015 at 5:59

2 Answers 2

Reset to default
1
$\begingroup$

So there are many ways to denote a series. How are you going to parse the series down to determine the values if you don't know the format?

Determining if the label has a series does not get you to the specific numbers in the series.

2,3,5,7 parses out to 4 numbers

Is 6 in 1996? I assume that is one number and 1996 != 6

"55,56,57" is series with a 6 but but not the number 6

Does 7-9 parse out to 2 numbers or 3 number
Is 6 in 7-9?
If 6 is in 7-9 identifying that as a series does not answer that question.

How many ways can there be represent a series that regex got out of hand? For each format of series you also need to parse the values. You need to know the format of the series to parse out the numbers. I would have a set of regex mapped to set of parsers.

Maybe use machine learning to identity new series formats but you are still going to need to parse out the series.

Improve this answer
$\endgroup$
0
$\begingroup$

I think your question is badly formulated, but if I understand it you are trying to separate numbers that are "good" numbers (here the number of the tv show season) from "bad" numbers (for example if the text contained the year "1996" and you were looking for "6", or a 6 in any other context).

Naive Bayes might do the trick, maybe using the neighbourhood of your number as a feature set with a predetermined window? I.e. take the X tokens on both sides of your number as they appear in the text and use them as your feature set. You could even enrich it with their part-of-speech and other contextual evidence.

Improve this answer
$\endgroup$
1
  • $\begingroup$ I'm sorry, I tried to be too generic it seems. But you understood me nonetheless mostly right. I'll look into this part-of-speech. $\endgroup$
    – harm
    Commented Sep 30, 2015 at 11:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.