6
$\begingroup$

I have a lot of sentences (500k) which looks like this:

"Penalty missed! Bad penalty by Felipe Brisola  - Riga FC -  shot with right foot is very close to the goal. Felipe Brisola should be disappointed."
"Penalty saved! Damir Kojasevic  - Sutjeska Niksic -  fails to capitalise on this great opportunity,  shot with right foot saved  in the centre of the goal."   
"Penalty saved! Stefan Panic  - Riga FC -  fails to capitalise on this great opportunity,  shot with right foot saved  in the centre of the goal."
"Penalty saved! Georgie Kelly  - Dundalk -  fails to capitalise on this great opportunity,  shot with right foot saved  in the centre of the goal."
"Penalty missed! Still  FC København 1, Crvena Zvezda 1. Marko Marin  - Crvena Zvezda -  hits the bar with a shot with right foot."

As you see, they are not really robotic, and after ending up writing 1500 lines of php code (with regex) and still being inconsistent, I decided to see my alternatives with machine learning.

What I am trying to achieve is:

For example this one:

"Penalty saved! Stefan Panic  - Riga FC -  fails to capitalise on this great opportunity,  shot with right foot saved  in the centre of the goal."

type => penalty
action => saved
reason => shot with right foot saved  in the centre of the goal
person => Stefan Panic

I stumbled upon spaCy and saw "Named Entity Recognition" and thought maybe I can use it for this purpose. Especially as I have huge training data.

I wanted to ask: Is spaCy's Named Entity Recognition is right for this task? If not, what should I try to learn for this task?

P.S: I know a little about python but nothing about ML

Improve this question
$\endgroup$

3 Answers 3

Reset to default
2
$\begingroup$

Named Entity Recognition (NER) would extract names of people, organizations and such. Example:

"Penalty missed! Bad penalty by <person>Felipe Brisola</person>  - <organization>Riga FC</organization> -  shot with right foot is very close to the goal. <person>Felipe Brisola</person> should be disappointed."

So it could be helpful for the "person" field, but probably not for the rest. Note that you could also train a system similar to NER in order to predict other fields, but it would require a good amount of annotated data and it's not sure to work well.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks for your input. $\endgroup$
    – senty
    Dec 1, 2019 at 0:06
0
$\begingroup$

You can use dependency parsing and POS tagging from spaCy here. This will help on 'Actions' tagging and with some additional brain storming, you should be able to train your model on the rest of statements as well.

Improve this answer
$\endgroup$
0
$\begingroup$

You can do the above task by making your own pattern for entity extraction- https://spacy.io/usage/rule-based-matching

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.