5
$\begingroup$

I have a lot of sentences (500k) which looks like this:

"Penalty missed! Bad penalty by Felipe Brisola  - Riga FC -  shot with right foot is very close to the goal. Felipe Brisola should be disappointed."
"Penalty saved! Damir Kojasevic  - Sutjeska Niksic -  fails to capitalise on this great opportunity,  shot with right foot saved  in the centre of the goal."   
"Penalty saved! Stefan Panic  - Riga FC -  fails to capitalise on this great opportunity,  shot with right foot saved  in the centre of the goal."
"Penalty saved! Georgie Kelly  - Dundalk -  fails to capitalise on this great opportunity,  shot with right foot saved  in the centre of the goal."
"Penalty missed! Still  FC København 1, Crvena Zvezda 1. Marko Marin  - Crvena Zvezda -  hits the bar with a shot with right foot."

As you see, they are not really robotic, and after ending up writing 1500 lines of php code (with regex) and still being inconsistent, I decided to see my alternatives with machine learning.

What I am trying to achieve is:

For example this one:

"Penalty saved! Stefan Panic  - Riga FC -  fails to capitalise on this great opportunity,  shot with right foot saved  in the centre of the goal."

type => penalty
action => saved
reason => shot with right foot saved  in the centre of the goal
person => Stefan Panic

I stumbled upon spaCy and saw "Named Entity Recognition" and thought maybe I can use it for this purpose. Especially as I have huge training data.

I wanted to ask: Is spaCy's Named Entity Recognition is right for this task? If not, what should I try to learn for this task?

P.S: I know a little about python but nothing about ML

$\endgroup$
2
$\begingroup$

Named Entity Recognition (NER) would extract names of people, organizations and such. Example:

"Penalty missed! Bad penalty by <person>Felipe Brisola</person>  - <organization>Riga FC</organization> -  shot with right foot is very close to the goal. <person>Felipe Brisola</person> should be disappointed."

So it could be helpful for the "person" field, but probably not for the rest. Note that you could also train a system similar to NER in order to predict other fields, but it would require a good amount of annotated data and it's not sure to work well.

$\endgroup$
  • $\begingroup$ Thanks for your input. $\endgroup$ – senty Dec 1 '19 at 0:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.