3
$\begingroup$

I'm looking for a stemmer/lemmatizer for Polish language, preferably in Python. What would you recommend?

I have a list of ingredients in a recipe. Plural forms are inflected differently, depending on the counter, e.g.: for tomatoes

5 pomidorów

2 pomidory

1 pomidor

I want my parser to recognize all those ingredients as for one product, hence the need for stemming.

| improve this question | | | | |
$\endgroup$
5
$\begingroup$

There is bunch of lemmatization solutions for polish language. One of the best implementation is in polish morphosyntactic analyser, which you can download here.

It has bindings to python, but you have to install them manually. It is "morphosyntactic analyser" which means, that you get all possible lemmas for a given word. If you want also disambiguation, you can use this tool which is provided as docker container.

My team is currently working on implementing polish language support for Spacy, and we will cover lemmatization, so tool with good integration and support will be available soon.

| improve this answer | | | | |
$\endgroup$
  • 1
    $\begingroup$ amazing! What tools are you using for that? Is there any web page about the project? $\endgroup$ – MkL Oct 30 '18 at 19:46
  • 1
    $\begingroup$ @MkL We have just started, and we don't have website about, but if you are interested follow spacy pull requests, i think our PR with tokenizer will be ready next week, and lemmatizer in 2 weeks. $\endgroup$ – Gizio Nov 3 '18 at 17:20
  • $\begingroup$ @Gizio Will you have any plans to implement full model for polish language in spacy? $\endgroup$ – Michał Orliński Feb 26 '19 at 10:40
  • $\begingroup$ Thanks. Do you have plans to include lemma disambiguation into SpaCy? Using docker container is a way to start but does not sound very scallable. $\endgroup$ – dzieciou Jul 17 '19 at 16:22
1
$\begingroup$

I have ported Stempel Stemmer from Java to Python: https://github.com/dzieciou/pystempel and improved its effectiveness by pretraining stemming tables on Polimorf dictionary from Morfeusz.

It appears slightly faster than wrapping Java implementation with pyjini but the main point is that I do not have to mix multiple languages in one project, avoiding debugging across Python and Java.

| improve this answer | | | | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.