I'm very new to machine learning (and statistics) and I'm struggling with some basics (I'm using R as my primary environment).
I have a lexicon with words and all of their forms (in different genders, persons - for a Slavic language). So each word with its word forms is one training sample. My training attributes are last character, last two characters, last three characters of each word form. So each training sample contains from around 100 categorical attributes. I've got 10000 samples. One attribute has around 1700 levels, one around 1200, 5 around 500, others have less than 100 levels.
I have used DBSCAN (and gower distance) to cluster similars words to the same cluster. This step was pretty successful. I get (depending whether is a noun, verb or adjective) from 40-100 clusters.
Now here comes the part where I don't know how to continue. If I get a new (unknown) word I want to predict to which cluster it belongs. Sometimes for this new word I only know its base form, sometimes I also part of its word forms and sometimes I know all of its word forms.
So this should be some kind of classification with multiple classes. Question: which algorithm should I use? I've tried to use multinom
(in R) but I get an error of too many weights. After I increased MaxNWts
I get an error:
weights: 641676 (633954 variable)
Error: cannot allocate vector of size 396 Kb
Where should I continue? Clean the data? Use a different algorithm?
Also, around 15 attributes have only 1 level or NA. How does this impact to clustering (and classification)? Is it safe to remove those attributes?
Many thanks in advance!