I am creating a relatively large hobby project in Scala that needs a few ml algorithms for text classification into topics. My dataset is not huge, it is < 500,000 items with dimensionality 5 (2 dimensions are free form text).
From what I've started with on Spark, it is heavily geared toward distributed computing and other production related concerns but it has a nice ml library. Is it worth using Spark if I only plan to run my project on a local dev machine level or is there a more appropriate library out there for me in Scala? Something like scikit-learn, but in Scala.