I have some training data which I am using to build a Spark MLLib model which is in a Hive database. I am using simple linear regression models and the PySpark API.
I have a code set up to train this model every day to get the most up-to-date model. (the real-world use case is that I am predicting vehicle unloading times, and my model must always be recently trained since the characteristics of the vehicles and locations change over time.)
However, when I use my model for inference I want to do it from an existing Java codebase. I need fast inference for individual data points, not batch inference. So I need a lightweight low latency way of calculating inference.
One solution that I have found is to export the parameters of my MLLib model into PMML or another representation, and then re-implement the inference code in pure Java without any of the boilerplate that would come with Spark. So I have a function like this:
public static double predict(double[] parameters) {
double prediction = bias + weights[0] * parameters[0] + ....;
return prediction;
}
where the array weights are updated every day with values exported from a trained MLLib model.
However, this seems inefficient, since the logic of the model design is now reproduced unnecessarily in the Java code, and also restricts me to the kind of simple models that can be represented this way. For example, I could not do this for random forest regression.
Ideally, I would like a lightweight inference call to Spark MLLib within Java, without any of the overhead of Spark sessions, servers, APIs, URLs, etc.
Is there such a lightweight Spark Java function that I could use which allows the inference of a single instance? I don't imagine it being an uncommon situation that somebody needs the benefit of Hive and parallel processing for training, but just fast and simple inference with minimal overheads.
