0
$\begingroup$

I want to do a very simple cross validation using LogisticRegression. Here is my code:

logreg = LogisticRegression(labelCol = "churn", featuresCol = "features") 

pipeline = Pipeline(stages = [logreg])
paramGrid = ParamGridBuilder().addGrid(logreg.regParam, [.1, .01]).build()

crossval = CrossValidator(
    estimator = pipeline,
    estimatorParamMaps = paramGrid,
    evaluator = BinaryClassificationEvaluator(),
    numFolds = 2)

bestLogReg = crossval.fit(df_train)

When I run this, I get the following error on bestLogReg = crossval.fit(df_train):

IllegalArgumentException: label does not exist. Available: features, churn, CrossValidator_764038c00edc_rand, rawPrediction, probability, prediction

Here is my df_train dataset's schema:

root
 |-- features: vector (nullable = true)
 |-- churn: integer (nullable = true)

I have fit this to a LogisticRegression before and it predicts fine.

Can you help me figure out what I did wrong?

$\endgroup$

1 Answer 1

2
$\begingroup$

For some reason in cross validation we also need to set the label column of the evaluator (even tho it's already set for the estimator. So all you need to do is change BinaryClassificationEvaluator() into BinaryClassificationEvaluator().setLabelCol("churn") where "churn" is the name of your target variable.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.