3
$\begingroup$

I was curious about the ANOVA RBF kernel provided by kernlab package available in R.

I tested it with a numeric dataSet of 34 input variables and one output variable. For each variable I have 700 different values. Comparing with other kernels, I got very bad results with this kernel. For example using the simple RBF kernel I could predict with 0,88 R2 however with the anova RBF I could only get 0,33 R2. I thought that ANOVA RBF would be a very good kernel. Any thoughts? Thanks

The code is as follows:

set.seed(100) #use the same seed to train different models
svrFitanovaacv <- train(R ~ .,
                       data = trainSet,
                       method = SVManova,
                       preProc = c("center", "scale"),
                       trControl = ctrl, tuneLength = 10) #By default, RMSE and R2 are computed for regression (in all cases, selects the tunning and cross-val model with best value) , metric = "ROC"

define custom model in caret package:

library(caret)
#RBF ANOVA KERNEL
SVManova <- list(type = "Regression", library = "kernlab", loop = NULL)
prmanova <- data.frame(parameter = c("C", "sigma", "degree", "epsilon"),
                     class = rep("numeric", 4),
                     label = c("Cost", "Sigma", "Degree", "Epsilon"))
SVManova$parameters <- prmanova
svmGridanova <- function(x, y, len = NULL) {
library(kernlab)
sigmas <- sigest(as.matrix(x), na.action = na.omit, scaled = TRUE, frac = 1)
expand.grid(sigma = mean(sigmas[-2]), epsilon = 0.000001,
            C = 2^(-40:len), degree = 1:2) # len = tuneLength in train
}
SVManova$grid <- svmGridanova
svmFitanova <- function(x, y, wts, param, lev, last, weights, classProbs, ...) {
  ksvm(x = as.matrix(x), y = y,
       kernel = "anovadot",
       kpar = list(sigma = param$sigma, degree = param$degree),
       C = param$C, epsilon = param$epsilon,
       prob.model = classProbs,
       ...) #default type = "eps-svr"
}
SVManova$fit <- svmFitanova
svmPredanova <- function(modelFit, newdata, preProc = NULL, submodels = NULL)
  predict(modelFit, newdata)
SVManova$predict <- svmPredanova
svmProb <- function(modelFit, newdata, preProc = NULL, submodels = NULL)
  predict(modelFit, newdata, type="probabilities")
SVManova$prob <- svmProb
svmSortanova <- function(x) x[order(x$C), ]
SVManova$sort <- svmSortanova

load data:

dataA2<-read.csv("C:/results/A2.txt",header = TRUE, 
                             blank.lines.skip = TRUE,sep = ",")
set.seed(1)
inTrainSet <- createDataPartition(dataA2$R, p = 0.75, list = FALSE) #[[1]]
trainSet <- dataA2[inTrainSet,]
testSet <- dataA2[-inTrainSet,]
#-----------------------------------------------------------------------------
#K-folds resampling method for fitting svr
ctrl <- trainControl(method = "repeatedcv", number = 10, repeats = 10,
                     allowParallel = TRUE) #10 separate 10-fold cross-validations

link to data:

wuala.com/jpcgandre/Documents/Data%20SVR/?key=BOD9NTINzRHG
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.